2026-02-17T05:41:55.117 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-17T05:41:55.133 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T05:41:55.215 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065 branch: wip-bharath19-testing-2026-02-13-0903-squid description: rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '51065' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath19-testing-2026-02-13-0903-squid ceph: conf: client: debug monc: 20 debug ms: 1 debug objecter: 20 debug rados: 20 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async osd heartbeat grace: 160 osd op thread timeout: 150 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osd crush smoke test: false mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 mon warn on pool no app: false paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 1 debug bluestore: 1 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd class default list: '*' osd class load list: '*' osd client watch timeout: 240 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd fast shutdown: false osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - reached quota - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(SLOW_OPS\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(OBJECT_MISPLACED\) - slow request - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(SLOW_OPS\) - slow request - OSD bench result sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e stop-daemons-timeout: 1200 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: debuginfo: true flavor: default sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath19-testing-2026-02-13-0903-squid sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 3603 sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath19-testing-2026-02-13-0903-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e targets: trial016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCOjwa/S7SIgHS0J6Rc+z8xdczVwmSiw674yoGf3MnN2u2eE72QcKYWAMwnIBoXYZbuW+iU9UR4UPtxbDHHms8= trial128.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAv6HpIRX32j0pnlKdV17Nlu4rkKitJRYQCcVQiuw7qymgmi010/TejjZJfvSg3e31GxB3swfyVvGG0hUjWXckA= trial148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1B6qsZT1vfivcc0xwAdKN6dsBIR3T4QrW36iADtgSj9tO05NUIGNn4g71dyyjoHBO9o5NhHeUtWlHJJ1TstW4= trial164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB985FXmImc249j7rRFpaYT3Ole+BKLL89gUND7dwdAO3d5hgUn5IviwpEAZldK4XwzgGHL3KtnmH0aakzKaauQ= tasks: - install: null - ceph: null - workunit: clients: client.0: - rados/test.sh env: ALLOW_TIMEOUTS: '1' timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - '-- Don''t run this test on tiny clusters if py_len(yaml.roles) < 4 then reject() end ' teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-17_04:03:08 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-17T05:41:55.215 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa; will attempt to use it 2026-02-17T05:41:55.215 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks 2026-02-17T05:41:55.215 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-17T05:41:55.220 INFO:teuthology.task.internal:Checking packages... 2026-02-17T05:41:55.220 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e' 2026-02-17T05:41:55.220 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-17T05:41:55.220 INFO:teuthology.packaging:ref: None 2026-02-17T05:41:55.220 INFO:teuthology.packaging:tag: None 2026-02-17T05:41:55.220 INFO:teuthology.packaging:branch: wip-bharath19-testing-2026-02-13-0903-squid 2026-02-17T05:41:55.220 INFO:teuthology.packaging:sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:41:55.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath19-testing-2026-02-13-0903-squid 2026-02-17T05:41:55.336 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-705.g04cf6dd4 2026-02-17T05:41:55.338 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-17T05:41:55.349 INFO:teuthology.task.internal:no buildpackages task found 2026-02-17T05:41:55.349 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-17T05:41:55.363 INFO:teuthology.task.internal:Saving configuration 2026-02-17T05:41:55.375 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-17T05:41:55.388 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-17T05:41:55.461 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial016.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 05:38:23.323651', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCOjwa/S7SIgHS0J6Rc+z8xdczVwmSiw674yoGf3MnN2u2eE72QcKYWAMwnIBoXYZbuW+iU9UR4UPtxbDHHms8='} 2026-02-17T05:41:55.517 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial128.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 05:38:23.322217', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAv6HpIRX32j0pnlKdV17Nlu4rkKitJRYQCcVQiuw7qymgmi010/TejjZJfvSg3e31GxB3swfyVvGG0hUjWXckA='} 2026-02-17T05:41:55.574 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial148.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 05:38:23.323272', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1B6qsZT1vfivcc0xwAdKN6dsBIR3T4QrW36iADtgSj9tO05NUIGNn4g71dyyjoHBO9o5NhHeUtWlHJJ1TstW4='} 2026-02-17T05:41:55.630 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial164.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 05:38:23.322867', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB985FXmImc249j7rRFpaYT3Ole+BKLL89gUND7dwdAO3d5hgUn5IviwpEAZldK4XwzgGHL3KtnmH0aakzKaauQ='} 2026-02-17T05:41:55.630 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-17T05:41:55.642 INFO:teuthology.task.internal:roles: ubuntu@trial016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2026-02-17T05:41:55.642 INFO:teuthology.task.internal:roles: ubuntu@trial128.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2026-02-17T05:41:55.642 INFO:teuthology.task.internal:roles: ubuntu@trial148.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2026-02-17T05:41:55.642 INFO:teuthology.task.internal:roles: ubuntu@trial164.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2026-02-17T05:41:55.642 INFO:teuthology.run_tasks:Running task console_log... 2026-02-17T05:41:55.894 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x76b05f596020>, signals=[15]) 2026-02-17T05:41:55.894 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-17T05:41:55.905 INFO:teuthology.task.internal:Opening connections... 2026-02-17T05:41:55.906 DEBUG:teuthology.task.internal:connecting to ubuntu@trial016.front.sepia.ceph.com 2026-02-17T05:41:55.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:41:55.967 DEBUG:teuthology.task.internal:connecting to ubuntu@trial128.front.sepia.ceph.com 2026-02-17T05:41:55.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial128.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:41:56.030 DEBUG:teuthology.task.internal:connecting to ubuntu@trial148.front.sepia.ceph.com 2026-02-17T05:41:56.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:41:56.099 DEBUG:teuthology.task.internal:connecting to ubuntu@trial164.front.sepia.ceph.com 2026-02-17T05:41:56.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:41:56.161 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-17T05:41:56.173 DEBUG:teuthology.orchestra.run.trial016:> uname -m 2026-02-17T05:41:56.188 INFO:teuthology.orchestra.run.trial016.stdout:x86_64 2026-02-17T05:41:56.189 DEBUG:teuthology.orchestra.run.trial016:> cat /etc/os-release 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:NAME="CentOS Stream" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:VERSION="9" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:ID="centos" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:ID_LIKE="rhel fedora" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:VERSION_ID="9" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:PLATFORM_ID="platform:el9" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:ANSI_COLOR="0;31" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:LOGO="fedora-logo-icon" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:HOME_URL="https://centos.org/" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-17T05:41:56.244 INFO:teuthology.orchestra.run.trial016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-17T05:41:56.245 INFO:teuthology.lock.ops:Updating trial016.front.sepia.ceph.com on lock server 2026-02-17T05:41:56.314 DEBUG:teuthology.orchestra.run.trial128:> uname -m 2026-02-17T05:41:56.331 INFO:teuthology.orchestra.run.trial128.stdout:x86_64 2026-02-17T05:41:56.331 DEBUG:teuthology.orchestra.run.trial128:> cat /etc/os-release 2026-02-17T05:41:56.386 INFO:teuthology.orchestra.run.trial128.stdout:NAME="CentOS Stream" 2026-02-17T05:41:56.386 INFO:teuthology.orchestra.run.trial128.stdout:VERSION="9" 2026-02-17T05:41:56.386 INFO:teuthology.orchestra.run.trial128.stdout:ID="centos" 2026-02-17T05:41:56.386 INFO:teuthology.orchestra.run.trial128.stdout:ID_LIKE="rhel fedora" 2026-02-17T05:41:56.386 INFO:teuthology.orchestra.run.trial128.stdout:VERSION_ID="9" 2026-02-17T05:41:56.386 INFO:teuthology.orchestra.run.trial128.stdout:PLATFORM_ID="platform:el9" 2026-02-17T05:41:56.386 INFO:teuthology.orchestra.run.trial128.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-17T05:41:56.386 INFO:teuthology.orchestra.run.trial128.stdout:ANSI_COLOR="0;31" 2026-02-17T05:41:56.387 INFO:teuthology.orchestra.run.trial128.stdout:LOGO="fedora-logo-icon" 2026-02-17T05:41:56.387 INFO:teuthology.orchestra.run.trial128.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-17T05:41:56.387 INFO:teuthology.orchestra.run.trial128.stdout:HOME_URL="https://centos.org/" 2026-02-17T05:41:56.387 INFO:teuthology.orchestra.run.trial128.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-17T05:41:56.387 INFO:teuthology.orchestra.run.trial128.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-17T05:41:56.387 INFO:teuthology.orchestra.run.trial128.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-17T05:41:56.387 INFO:teuthology.lock.ops:Updating trial128.front.sepia.ceph.com on lock server 2026-02-17T05:41:56.460 DEBUG:teuthology.orchestra.run.trial148:> uname -m 2026-02-17T05:41:56.476 INFO:teuthology.orchestra.run.trial148.stdout:x86_64 2026-02-17T05:41:56.476 DEBUG:teuthology.orchestra.run.trial148:> cat /etc/os-release 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:NAME="CentOS Stream" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:VERSION="9" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:ID="centos" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:ID_LIKE="rhel fedora" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:VERSION_ID="9" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:PLATFORM_ID="platform:el9" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:ANSI_COLOR="0;31" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:LOGO="fedora-logo-icon" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:HOME_URL="https://centos.org/" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-17T05:41:56.531 INFO:teuthology.orchestra.run.trial148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-17T05:41:56.531 INFO:teuthology.lock.ops:Updating trial148.front.sepia.ceph.com on lock server 2026-02-17T05:41:56.591 DEBUG:teuthology.orchestra.run.trial164:> uname -m 2026-02-17T05:41:56.606 INFO:teuthology.orchestra.run.trial164.stdout:x86_64 2026-02-17T05:41:56.606 DEBUG:teuthology.orchestra.run.trial164:> cat /etc/os-release 2026-02-17T05:41:56.661 INFO:teuthology.orchestra.run.trial164.stdout:NAME="CentOS Stream" 2026-02-17T05:41:56.661 INFO:teuthology.orchestra.run.trial164.stdout:VERSION="9" 2026-02-17T05:41:56.661 INFO:teuthology.orchestra.run.trial164.stdout:ID="centos" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:ID_LIKE="rhel fedora" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:VERSION_ID="9" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:PLATFORM_ID="platform:el9" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:ANSI_COLOR="0;31" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:LOGO="fedora-logo-icon" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:HOME_URL="https://centos.org/" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-17T05:41:56.662 INFO:teuthology.orchestra.run.trial164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-17T05:41:56.662 INFO:teuthology.lock.ops:Updating trial164.front.sepia.ceph.com on lock server 2026-02-17T05:41:56.726 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-17T05:41:56.740 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-17T05:41:56.752 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-17T05:41:56.752 DEBUG:teuthology.orchestra.run.trial016:> test '!' -e /home/ubuntu/cephtest 2026-02-17T05:41:56.753 DEBUG:teuthology.orchestra.run.trial128:> test '!' -e /home/ubuntu/cephtest 2026-02-17T05:41:56.758 DEBUG:teuthology.orchestra.run.trial148:> test '!' -e /home/ubuntu/cephtest 2026-02-17T05:41:56.760 DEBUG:teuthology.orchestra.run.trial164:> test '!' -e /home/ubuntu/cephtest 2026-02-17T05:41:56.776 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-17T05:41:56.787 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-17T05:41:56.788 DEBUG:teuthology.orchestra.run.trial016:> test -z $(ls -A /var/lib/ceph) 2026-02-17T05:41:56.824 DEBUG:teuthology.orchestra.run.trial128:> test -z $(ls -A /var/lib/ceph) 2026-02-17T05:41:56.830 DEBUG:teuthology.orchestra.run.trial148:> test -z $(ls -A /var/lib/ceph) 2026-02-17T05:41:56.837 DEBUG:teuthology.orchestra.run.trial164:> test -z $(ls -A /var/lib/ceph) 2026-02-17T05:41:56.853 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-17T05:41:57.102 INFO:teuthology.run_tasks:Running task kernel... 2026-02-17T05:41:57.123 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-17T05:41:57.123 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-17T05:41:57.123 DEBUG:teuthology.orchestra.run.trial016:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T05:41:57.124 DEBUG:teuthology.orchestra.run.trial128:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T05:41:57.124 DEBUG:teuthology.orchestra.run.trial148:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T05:41:57.124 DEBUG:teuthology.orchestra.run.trial164:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T05:41:57.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:41:57.138 DEBUG:teuthology.orchestra.run.trial164:> uname -r 2026-02-17T05:41:57.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:41:57.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:41:57.139 DEBUG:teuthology.orchestra.run.trial128:> uname -r 2026-02-17T05:41:57.139 DEBUG:teuthology.orchestra.run.trial148:> uname -r 2026-02-17T05:41:57.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:41:57.139 DEBUG:teuthology.orchestra.run.trial016:> uname -r 2026-02-17T05:41:57.194 INFO:teuthology.orchestra.run.trial164.stdout:5.14.0-645.el9.x86_64 2026-02-17T05:41:57.194 INFO:teuthology.task.kernel:Running kernel on trial164: 5.14.0-645.el9.x86_64 2026-02-17T05:41:57.194 DEBUG:teuthology.orchestra.run.trial164:> sudo yum install -y kernel 2026-02-17T05:41:57.195 INFO:teuthology.orchestra.run.trial016.stdout:5.14.0-645.el9.x86_64 2026-02-17T05:41:57.195 INFO:teuthology.orchestra.run.trial148.stdout:5.14.0-645.el9.x86_64 2026-02-17T05:41:57.195 INFO:teuthology.task.kernel:Running kernel on trial016: 5.14.0-645.el9.x86_64 2026-02-17T05:41:57.195 DEBUG:teuthology.orchestra.run.trial016:> sudo yum install -y kernel 2026-02-17T05:41:57.195 INFO:teuthology.task.kernel:Running kernel on trial148: 5.14.0-645.el9.x86_64 2026-02-17T05:41:57.195 DEBUG:teuthology.orchestra.run.trial148:> sudo yum install -y kernel 2026-02-17T05:41:57.196 INFO:teuthology.orchestra.run.trial128.stdout:5.14.0-645.el9.x86_64 2026-02-17T05:41:57.196 INFO:teuthology.task.kernel:Running kernel on trial128: 5.14.0-645.el9.x86_64 2026-02-17T05:41:57.196 DEBUG:teuthology.orchestra.run.trial128:> sudo yum install -y kernel 2026-02-17T05:41:57.763 INFO:teuthology.orchestra.run.trial148.stdout:CentOS Stream 9 - BaseOS 43 MB/s | 8.9 MB 00:00 2026-02-17T05:41:58.911 INFO:teuthology.orchestra.run.trial164.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.9 MB 00:01 2026-02-17T05:41:58.926 INFO:teuthology.orchestra.run.trial128.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.9 MB 00:01 2026-02-17T05:42:02.204 INFO:teuthology.orchestra.run.trial164.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 27 MB 00:02 2026-02-17T05:42:02.530 INFO:teuthology.orchestra.run.trial128.stdout:CentOS Stream 9 - AppStream 8.7 MB/s | 27 MB 00:03 2026-02-17T05:42:04.799 INFO:teuthology.orchestra.run.trial164.stdout:CentOS Stream 9 - CRB 33 MB/s | 7.8 MB 00:00 2026-02-17T05:42:05.594 INFO:teuthology.orchestra.run.trial164.stdout:CentOS Stream 9 - Extras packages 258 kB/s | 20 kB 00:00 2026-02-17T05:42:05.870 INFO:teuthology.orchestra.run.trial164.stdout:Extra Packages for Enterprise Linux 96 MB/s | 20 MB 00:00 2026-02-17T05:42:06.672 INFO:teuthology.orchestra.run.trial016.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.9 MB 00:09 2026-02-17T05:42:09.016 INFO:teuthology.orchestra.run.trial164.stdout:lab-extras 1.2 MB/s | 50 kB 00:00 2026-02-17T05:42:09.341 INFO:teuthology.orchestra.run.trial128.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 7.8 MB 00:04 2026-02-17T05:42:09.670 INFO:teuthology.orchestra.run.trial016.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-17T05:42:10.031 INFO:teuthology.orchestra.run.trial164.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-17T05:42:10.051 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:42:10.051 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:42:10.051 INFO:teuthology.orchestra.run.trial164.stdout: Package Architecture Version Repository Size 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout:Installing dependencies: 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout: kernel-core x86_64 5.14.0-681.el9 baseos 17 M 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout: kernel-modules x86_64 5.14.0-681.el9 baseos 40 M 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout: kernel-modules-core x86_64 5.14.0-681.el9 baseos 31 M 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout:Install 4 Packages 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 88 M 2026-02-17T05:42:10.052 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 135 M 2026-02-17T05:42:10.053 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:42:10.244 INFO:teuthology.orchestra.run.trial164.stdout:(1/4): kernel-5.14.0-681.el9.x86_64.rpm 5.9 MB/s | 803 kB 00:00 2026-02-17T05:42:10.295 INFO:teuthology.orchestra.run.trial128.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 20 kB 00:00 2026-02-17T05:42:10.398 INFO:teuthology.orchestra.run.trial164.stdout:(2/4): kernel-core-5.14.0-681.el9.x86_64.rpm 60 MB/s | 17 MB 00:00 2026-02-17T05:42:10.569 INFO:teuthology.orchestra.run.trial164.stdout:(3/4): kernel-modules-5.14.0-681.el9.x86_64.rpm 87 MB/s | 40 MB 00:00 2026-02-17T05:42:10.822 INFO:teuthology.orchestra.run.trial164.stdout:(4/4): kernel-modules-core-5.14.0-681.el9.x86_6 53 MB/s | 31 MB 00:00 2026-02-17T05:42:10.822 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:42:10.822 INFO:teuthology.orchestra.run.trial164.stdout:Total 115 MB/s | 88 MB 00:00 2026-02-17T05:42:11.008 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:42:11.211 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:42:11.211 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:42:11.556 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:42:11.556 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:42:11.607 INFO:teuthology.orchestra.run.trial148.stdout:CentOS Stream 9 - AppStream 2.0 MB/s | 27 MB 00:13 2026-02-17T05:42:12.387 INFO:teuthology.orchestra.run.trial128.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:02 2026-02-17T05:42:12.638 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:42:12.921 INFO:teuthology.orchestra.run.trial164.stdout: Installing : kernel-modules-core-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:42:12.940 INFO:teuthology.orchestra.run.trial164.stdout: Installing : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:42:13.078 INFO:teuthology.orchestra.run.trial016.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 7.8 MB 00:01 2026-02-17T05:42:14.732 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:42:14.761 INFO:teuthology.orchestra.run.trial164.stdout: Installing : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:42:15.066 INFO:teuthology.orchestra.run.trial016.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 20 kB 00:01 2026-02-17T05:42:15.602 INFO:teuthology.orchestra.run.trial128.stdout:lab-extras 1.3 MB/s | 50 kB 00:00 2026-02-17T05:42:16.612 INFO:teuthology.orchestra.run.trial128.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-17T05:42:16.631 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:42:16.633 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:42:16.633 INFO:teuthology.orchestra.run.trial128.stdout: Package Architecture Version Repository Size 2026-02-17T05:42:16.633 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:42:16.633 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:42:16.633 INFO:teuthology.orchestra.run.trial128.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-17T05:42:16.633 INFO:teuthology.orchestra.run.trial128.stdout:Installing dependencies: 2026-02-17T05:42:16.633 INFO:teuthology.orchestra.run.trial128.stdout: kernel-core x86_64 5.14.0-681.el9 baseos 17 M 2026-02-17T05:42:16.633 INFO:teuthology.orchestra.run.trial128.stdout: kernel-modules x86_64 5.14.0-681.el9 baseos 40 M 2026-02-17T05:42:16.634 INFO:teuthology.orchestra.run.trial128.stdout: kernel-modules-core x86_64 5.14.0-681.el9 baseos 31 M 2026-02-17T05:42:16.634 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:42:16.634 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:42:16.634 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:42:16.634 INFO:teuthology.orchestra.run.trial128.stdout:Install 4 Packages 2026-02-17T05:42:16.634 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:42:16.634 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 88 M 2026-02-17T05:42:16.634 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 135 M 2026-02-17T05:42:16.634 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:42:17.212 INFO:teuthology.orchestra.run.trial016.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 20 MB 00:02 2026-02-17T05:42:17.783 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:42:17.789 INFO:teuthology.orchestra.run.trial164.stdout: Installing : kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:18.479 INFO:teuthology.orchestra.run.trial128.stdout:(1/4): kernel-5.14.0-681.el9.x86_64.rpm 520 kB/s | 803 kB 00:01 2026-02-17T05:42:20.399 INFO:teuthology.orchestra.run.trial016.stdout:lab-extras 1.9 MB/s | 50 kB 00:00 2026-02-17T05:42:20.885 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:21.411 INFO:teuthology.orchestra.run.trial016.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-17T05:42:21.431 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:42:21.432 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:42:21.432 INFO:teuthology.orchestra.run.trial016.stdout: Package Architecture Version Repository Size 2026-02-17T05:42:21.432 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:42:21.432 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:42:21.432 INFO:teuthology.orchestra.run.trial016.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-17T05:42:21.432 INFO:teuthology.orchestra.run.trial016.stdout:Installing dependencies: 2026-02-17T05:42:21.432 INFO:teuthology.orchestra.run.trial016.stdout: kernel-core x86_64 5.14.0-681.el9 baseos 17 M 2026-02-17T05:42:21.432 INFO:teuthology.orchestra.run.trial016.stdout: kernel-modules x86_64 5.14.0-681.el9 baseos 40 M 2026-02-17T05:42:21.432 INFO:teuthology.orchestra.run.trial016.stdout: kernel-modules-core x86_64 5.14.0-681.el9 baseos 31 M 2026-02-17T05:42:21.433 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:42:21.433 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:42:21.433 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:42:21.433 INFO:teuthology.orchestra.run.trial016.stdout:Install 4 Packages 2026-02-17T05:42:21.433 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:42:21.433 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 88 M 2026-02-17T05:42:21.433 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 135 M 2026-02-17T05:42:21.433 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:42:21.571 INFO:teuthology.orchestra.run.trial016.stdout:(1/4): kernel-5.14.0-681.el9.x86_64.rpm 11 MB/s | 803 kB 00:00 2026-02-17T05:42:21.672 INFO:teuthology.orchestra.run.trial016.stdout:(2/4): kernel-core-5.14.0-681.el9.x86_64.rpm 97 MB/s | 17 MB 00:00 2026-02-17T05:42:21.762 INFO:teuthology.orchestra.run.trial016.stdout:(3/4): kernel-modules-core-5.14.0-681.el9.x86_6 161 MB/s | 31 MB 00:00 2026-02-17T05:42:21.816 INFO:teuthology.orchestra.run.trial016.stdout:(4/4): kernel-modules-5.14.0-681.el9.x86_64.rpm 124 MB/s | 40 MB 00:00 2026-02-17T05:42:21.816 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:42:21.816 INFO:teuthology.orchestra.run.trial016.stdout:Total 230 MB/s | 88 MB 00:00 2026-02-17T05:42:22.006 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:42:22.200 INFO:teuthology.orchestra.run.trial148.stdout:CentOS Stream 9 - CRB 968 kB/s | 7.8 MB 00:08 2026-02-17T05:42:22.219 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:42:22.219 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:42:22.568 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:42:22.568 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:42:22.764 INFO:teuthology.orchestra.run.trial128.stdout:(2/4): kernel-core-5.14.0-681.el9.x86_64.rpm 2.9 MB/s | 17 MB 00:05 2026-02-17T05:42:23.271 INFO:teuthology.orchestra.run.trial148.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 20 kB 00:00 2026-02-17T05:42:23.667 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:42:23.965 INFO:teuthology.orchestra.run.trial016.stdout: Installing : kernel-modules-core-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:42:23.984 INFO:teuthology.orchestra.run.trial016.stdout: Installing : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:42:25.331 INFO:teuthology.orchestra.run.trial128.stdout:(3/4): kernel-modules-core-5.14.0-681.el9.x86_6 4.5 MB/s | 31 MB 00:06 2026-02-17T05:42:25.634 INFO:teuthology.orchestra.run.trial128.stdout:(4/4): kernel-modules-5.14.0-681.el9.x86_64.rpm 4.6 MB/s | 40 MB 00:08 2026-02-17T05:42:25.635 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:42:25.636 INFO:teuthology.orchestra.run.trial128.stdout:Total 9.8 MB/s | 88 MB 00:09 2026-02-17T05:42:25.765 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:42:25.794 INFO:teuthology.orchestra.run.trial016.stdout: Installing : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:42:25.829 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:42:25.927 INFO:teuthology.orchestra.run.trial148.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 20 MB 00:02 2026-02-17T05:42:26.033 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:42:26.033 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:42:26.382 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:42:26.383 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:42:27.475 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:42:27.762 INFO:teuthology.orchestra.run.trial128.stdout: Installing : kernel-modules-core-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:42:27.781 INFO:teuthology.orchestra.run.trial128.stdout: Installing : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:42:28.815 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:42:28.822 INFO:teuthology.orchestra.run.trial016.stdout: Installing : kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:29.070 INFO:teuthology.orchestra.run.trial148.stdout:lab-extras 1.3 MB/s | 50 kB 00:00 2026-02-17T05:42:29.558 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:42:29.586 INFO:teuthology.orchestra.run.trial128.stdout: Installing : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:42:30.078 INFO:teuthology.orchestra.run.trial148.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-17T05:42:30.098 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:42:30.099 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:42:30.099 INFO:teuthology.orchestra.run.trial148.stdout: Package Architecture Version Repository Size 2026-02-17T05:42:30.099 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:42:30.100 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:42:30.100 INFO:teuthology.orchestra.run.trial148.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-17T05:42:30.100 INFO:teuthology.orchestra.run.trial148.stdout:Installing dependencies: 2026-02-17T05:42:30.100 INFO:teuthology.orchestra.run.trial148.stdout: kernel-core x86_64 5.14.0-681.el9 baseos 17 M 2026-02-17T05:42:30.100 INFO:teuthology.orchestra.run.trial148.stdout: kernel-modules x86_64 5.14.0-681.el9 baseos 40 M 2026-02-17T05:42:30.100 INFO:teuthology.orchestra.run.trial148.stdout: kernel-modules-core x86_64 5.14.0-681.el9 baseos 31 M 2026-02-17T05:42:30.100 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:42:30.100 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:42:30.100 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:42:30.100 INFO:teuthology.orchestra.run.trial148.stdout:Install 4 Packages 2026-02-17T05:42:30.101 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:42:30.101 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 88 M 2026-02-17T05:42:30.101 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 135 M 2026-02-17T05:42:30.101 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:42:31.567 INFO:teuthology.orchestra.run.trial148.stdout:(1/4): kernel-5.14.0-681.el9.x86_64.rpm 570 kB/s | 803 kB 00:01 2026-02-17T05:42:32.044 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:32.677 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:42:32.683 INFO:teuthology.orchestra.run.trial128.stdout: Installing : kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:34.265 INFO:teuthology.orchestra.run.trial148.stdout:(2/4): kernel-modules-5.14.0-681.el9.x86_64.rpm 9.6 MB/s | 40 MB 00:04 2026-02-17T05:42:34.306 INFO:teuthology.orchestra.run.trial148.stdout:(3/4): kernel-modules-core-5.14.0-681.el9.x86_6 11 MB/s | 31 MB 00:02 2026-02-17T05:42:35.779 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:35.997 INFO:teuthology.orchestra.run.trial148.stdout:(4/4): kernel-core-5.14.0-681.el9.x86_64.rpm 2.9 MB/s | 17 MB 00:05 2026-02-17T05:42:35.998 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:42:35.998 INFO:teuthology.orchestra.run.trial148.stdout:Total 15 MB/s | 88 MB 00:05 2026-02-17T05:42:36.188 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:42:36.385 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:42:36.386 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:42:36.730 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:42:36.731 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:42:37.813 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:42:38.095 INFO:teuthology.orchestra.run.trial148.stdout: Installing : kernel-modules-core-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:42:38.114 INFO:teuthology.orchestra.run.trial148.stdout: Installing : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:42:39.884 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:42:39.912 INFO:teuthology.orchestra.run.trial148.stdout: Installing : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:42:42.936 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:42:42.942 INFO:teuthology.orchestra.run.trial148.stdout: Installing : kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:46.036 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:51.182 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:51.204 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:51.376 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:51.377 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:42:51.377 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:42:51.377 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:42:51.429 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:42:51.429 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:42:51.429 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:42:51.429 INFO:teuthology.orchestra.run.trial164.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-17T05:42:51.429 INFO:teuthology.orchestra.run.trial164.stdout: kernel-core-5.14.0-681.el9.x86_64 2026-02-17T05:42:51.429 INFO:teuthology.orchestra.run.trial164.stdout: kernel-modules-5.14.0-681.el9.x86_64 2026-02-17T05:42:51.430 INFO:teuthology.orchestra.run.trial164.stdout: kernel-modules-core-5.14.0-681.el9.x86_64 2026-02-17T05:42:51.430 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:42:51.430 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:42:51.488 DEBUG:teuthology.orchestra.run.trial164:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:42:51.514 INFO:teuthology.orchestra.run.trial164.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:42:51.515 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:42:51.515 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-681.el9.x86_64 2026-02-17T05:42:51.515 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T05:42:51.516 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T05:42:51.516 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2026-02-17T05:42:51.516 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial164.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T05:42:51.516 DEBUG:teuthology.orchestra.run.trial164:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:42:51.579 INFO:teuthology.orchestra.run.trial164.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:42:51.580 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:42:51.580 DEBUG:teuthology.orchestra.run.trial164:> sudo rpm -qi grub2-tools 2026-02-17T05:42:51.650 INFO:teuthology.orchestra.run.trial164.stdout:Name : grub2-tools 2026-02-17T05:42:51.650 INFO:teuthology.orchestra.run.trial164.stdout:Epoch : 1 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:Version : 2.06 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:Release : 118.el9 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:Architecture: x86_64 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:Group : Unspecified 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:Size : 8393909 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:License : GPLv3+ 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-17T05:42:51.651 INFO:teuthology.orchestra.run.trial164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout:Packager : builder@centos.org 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout:Vendor : CentOS 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout:Summary : Support tools for GRUB. 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout:Description : 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout:hardware devices. 2026-02-17T05:42:51.652 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:42:51.653 INFO:teuthology.orchestra.run.trial164.stdout:This subpackage provides tools for support of all platforms. 2026-02-17T05:42:51.654 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-17T05:42:51.654 INFO:teuthology.task.kernel:Updating grub on trial164 to boot 5.14.0-681.el9.x86_64 2026-02-17T05:42:51.654 DEBUG:teuthology.orchestra.run.trial164:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-17T05:42:51.715 INFO:teuthology.orchestra.run.trial164.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:42:51.716 DEBUG:teuthology.orchestra.run.trial164:> sudo command -v grubby && echo yes || echo no 2026-02-17T05:42:51.779 INFO:teuthology.orchestra.run.trial164.stdout:/sbin/grubby 2026-02-17T05:42:51.780 INFO:teuthology.orchestra.run.trial164.stdout:yes 2026-02-17T05:42:51.780 DEBUG:teuthology.orchestra.run.trial164:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-681.el9.x86_64" | head -n 1 2026-02-17T05:42:51.844 INFO:teuthology.orchestra.run.trial164.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:42:51.844 DEBUG:teuthology.orchestra.run.trial164:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64 2026-02-17T05:42:51.918 DEBUG:teuthology.orchestra.run.trial164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-17T05:42:52.124 INFO:teuthology.orchestra.run.trial164.stderr:Generating grub configuration file ... 2026-02-17T05:42:52.187 INFO:teuthology.orchestra.run.trial164.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-17T05:42:52.193 INFO:teuthology.orchestra.run.trial164.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-17T05:42:52.438 INFO:teuthology.orchestra.run.trial164.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-17T05:42:52.444 INFO:teuthology.orchestra.run.trial164.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-17T05:42:52.475 INFO:teuthology.orchestra.run.trial164.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-17T05:42:52.481 INFO:teuthology.orchestra.run.trial164.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-17T05:42:52.505 INFO:teuthology.orchestra.run.trial164.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-17T05:42:52.512 INFO:teuthology.orchestra.run.trial164.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-17T05:42:52.593 INFO:teuthology.orchestra.run.trial164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T05:42:52.601 INFO:teuthology.orchestra.run.trial164.stderr:done 2026-02-17T05:42:52.603 DEBUG:teuthology.orchestra.run.trial164:> test -d /sys/firmware/efi 2026-02-17T05:42:52.620 DEBUG:teuthology.orchestra.run.trial164:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-17T05:42:52.685 INFO:teuthology.orchestra.run.trial164.stdout:/boot/efi/EFI/BOOT 2026-02-17T05:42:52.685 INFO:teuthology.orchestra.run.trial164.stdout:/boot/efi/EFI/centos 2026-02-17T05:42:52.686 DEBUG:teuthology.orchestra.run.trial164:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-17T05:42:52.750 INFO:teuthology.orchestra.run.trial164.stdout:no 2026-02-17T05:42:52.750 DEBUG:teuthology.orchestra.run.trial164:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-17T05:42:52.815 INFO:teuthology.orchestra.run.trial164.stdout:yes 2026-02-17T05:42:52.815 DEBUG:teuthology.orchestra.run.trial164:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-17T05:42:52.880 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial164 2026-02-17T05:42:52.880 DEBUG:teuthology.orchestra.run.trial164:> sudo bash -c ' 2026-02-17T05:42:52.880 DEBUG:teuthology.orchestra.run.trial164:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T05:42:52.880 DEBUG:teuthology.orchestra.run.trial164:> ' 2026-02-17T05:43:02.331 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:02.352 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:02.522 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:02.523 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:43:02.523 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:43:02.523 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:43:02.576 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:02.576 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:43:02.576 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:43:02.576 INFO:teuthology.orchestra.run.trial016.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-17T05:43:02.576 INFO:teuthology.orchestra.run.trial016.stdout: kernel-core-5.14.0-681.el9.x86_64 2026-02-17T05:43:02.576 INFO:teuthology.orchestra.run.trial016.stdout: kernel-modules-5.14.0-681.el9.x86_64 2026-02-17T05:43:02.576 INFO:teuthology.orchestra.run.trial016.stdout: kernel-modules-core-5.14.0-681.el9.x86_64 2026-02-17T05:43:02.577 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:43:02.577 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:43:02.637 DEBUG:teuthology.orchestra.run.trial016:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:43:02.663 INFO:teuthology.orchestra.run.trial016.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:43:02.664 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:43:02.664 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-681.el9.x86_64 2026-02-17T05:43:02.664 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T05:43:02.665 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T05:43:02.665 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-17T05:43:02.665 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial016.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T05:43:02.665 DEBUG:teuthology.orchestra.run.trial016:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:43:02.729 INFO:teuthology.orchestra.run.trial016.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:43:02.729 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:43:02.729 DEBUG:teuthology.orchestra.run.trial016:> sudo rpm -qi grub2-tools 2026-02-17T05:43:02.796 INFO:teuthology.orchestra.run.trial016.stdout:Name : grub2-tools 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:Epoch : 1 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:Version : 2.06 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:Release : 118.el9 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:Architecture: x86_64 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:Group : Unspecified 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:Size : 8393909 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:License : GPLv3+ 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-17T05:43:02.797 INFO:teuthology.orchestra.run.trial016.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout:Packager : builder@centos.org 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout:Vendor : CentOS 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout:Summary : Support tools for GRUB. 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout:Description : 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout:hardware devices. 2026-02-17T05:43:02.798 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:43:02.799 INFO:teuthology.orchestra.run.trial016.stdout:This subpackage provides tools for support of all platforms. 2026-02-17T05:43:02.800 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-17T05:43:02.800 INFO:teuthology.task.kernel:Updating grub on trial016 to boot 5.14.0-681.el9.x86_64 2026-02-17T05:43:02.800 DEBUG:teuthology.orchestra.run.trial016:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-17T05:43:02.863 INFO:teuthology.orchestra.run.trial016.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:43:02.865 DEBUG:teuthology.orchestra.run.trial016:> sudo command -v grubby && echo yes || echo no 2026-02-17T05:43:02.930 INFO:teuthology.orchestra.run.trial016.stdout:/sbin/grubby 2026-02-17T05:43:02.932 INFO:teuthology.orchestra.run.trial016.stdout:yes 2026-02-17T05:43:02.933 DEBUG:teuthology.orchestra.run.trial016:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-681.el9.x86_64" | head -n 1 2026-02-17T05:43:02.998 INFO:teuthology.orchestra.run.trial016.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:43:02.999 DEBUG:teuthology.orchestra.run.trial016:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64 2026-02-17T05:43:03.071 DEBUG:teuthology.orchestra.run.trial016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-17T05:43:03.278 INFO:teuthology.orchestra.run.trial016.stderr:Generating grub configuration file ... 2026-02-17T05:43:03.339 INFO:teuthology.orchestra.run.trial016.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-17T05:43:03.345 INFO:teuthology.orchestra.run.trial016.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-17T05:43:03.590 INFO:teuthology.orchestra.run.trial016.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-17T05:43:03.597 INFO:teuthology.orchestra.run.trial016.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-17T05:43:03.629 INFO:teuthology.orchestra.run.trial016.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-17T05:43:03.635 INFO:teuthology.orchestra.run.trial016.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-17T05:43:03.659 INFO:teuthology.orchestra.run.trial016.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-17T05:43:03.666 INFO:teuthology.orchestra.run.trial016.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-17T05:43:03.748 INFO:teuthology.orchestra.run.trial016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T05:43:03.757 INFO:teuthology.orchestra.run.trial016.stderr:done 2026-02-17T05:43:03.760 DEBUG:teuthology.orchestra.run.trial016:> test -d /sys/firmware/efi 2026-02-17T05:43:03.775 DEBUG:teuthology.orchestra.run.trial016:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-17T05:43:03.839 INFO:teuthology.orchestra.run.trial016.stdout:/boot/efi/EFI/BOOT 2026-02-17T05:43:03.840 INFO:teuthology.orchestra.run.trial016.stdout:/boot/efi/EFI/centos 2026-02-17T05:43:03.841 DEBUG:teuthology.orchestra.run.trial016:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-17T05:43:03.906 INFO:teuthology.orchestra.run.trial016.stdout:no 2026-02-17T05:43:03.907 DEBUG:teuthology.orchestra.run.trial016:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-17T05:43:03.970 INFO:teuthology.orchestra.run.trial016.stdout:yes 2026-02-17T05:43:03.970 DEBUG:teuthology.orchestra.run.trial016:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-17T05:43:04.034 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial016 2026-02-17T05:43:04.034 DEBUG:teuthology.orchestra.run.trial016:> sudo bash -c ' 2026-02-17T05:43:04.035 DEBUG:teuthology.orchestra.run.trial016:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T05:43:04.035 DEBUG:teuthology.orchestra.run.trial016:> ' 2026-02-17T05:43:06.080 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:06.100 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:06.164 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:06.164 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:43:06.164 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:43:06.164 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:43:06.216 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:06.216 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:43:06.217 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:43:06.217 INFO:teuthology.orchestra.run.trial128.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-17T05:43:06.217 INFO:teuthology.orchestra.run.trial128.stdout: kernel-core-5.14.0-681.el9.x86_64 2026-02-17T05:43:06.217 INFO:teuthology.orchestra.run.trial128.stdout: kernel-modules-5.14.0-681.el9.x86_64 2026-02-17T05:43:06.217 INFO:teuthology.orchestra.run.trial128.stdout: kernel-modules-core-5.14.0-681.el9.x86_64 2026-02-17T05:43:06.217 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:43:06.217 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:43:06.274 DEBUG:teuthology.orchestra.run.trial128:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:43:06.298 INFO:teuthology.orchestra.run.trial128.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:43:06.299 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:43:06.299 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-681.el9.x86_64 2026-02-17T05:43:06.299 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T05:43:06.299 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T05:43:06.299 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2026-02-17T05:43:06.299 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial128.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T05:43:06.299 DEBUG:teuthology.orchestra.run.trial128:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:43:06.361 INFO:teuthology.orchestra.run.trial128.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:43:06.362 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:43:06.362 DEBUG:teuthology.orchestra.run.trial128:> sudo rpm -qi grub2-tools 2026-02-17T05:43:06.430 INFO:teuthology.orchestra.run.trial128.stdout:Name : grub2-tools 2026-02-17T05:43:06.430 INFO:teuthology.orchestra.run.trial128.stdout:Epoch : 1 2026-02-17T05:43:06.430 INFO:teuthology.orchestra.run.trial128.stdout:Version : 2.06 2026-02-17T05:43:06.430 INFO:teuthology.orchestra.run.trial128.stdout:Release : 118.el9 2026-02-17T05:43:06.430 INFO:teuthology.orchestra.run.trial128.stdout:Architecture: x86_64 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:Group : Unspecified 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:Size : 8393909 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:License : GPLv3+ 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:Packager : builder@centos.org 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:Vendor : CentOS 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-17T05:43:06.431 INFO:teuthology.orchestra.run.trial128.stdout:Summary : Support tools for GRUB. 2026-02-17T05:43:06.432 INFO:teuthology.orchestra.run.trial128.stdout:Description : 2026-02-17T05:43:06.432 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:43:06.432 INFO:teuthology.orchestra.run.trial128.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-17T05:43:06.432 INFO:teuthology.orchestra.run.trial128.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-17T05:43:06.432 INFO:teuthology.orchestra.run.trial128.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-17T05:43:06.432 INFO:teuthology.orchestra.run.trial128.stdout:hardware devices. 2026-02-17T05:43:06.432 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:43:06.432 INFO:teuthology.orchestra.run.trial128.stdout:This subpackage provides tools for support of all platforms. 2026-02-17T05:43:06.433 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-17T05:43:06.434 INFO:teuthology.task.kernel:Updating grub on trial128 to boot 5.14.0-681.el9.x86_64 2026-02-17T05:43:06.434 DEBUG:teuthology.orchestra.run.trial128:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-17T05:43:06.498 INFO:teuthology.orchestra.run.trial128.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:43:06.500 DEBUG:teuthology.orchestra.run.trial128:> sudo command -v grubby && echo yes || echo no 2026-02-17T05:43:06.566 INFO:teuthology.orchestra.run.trial128.stdout:/sbin/grubby 2026-02-17T05:43:06.567 INFO:teuthology.orchestra.run.trial128.stdout:yes 2026-02-17T05:43:06.567 DEBUG:teuthology.orchestra.run.trial128:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-681.el9.x86_64" | head -n 1 2026-02-17T05:43:06.630 INFO:teuthology.orchestra.run.trial128.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:43:06.631 DEBUG:teuthology.orchestra.run.trial128:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64 2026-02-17T05:43:06.704 DEBUG:teuthology.orchestra.run.trial128:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-17T05:43:06.912 INFO:teuthology.orchestra.run.trial128.stderr:Generating grub configuration file ... 2026-02-17T05:43:06.975 INFO:teuthology.orchestra.run.trial128.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-17T05:43:06.981 INFO:teuthology.orchestra.run.trial128.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-17T05:43:07.227 INFO:teuthology.orchestra.run.trial128.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-17T05:43:07.234 INFO:teuthology.orchestra.run.trial128.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-17T05:43:07.266 INFO:teuthology.orchestra.run.trial128.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-17T05:43:07.273 INFO:teuthology.orchestra.run.trial128.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-17T05:43:07.297 INFO:teuthology.orchestra.run.trial128.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-17T05:43:07.303 INFO:teuthology.orchestra.run.trial128.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-17T05:43:07.385 INFO:teuthology.orchestra.run.trial128.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T05:43:07.393 INFO:teuthology.orchestra.run.trial128.stderr:done 2026-02-17T05:43:07.395 DEBUG:teuthology.orchestra.run.trial128:> test -d /sys/firmware/efi 2026-02-17T05:43:07.412 DEBUG:teuthology.orchestra.run.trial128:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-17T05:43:07.476 INFO:teuthology.orchestra.run.trial128.stdout:/boot/efi/EFI/BOOT 2026-02-17T05:43:07.476 INFO:teuthology.orchestra.run.trial128.stdout:/boot/efi/EFI/centos 2026-02-17T05:43:07.478 DEBUG:teuthology.orchestra.run.trial128:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-17T05:43:07.543 INFO:teuthology.orchestra.run.trial128.stdout:no 2026-02-17T05:43:07.543 DEBUG:teuthology.orchestra.run.trial128:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-17T05:43:07.607 INFO:teuthology.orchestra.run.trial128.stdout:yes 2026-02-17T05:43:07.607 DEBUG:teuthology.orchestra.run.trial128:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-17T05:43:07.672 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial128 2026-02-17T05:43:07.672 DEBUG:teuthology.orchestra.run.trial128:> sudo bash -c ' 2026-02-17T05:43:07.672 DEBUG:teuthology.orchestra.run.trial128:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T05:43:07.672 DEBUG:teuthology.orchestra.run.trial128:> ' 2026-02-17T05:43:16.341 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:16.361 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:16.425 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:16.425 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:43:16.425 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:43:16.425 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:43:16.477 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:43:16.478 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:43:16.478 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:43:16.478 INFO:teuthology.orchestra.run.trial148.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-17T05:43:16.478 INFO:teuthology.orchestra.run.trial148.stdout: kernel-core-5.14.0-681.el9.x86_64 2026-02-17T05:43:16.478 INFO:teuthology.orchestra.run.trial148.stdout: kernel-modules-5.14.0-681.el9.x86_64 2026-02-17T05:43:16.478 INFO:teuthology.orchestra.run.trial148.stdout: kernel-modules-core-5.14.0-681.el9.x86_64 2026-02-17T05:43:16.478 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:43:16.478 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:43:16.534 DEBUG:teuthology.orchestra.run.trial148:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:43:16.560 INFO:teuthology.orchestra.run.trial148.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:43:16.560 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:43:16.561 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-681.el9.x86_64 2026-02-17T05:43:16.561 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T05:43:16.561 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T05:43:16.561 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2026-02-17T05:43:16.561 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial148.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T05:43:16.561 DEBUG:teuthology.orchestra.run.trial148:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:43:16.625 INFO:teuthology.orchestra.run.trial148.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:43:16.625 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:43:16.626 DEBUG:teuthology.orchestra.run.trial148:> sudo rpm -qi grub2-tools 2026-02-17T05:43:16.693 INFO:teuthology.orchestra.run.trial148.stdout:Name : grub2-tools 2026-02-17T05:43:16.693 INFO:teuthology.orchestra.run.trial148.stdout:Epoch : 1 2026-02-17T05:43:16.693 INFO:teuthology.orchestra.run.trial148.stdout:Version : 2.06 2026-02-17T05:43:16.693 INFO:teuthology.orchestra.run.trial148.stdout:Release : 118.el9 2026-02-17T05:43:16.693 INFO:teuthology.orchestra.run.trial148.stdout:Architecture: x86_64 2026-02-17T05:43:16.693 INFO:teuthology.orchestra.run.trial148.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:Group : Unspecified 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:Size : 8393909 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:License : GPLv3+ 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:Packager : builder@centos.org 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:Vendor : CentOS 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:Summary : Support tools for GRUB. 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:Description : 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-17T05:43:16.694 INFO:teuthology.orchestra.run.trial148.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-17T05:43:16.695 INFO:teuthology.orchestra.run.trial148.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-17T05:43:16.695 INFO:teuthology.orchestra.run.trial148.stdout:hardware devices. 2026-02-17T05:43:16.695 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:43:16.695 INFO:teuthology.orchestra.run.trial148.stdout:This subpackage provides tools for support of all platforms. 2026-02-17T05:43:16.695 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-17T05:43:16.695 INFO:teuthology.task.kernel:Updating grub on trial148 to boot 5.14.0-681.el9.x86_64 2026-02-17T05:43:16.696 DEBUG:teuthology.orchestra.run.trial148:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-17T05:43:16.760 INFO:teuthology.orchestra.run.trial148.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:43:16.761 DEBUG:teuthology.orchestra.run.trial148:> sudo command -v grubby && echo yes || echo no 2026-02-17T05:43:16.824 INFO:teuthology.orchestra.run.trial148.stdout:/sbin/grubby 2026-02-17T05:43:16.826 INFO:teuthology.orchestra.run.trial148.stdout:yes 2026-02-17T05:43:16.826 DEBUG:teuthology.orchestra.run.trial148:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-681.el9.x86_64" | head -n 1 2026-02-17T05:43:16.894 INFO:teuthology.orchestra.run.trial148.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:43:16.894 DEBUG:teuthology.orchestra.run.trial148:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64 2026-02-17T05:43:16.965 DEBUG:teuthology.orchestra.run.trial148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-17T05:43:17.173 INFO:teuthology.orchestra.run.trial148.stderr:Generating grub configuration file ... 2026-02-17T05:43:17.235 INFO:teuthology.orchestra.run.trial148.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-17T05:43:17.241 INFO:teuthology.orchestra.run.trial148.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-17T05:43:17.493 INFO:teuthology.orchestra.run.trial148.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-17T05:43:17.499 INFO:teuthology.orchestra.run.trial148.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-17T05:43:17.530 INFO:teuthology.orchestra.run.trial148.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-17T05:43:17.537 INFO:teuthology.orchestra.run.trial148.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-17T05:43:17.561 INFO:teuthology.orchestra.run.trial148.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-17T05:43:17.569 INFO:teuthology.orchestra.run.trial148.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-17T05:43:17.651 INFO:teuthology.orchestra.run.trial148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T05:43:17.659 INFO:teuthology.orchestra.run.trial148.stderr:done 2026-02-17T05:43:17.661 DEBUG:teuthology.orchestra.run.trial148:> test -d /sys/firmware/efi 2026-02-17T05:43:17.677 DEBUG:teuthology.orchestra.run.trial148:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-17T05:43:17.742 INFO:teuthology.orchestra.run.trial148.stdout:/boot/efi/EFI/BOOT 2026-02-17T05:43:17.742 INFO:teuthology.orchestra.run.trial148.stdout:/boot/efi/EFI/centos 2026-02-17T05:43:17.744 DEBUG:teuthology.orchestra.run.trial148:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-17T05:43:17.808 INFO:teuthology.orchestra.run.trial148.stdout:no 2026-02-17T05:43:17.809 DEBUG:teuthology.orchestra.run.trial148:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-17T05:43:17.873 INFO:teuthology.orchestra.run.trial148.stdout:yes 2026-02-17T05:43:17.873 DEBUG:teuthology.orchestra.run.trial148:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-17T05:43:17.936 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial148 2026-02-17T05:43:17.936 DEBUG:teuthology.orchestra.run.trial148:> sudo bash -c ' 2026-02-17T05:43:17.937 DEBUG:teuthology.orchestra.run.trial148:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T05:43:17.937 DEBUG:teuthology.orchestra.run.trial148:> ' 2026-02-17T05:43:22.924 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2026-02-17T05:43:22.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial164.front.sepia.ceph.com' 2026-02-17T05:43:22.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:43:34.077 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-17T05:43:34.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial016.front.sepia.ceph.com' 2026-02-17T05:43:34.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:43:37.216 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.164 2026-02-17T05:43:37.714 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2026-02-17T05:43:37.715 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial128.front.sepia.ceph.com' 2026-02-17T05:43:37.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial128.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:43:46.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial164.front.sepia.ceph.com' 2026-02-17T05:43:46.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:43:47.980 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2026-02-17T05:43:47.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial148.front.sepia.ceph.com' 2026-02-17T05:43:47.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:43:48.351 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.16 2026-02-17T05:43:52.000 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.128 2026-02-17T05:43:57.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial016.front.sepia.ceph.com' 2026-02-17T05:43:57.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:44:01.004 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial128.front.sepia.ceph.com' 2026-02-17T05:44:01.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial128.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:44:02.240 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.148 2026-02-17T05:44:11.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial148.front.sepia.ceph.com' 2026-02-17T05:44:11.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:44:22.278 DEBUG:teuthology.orchestra.run.trial164:> true 2026-02-17T05:44:22.426 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial164.front.sepia.ceph.com' 2026-02-17T05:44:22.426 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-681.el9.x86_64"... 2026-02-17T05:44:22.427 DEBUG:teuthology.orchestra.run.trial164:> uname -r 2026-02-17T05:44:22.482 INFO:teuthology.orchestra.run.trial164.stdout:5.14.0-681.el9.x86_64 2026-02-17T05:44:22.483 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 5.14.0-681.el9.x86_64 2026-02-17T05:44:22.483 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T05:44:22.483 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-17T05:44:23.484 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-17T05:44:23.484 DEBUG:teuthology.orchestra.run.trial164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T05:44:23.513 INFO:teuthology.orchestra.run.trial164.stdout:ttyS1 2026-02-17T05:44:23.535 DEBUG:teuthology.parallel:result is None 2026-02-17T05:44:33.538 DEBUG:teuthology.orchestra.run.trial016:> true 2026-02-17T05:44:33.693 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial016.front.sepia.ceph.com' 2026-02-17T05:44:33.693 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-681.el9.x86_64"... 2026-02-17T05:44:33.694 DEBUG:teuthology.orchestra.run.trial016:> uname -r 2026-02-17T05:44:33.749 INFO:teuthology.orchestra.run.trial016.stdout:5.14.0-681.el9.x86_64 2026-02-17T05:44:33.749 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 5.14.0-681.el9.x86_64 2026-02-17T05:44:33.749 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T05:44:33.749 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-17T05:44:34.750 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-17T05:44:34.751 DEBUG:teuthology.orchestra.run.trial016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T05:44:34.780 INFO:teuthology.orchestra.run.trial016.stdout:ttyS1 2026-02-17T05:44:34.803 DEBUG:teuthology.parallel:result is None 2026-02-17T05:44:37.125 DEBUG:teuthology.orchestra.run.trial128:> true 2026-02-17T05:44:37.277 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial128.front.sepia.ceph.com' 2026-02-17T05:44:37.277 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-681.el9.x86_64"... 2026-02-17T05:44:37.277 DEBUG:teuthology.orchestra.run.trial128:> uname -r 2026-02-17T05:44:37.335 INFO:teuthology.orchestra.run.trial128.stdout:5.14.0-681.el9.x86_64 2026-02-17T05:44:37.335 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 5.14.0-681.el9.x86_64 2026-02-17T05:44:37.336 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T05:44:37.336 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-17T05:44:38.337 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2026-02-17T05:44:38.337 DEBUG:teuthology.orchestra.run.trial128:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T05:44:38.366 INFO:teuthology.orchestra.run.trial128.stdout:ttyS1 2026-02-17T05:44:38.386 DEBUG:teuthology.parallel:result is None 2026-02-17T05:44:47.330 DEBUG:teuthology.orchestra.run.trial148:> true 2026-02-17T05:44:47.484 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial148.front.sepia.ceph.com' 2026-02-17T05:44:47.484 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-681.el9.x86_64"... 2026-02-17T05:44:47.484 DEBUG:teuthology.orchestra.run.trial148:> uname -r 2026-02-17T05:44:47.540 INFO:teuthology.orchestra.run.trial148.stdout:5.14.0-681.el9.x86_64 2026-02-17T05:44:47.540 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 5.14.0-681.el9.x86_64 2026-02-17T05:44:47.540 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T05:44:47.540 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-17T05:44:48.541 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2026-02-17T05:44:48.541 DEBUG:teuthology.orchestra.run.trial148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T05:44:48.571 INFO:teuthology.orchestra.run.trial148.stdout:ttyS1 2026-02-17T05:44:48.597 DEBUG:teuthology.parallel:result is None 2026-02-17T05:44:48.597 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-17T05:44:48.611 INFO:teuthology.task.internal:Creating test directory... 2026-02-17T05:44:48.611 DEBUG:teuthology.orchestra.run.trial016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T05:44:48.614 DEBUG:teuthology.orchestra.run.trial128:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T05:44:48.616 DEBUG:teuthology.orchestra.run.trial148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T05:44:48.639 DEBUG:teuthology.orchestra.run.trial164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T05:44:48.656 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-17T05:44:48.667 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-17T05:44:48.683 INFO:teuthology.task.internal:Creating archive directory... 2026-02-17T05:44:48.684 DEBUG:teuthology.orchestra.run.trial016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T05:44:48.687 DEBUG:teuthology.orchestra.run.trial128:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T05:44:48.690 DEBUG:teuthology.orchestra.run.trial148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T05:44:48.695 DEBUG:teuthology.orchestra.run.trial164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T05:44:48.722 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-17T05:44:48.733 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-17T05:44:48.734 DEBUG:teuthology.orchestra.run.trial016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T05:44:48.744 DEBUG:teuthology.orchestra.run.trial128:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T05:44:48.747 DEBUG:teuthology.orchestra.run.trial148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T05:44:48.753 DEBUG:teuthology.orchestra.run.trial164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T05:44:48.767 INFO:teuthology.orchestra.run.trial016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:44:48.770 INFO:teuthology.orchestra.run.trial128.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:44:48.775 INFO:teuthology.orchestra.run.trial148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:44:48.776 INFO:teuthology.orchestra.run.trial016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:44:48.777 INFO:teuthology.orchestra.run.trial164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:44:48.779 INFO:teuthology.orchestra.run.trial128.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:44:48.783 INFO:teuthology.orchestra.run.trial148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:44:48.785 INFO:teuthology.orchestra.run.trial164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:44:48.787 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-17T05:44:48.799 INFO:teuthology.task.internal:Configuring sudo... 2026-02-17T05:44:48.799 DEBUG:teuthology.orchestra.run.trial016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T05:44:48.820 DEBUG:teuthology.orchestra.run.trial128:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T05:44:48.823 DEBUG:teuthology.orchestra.run.trial148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T05:44:48.827 DEBUG:teuthology.orchestra.run.trial164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T05:44:48.853 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-17T05:44:48.870 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-17T05:44:48.870 DEBUG:teuthology.orchestra.run.trial016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T05:44:48.885 DEBUG:teuthology.orchestra.run.trial128:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T05:44:48.888 DEBUG:teuthology.orchestra.run.trial148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T05:44:48.891 DEBUG:teuthology.orchestra.run.trial164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T05:44:48.909 DEBUG:teuthology.orchestra.run.trial016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:44:49.056 DEBUG:teuthology.orchestra.run.trial016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:44:49.082 DEBUG:teuthology.orchestra.run.trial016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:44:49.225 DEBUG:teuthology.orchestra.run.trial016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:44:49.250 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:44:49.250 DEBUG:teuthology.orchestra.run.trial016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T05:44:49.314 DEBUG:teuthology.orchestra.run.trial128:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:44:49.425 DEBUG:teuthology.orchestra.run.trial128:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:44:49.450 DEBUG:teuthology.orchestra.run.trial128:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:44:49.592 DEBUG:teuthology.orchestra.run.trial128:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:44:49.616 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:44:49.616 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T05:44:49.680 DEBUG:teuthology.orchestra.run.trial148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:44:49.790 DEBUG:teuthology.orchestra.run.trial148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:44:49.815 DEBUG:teuthology.orchestra.run.trial148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:44:49.961 DEBUG:teuthology.orchestra.run.trial148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:44:49.986 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:44:49.986 DEBUG:teuthology.orchestra.run.trial148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T05:44:50.049 DEBUG:teuthology.orchestra.run.trial164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:44:50.155 DEBUG:teuthology.orchestra.run.trial164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:44:50.179 DEBUG:teuthology.orchestra.run.trial164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:44:50.323 DEBUG:teuthology.orchestra.run.trial164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:44:50.347 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:44:50.347 DEBUG:teuthology.orchestra.run.trial164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T05:44:50.412 DEBUG:teuthology.orchestra.run.trial016:> sudo service rsyslog restart 2026-02-17T05:44:50.415 DEBUG:teuthology.orchestra.run.trial128:> sudo service rsyslog restart 2026-02-17T05:44:50.418 DEBUG:teuthology.orchestra.run.trial148:> sudo service rsyslog restart 2026-02-17T05:44:50.420 DEBUG:teuthology.orchestra.run.trial164:> sudo service rsyslog restart 2026-02-17T05:44:50.441 INFO:teuthology.orchestra.run.trial016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T05:44:50.445 INFO:teuthology.orchestra.run.trial128.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T05:44:50.447 INFO:teuthology.orchestra.run.trial148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T05:44:50.480 INFO:teuthology.orchestra.run.trial164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T05:44:50.964 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-17T05:44:50.978 INFO:teuthology.task.internal:Starting timer... 2026-02-17T05:44:50.979 INFO:teuthology.run_tasks:Running task pcp... 2026-02-17T05:44:51.000 INFO:teuthology.run_tasks:Running task selinux... 2026-02-17T05:44:51.018 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2026-02-17T05:44:51.018 DEBUG:teuthology.orchestra.run.trial016:> sudo service auditd rotate 2026-02-17T05:44:51.049 INFO:teuthology.orchestra.run.trial016.stdout:Rotating logs: 2026-02-17T05:44:51.051 DEBUG:teuthology.orchestra.run.trial128:> sudo service auditd rotate 2026-02-17T05:44:51.080 INFO:teuthology.orchestra.run.trial128.stdout:Rotating logs: 2026-02-17T05:44:51.082 DEBUG:teuthology.orchestra.run.trial148:> sudo service auditd rotate 2026-02-17T05:44:51.112 INFO:teuthology.orchestra.run.trial148.stdout:Rotating logs: 2026-02-17T05:44:51.114 DEBUG:teuthology.orchestra.run.trial164:> sudo service auditd rotate 2026-02-17T05:44:51.143 INFO:teuthology.orchestra.run.trial164.stdout:Rotating logs: 2026-02-17T05:44:51.145 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-17T05:44:51.145 DEBUG:teuthology.orchestra.run.trial016:> /usr/sbin/getenforce 2026-02-17T05:44:51.163 INFO:teuthology.orchestra.run.trial016.stdout:Permissive 2026-02-17T05:44:51.163 DEBUG:teuthology.orchestra.run.trial128:> /usr/sbin/getenforce 2026-02-17T05:44:51.181 INFO:teuthology.orchestra.run.trial128.stdout:Permissive 2026-02-17T05:44:51.181 DEBUG:teuthology.orchestra.run.trial148:> /usr/sbin/getenforce 2026-02-17T05:44:51.199 INFO:teuthology.orchestra.run.trial148.stdout:Permissive 2026-02-17T05:44:51.200 DEBUG:teuthology.orchestra.run.trial164:> /usr/sbin/getenforce 2026-02-17T05:44:51.217 INFO:teuthology.orchestra.run.trial164.stdout:Permissive 2026-02-17T05:44:51.217 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial016.front.sepia.ceph.com': 'permissive', 'ubuntu@trial128.front.sepia.ceph.com': 'permissive', 'ubuntu@trial148.front.sepia.ceph.com': 'permissive', 'ubuntu@trial164.front.sepia.ceph.com': 'permissive'} 2026-02-17T05:44:51.217 DEBUG:teuthology.orchestra.run.trial016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T05:44:51.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:44:51.245 DEBUG:teuthology.orchestra.run.trial128:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T05:44:51.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:44:51.272 DEBUG:teuthology.orchestra.run.trial148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T05:44:51.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:44:51.299 DEBUG:teuthology.orchestra.run.trial164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T05:44:51.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:44:51.325 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-17T05:44:51.325 DEBUG:teuthology.orchestra.run.trial016:> sudo /usr/sbin/setenforce permissive 2026-02-17T05:44:51.352 DEBUG:teuthology.orchestra.run.trial128:> sudo /usr/sbin/setenforce permissive 2026-02-17T05:44:51.378 DEBUG:teuthology.orchestra.run.trial148:> sudo /usr/sbin/setenforce permissive 2026-02-17T05:44:51.403 DEBUG:teuthology.orchestra.run.trial164:> sudo /usr/sbin/setenforce permissive 2026-02-17T05:44:51.429 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-17T05:44:51.445 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-17T05:44:51.458 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-17T05:44:51.459 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial016.front.sepia.ceph.com,trial128.front.sepia.ceph.com,trial148.front.sepia.ceph.com,trial164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-17T05:47:42.731 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial016.front.sepia.ceph.com'), Remote(name='ubuntu@trial128.front.sepia.ceph.com'), Remote(name='ubuntu@trial148.front.sepia.ceph.com'), Remote(name='ubuntu@trial164.front.sepia.ceph.com')] 2026-02-17T05:47:42.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial016.front.sepia.ceph.com' 2026-02-17T05:47:42.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:47:42.847 DEBUG:teuthology.orchestra.run.trial016:> true 2026-02-17T05:47:42.922 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial016.front.sepia.ceph.com' 2026-02-17T05:47:42.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial128.front.sepia.ceph.com' 2026-02-17T05:47:42.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial128.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:47:42.995 DEBUG:teuthology.orchestra.run.trial128:> true 2026-02-17T05:47:43.070 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial128.front.sepia.ceph.com' 2026-02-17T05:47:43.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial148.front.sepia.ceph.com' 2026-02-17T05:47:43.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:47:43.150 DEBUG:teuthology.orchestra.run.trial148:> true 2026-02-17T05:47:43.226 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial148.front.sepia.ceph.com' 2026-02-17T05:47:43.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial164.front.sepia.ceph.com' 2026-02-17T05:47:43.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:47:43.328 DEBUG:teuthology.orchestra.run.trial164:> true 2026-02-17T05:47:43.400 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial164.front.sepia.ceph.com' 2026-02-17T05:47:43.401 INFO:teuthology.run_tasks:Running task clock... 2026-02-17T05:47:43.415 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-17T05:47:43.415 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T05:47:43.416 DEBUG:teuthology.orchestra.run.trial016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T05:47:43.418 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T05:47:43.418 DEBUG:teuthology.orchestra.run.trial128:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T05:47:43.420 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T05:47:43.420 DEBUG:teuthology.orchestra.run.trial148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T05:47:43.423 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T05:47:43.423 DEBUG:teuthology.orchestra.run.trial164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T05:47:43.444 INFO:teuthology.orchestra.run.trial016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-17T05:47:43.446 INFO:teuthology.orchestra.run.trial128.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-17T05:47:43.447 INFO:teuthology.orchestra.run.trial148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-17T05:47:43.456 INFO:teuthology.orchestra.run.trial016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-17T05:47:43.457 INFO:teuthology.orchestra.run.trial128.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-17T05:47:43.459 INFO:teuthology.orchestra.run.trial148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-17T05:47:43.465 INFO:teuthology.orchestra.run.trial164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-17T05:47:43.477 INFO:teuthology.orchestra.run.trial128.stderr:sudo: ntpd: command not found 2026-02-17T05:47:43.478 INFO:teuthology.orchestra.run.trial164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-17T05:47:43.478 INFO:teuthology.orchestra.run.trial016.stderr:sudo: ntpd: command not found 2026-02-17T05:47:43.479 INFO:teuthology.orchestra.run.trial148.stderr:sudo: ntpd: command not found 2026-02-17T05:47:43.486 INFO:teuthology.orchestra.run.trial128.stdout:506 Cannot talk to daemon 2026-02-17T05:47:43.487 INFO:teuthology.orchestra.run.trial016.stdout:506 Cannot talk to daemon 2026-02-17T05:47:43.489 INFO:teuthology.orchestra.run.trial148.stdout:506 Cannot talk to daemon 2026-02-17T05:47:43.498 INFO:teuthology.orchestra.run.trial164.stderr:sudo: ntpd: command not found 2026-02-17T05:47:43.498 INFO:teuthology.orchestra.run.trial128.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-17T05:47:43.499 INFO:teuthology.orchestra.run.trial016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-17T05:47:43.501 INFO:teuthology.orchestra.run.trial148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-17T05:47:43.507 INFO:teuthology.orchestra.run.trial164.stdout:506 Cannot talk to daemon 2026-02-17T05:47:43.509 INFO:teuthology.orchestra.run.trial016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-17T05:47:43.510 INFO:teuthology.orchestra.run.trial128.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-17T05:47:43.512 INFO:teuthology.orchestra.run.trial148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-17T05:47:43.519 INFO:teuthology.orchestra.run.trial164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-17T05:47:43.531 INFO:teuthology.orchestra.run.trial164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-17T05:47:43.560 INFO:teuthology.orchestra.run.trial128.stderr:bash: line 1: ntpq: command not found 2026-02-17T05:47:43.560 INFO:teuthology.orchestra.run.trial016.stderr:bash: line 1: ntpq: command not found 2026-02-17T05:47:43.561 INFO:teuthology.orchestra.run.trial128.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T05:47:43.561 INFO:teuthology.orchestra.run.trial128.stdout:=============================================================================== 2026-02-17T05:47:43.564 INFO:teuthology.orchestra.run.trial016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T05:47:43.564 INFO:teuthology.orchestra.run.trial016.stdout:=============================================================================== 2026-02-17T05:47:43.566 INFO:teuthology.orchestra.run.trial148.stderr:bash: line 1: ntpq: command not found 2026-02-17T05:47:43.568 INFO:teuthology.orchestra.run.trial148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T05:47:43.568 INFO:teuthology.orchestra.run.trial148.stdout:=============================================================================== 2026-02-17T05:47:43.576 INFO:teuthology.orchestra.run.trial164.stderr:bash: line 1: ntpq: command not found 2026-02-17T05:47:43.581 INFO:teuthology.orchestra.run.trial164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T05:47:43.581 INFO:teuthology.orchestra.run.trial164.stdout:=============================================================================== 2026-02-17T05:47:43.581 INFO:teuthology.orchestra.run.trial164.stdout:^? callisto.psychotron.org 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T05:47:43.583 INFO:teuthology.run_tasks:Running task install... 2026-02-17T05:47:43.597 DEBUG:teuthology.task.install:project ceph 2026-02-17T05:47:43.597 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'}} 2026-02-17T05:47:43.597 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'} 2026-02-17T05:47:43.597 INFO:teuthology.task.install:Using flavor: default 2026-02-17T05:47:43.606 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2026-02-17T05:47:43.606 INFO:teuthology.task.install:extra packages: [] 2026-02-17T05:47:43.606 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'tag': None, 'wait_for_package': False} 2026-02-17T05:47:43.607 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:47:43.611 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'tag': None, 'wait_for_package': False} 2026-02-17T05:47:43.611 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:47:43.613 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'tag': None, 'wait_for_package': False} 2026-02-17T05:47:43.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:47:43.614 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'tag': None, 'wait_for_package': False} 2026-02-17T05:47:43.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:47:43.900 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/ 2026-02-17T05:47:43.900 INFO:teuthology.task.install.rpm:Package version is 19.2.3-705.g04cf6dd4 2026-02-17T05:47:43.912 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/ 2026-02-17T05:47:43.913 INFO:teuthology.task.install.rpm:Package version is 19.2.3-705.g04cf6dd4 2026-02-17T05:47:43.913 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/ 2026-02-17T05:47:43.913 INFO:teuthology.task.install.rpm:Package version is 19.2.3-705.g04cf6dd4 2026-02-17T05:47:43.914 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/ 2026-02-17T05:47:43.914 INFO:teuthology.task.install.rpm:Package version is 19.2.3-705.g04cf6dd4 2026-02-17T05:47:44.137 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T05:47:44.137 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:47:44.137 DEBUG:teuthology.orchestra.run.trial016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T05:47:44.138 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T05:47:44.139 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:47:44.139 DEBUG:teuthology.orchestra.run.trial164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T05:47:44.140 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T05:47:44.140 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:47:44.140 DEBUG:teuthology.orchestra.run.trial148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T05:47:44.148 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T05:47:44.148 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:47:44.148 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T05:47:44.163 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-17T05:47:44.163 DEBUG:teuthology.orchestra.run.trial016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T05:47:44.165 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-17T05:47:44.166 DEBUG:teuthology.orchestra.run.trial148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T05:47:44.166 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-17T05:47:44.166 DEBUG:teuthology.orchestra.run.trial164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T05:47:44.174 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-17T05:47:44.174 DEBUG:teuthology.orchestra.run.trial128:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T05:47:44.228 DEBUG:teuthology.orchestra.run.trial016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T05:47:44.230 DEBUG:teuthology.orchestra.run.trial148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T05:47:44.231 DEBUG:teuthology.orchestra.run.trial164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T05:47:44.239 DEBUG:teuthology.orchestra.run.trial128:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T05:47:44.302 DEBUG:teuthology.orchestra.run.trial148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T05:47:44.303 DEBUG:teuthology.orchestra.run.trial016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T05:47:44.307 DEBUG:teuthology.orchestra.run.trial164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T05:47:44.314 DEBUG:teuthology.orchestra.run.trial128:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T05:47:44.366 INFO:teuthology.orchestra.run.trial148.stdout:check_obsoletes = 1 2026-02-17T05:47:44.368 DEBUG:teuthology.orchestra.run.trial148:> sudo yum clean all 2026-02-17T05:47:44.370 INFO:teuthology.orchestra.run.trial016.stdout:check_obsoletes = 1 2026-02-17T05:47:44.370 DEBUG:teuthology.orchestra.run.trial016:> sudo yum clean all 2026-02-17T05:47:44.371 INFO:teuthology.orchestra.run.trial164.stdout:check_obsoletes = 1 2026-02-17T05:47:44.373 DEBUG:teuthology.orchestra.run.trial164:> sudo yum clean all 2026-02-17T05:47:44.376 INFO:teuthology.orchestra.run.trial128.stdout:check_obsoletes = 1 2026-02-17T05:47:44.378 DEBUG:teuthology.orchestra.run.trial128:> sudo yum clean all 2026-02-17T05:47:44.517 INFO:teuthology.orchestra.run.trial148.stdout:41 files removed 2026-02-17T05:47:44.518 INFO:teuthology.orchestra.run.trial016.stdout:41 files removed 2026-02-17T05:47:44.523 INFO:teuthology.orchestra.run.trial164.stdout:41 files removed 2026-02-17T05:47:44.526 INFO:teuthology.orchestra.run.trial128.stdout:41 files removed 2026-02-17T05:47:44.532 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-radosgw 2026-02-17T05:47:44.532 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-radosgw 2026-02-17T05:47:44.538 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-radosgw 2026-02-17T05:47:44.541 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-radosgw 2026-02-17T05:47:44.816 INFO:teuthology.orchestra.run.trial164.stdout:ceph packages for x86_64 654 kB/s | 86 kB 00:00 2026-02-17T05:47:44.818 INFO:teuthology.orchestra.run.trial148.stdout:ceph packages for x86_64 611 kB/s | 86 kB 00:00 2026-02-17T05:47:44.818 INFO:teuthology.orchestra.run.trial128.stdout:ceph packages for x86_64 639 kB/s | 86 kB 00:00 2026-02-17T05:47:44.821 INFO:teuthology.orchestra.run.trial016.stdout:ceph packages for x86_64 600 kB/s | 86 kB 00:00 2026-02-17T05:47:44.947 INFO:teuthology.orchestra.run.trial164.stdout:ceph noarch packages 113 kB/s | 12 kB 00:00 2026-02-17T05:47:44.948 INFO:teuthology.orchestra.run.trial148.stdout:ceph noarch packages 114 kB/s | 12 kB 00:00 2026-02-17T05:47:44.949 INFO:teuthology.orchestra.run.trial128.stdout:ceph noarch packages 113 kB/s | 12 kB 00:00 2026-02-17T05:47:44.950 INFO:teuthology.orchestra.run.trial016.stdout:ceph noarch packages 115 kB/s | 12 kB 00:00 2026-02-17T05:47:45.075 INFO:teuthology.orchestra.run.trial148.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-17T05:47:45.075 INFO:teuthology.orchestra.run.trial164.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-17T05:47:45.077 INFO:teuthology.orchestra.run.trial016.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-17T05:47:45.078 INFO:teuthology.orchestra.run.trial128.stdout:ceph source packages 17 kB/s | 1.9 kB 00:00 2026-02-17T05:47:45.834 INFO:teuthology.orchestra.run.trial148.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.9 MB 00:00 2026-02-17T05:47:45.928 INFO:teuthology.orchestra.run.trial016.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-17T05:47:46.333 INFO:teuthology.orchestra.run.trial128.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 8.9 MB 00:01 2026-02-17T05:47:46.471 INFO:teuthology.orchestra.run.trial164.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.9 MB 00:01 2026-02-17T05:47:48.998 INFO:teuthology.orchestra.run.trial148.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-17T05:47:49.139 INFO:teuthology.orchestra.run.trial016.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-17T05:47:49.709 INFO:teuthology.orchestra.run.trial164.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 27 MB 00:02 2026-02-17T05:47:50.351 INFO:teuthology.orchestra.run.trial128.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 27 MB 00:03 2026-02-17T05:47:52.524 INFO:teuthology.orchestra.run.trial148.stdout:CentOS Stream 9 - CRB 6.8 MB/s | 7.8 MB 00:01 2026-02-17T05:47:53.386 INFO:teuthology.orchestra.run.trial148.stdout:CentOS Stream 9 - Extras packages 145 kB/s | 20 kB 00:00 2026-02-17T05:47:53.407 INFO:teuthology.orchestra.run.trial016.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 7.8 MB 00:01 2026-02-17T05:47:54.204 INFO:teuthology.orchestra.run.trial148.stdout:Extra Packages for Enterprise Linux 27 MB/s | 20 MB 00:00 2026-02-17T05:47:54.618 INFO:teuthology.orchestra.run.trial016.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 20 kB 00:00 2026-02-17T05:47:54.759 INFO:teuthology.orchestra.run.trial128.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 7.8 MB 00:02 2026-02-17T05:47:55.170 INFO:teuthology.orchestra.run.trial016.stdout:Extra Packages for Enterprise Linux 41 MB/s | 20 MB 00:00 2026-02-17T05:47:56.615 INFO:teuthology.orchestra.run.trial128.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 20 kB 00:01 2026-02-17T05:47:57.216 INFO:teuthology.orchestra.run.trial128.stdout:Extra Packages for Enterprise Linux 38 MB/s | 20 MB 00:00 2026-02-17T05:47:57.382 INFO:teuthology.orchestra.run.trial148.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-17T05:47:57.922 INFO:teuthology.orchestra.run.trial164.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 7.8 MB 00:05 2026-02-17T05:47:58.358 INFO:teuthology.orchestra.run.trial016.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-17T05:47:58.608 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:47:58.610 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:47:58.610 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repository Size 2026-02-17T05:47:58.610 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:47:58.610 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:47:58.610 INFO:teuthology.orchestra.run.trial148.stdout: ceph-radosgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 11 M 2026-02-17T05:47:58.610 INFO:teuthology.orchestra.run.trial148.stdout:Upgrading: 2026-02-17T05:47:58.610 INFO:teuthology.orchestra.run.trial148.stdout: librados2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.4 M 2026-02-17T05:47:58.610 INFO:teuthology.orchestra.run.trial148.stdout: librbd1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.2 M 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout:Installing dependencies: 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: ceph-base x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.5 M 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 22 M 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: ceph-selinux x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 25 k 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: libcephfs2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.0 M 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-17T05:47:58.611 INFO:teuthology.orchestra.run.trial148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: libradosstriper1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 503 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: librgw2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.4 M 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: python3-ceph-argparse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 45 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: python3-ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 154 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: python3-cephfs x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 165 k 2026-02-17T05:47:58.612 INFO:teuthology.orchestra.run.trial148.stdout: python3-rados x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 323 k 2026-02-17T05:47:58.613 INFO:teuthology.orchestra.run.trial148.stdout: python3-rbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 303 k 2026-02-17T05:47:58.613 INFO:teuthology.orchestra.run.trial148.stdout: python3-rgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 100 k 2026-02-17T05:47:58.613 INFO:teuthology.orchestra.run.trial148.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-17T05:47:58.613 INFO:teuthology.orchestra.run.trial148.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-17T05:47:58.613 INFO:teuthology.orchestra.run.trial148.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-17T05:47:58.613 INFO:teuthology.orchestra.run.trial148.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-17T05:47:58.613 INFO:teuthology.orchestra.run.trial148.stdout:Installing weak dependencies: 2026-02-17T05:47:58.613 INFO:teuthology.orchestra.run.trial148.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-17T05:47:58.613 INFO:teuthology.orchestra.run.trial148.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-17T05:47:58.613 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:47:58.614 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:47:58.614 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:47:58.614 INFO:teuthology.orchestra.run.trial148.stdout:Install 33 Packages 2026-02-17T05:47:58.614 INFO:teuthology.orchestra.run.trial148.stdout:Upgrade 2 Packages 2026-02-17T05:47:58.614 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:47:58.614 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 63 M 2026-02-17T05:47:58.614 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:47:59.295 INFO:teuthology.orchestra.run.trial148.stdout:(1/35): ceph-base-19.2.3-705.g04cf6dd4.el9.x86_ 13 MB/s | 5.5 MB 00:00 2026-02-17T05:47:59.342 INFO:teuthology.orchestra.run.trial148.stdout:(2/35): ceph-selinux-19.2.3-705.g04cf6dd4.el9.x 530 kB/s | 25 kB 00:00 2026-02-17T05:47:59.460 INFO:teuthology.orchestra.run.trial148.stdout:(3/35): libcephfs2-19.2.3-705.g04cf6dd4.el9.x86 8.3 MB/s | 1.0 MB 00:00 2026-02-17T05:47:59.509 INFO:teuthology.orchestra.run.trial148.stdout:(4/35): ceph-radosgw-19.2.3-705.g04cf6dd4.el9.x 16 MB/s | 11 MB 00:00 2026-02-17T05:47:59.518 INFO:teuthology.orchestra.run.trial148.stdout:(5/35): libradosstriper1-19.2.3-705.g04cf6dd4.e 8.6 MB/s | 503 kB 00:00 2026-02-17T05:47:59.568 INFO:teuthology.orchestra.run.trial164.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 20 kB 00:00 2026-02-17T05:47:59.574 INFO:teuthology.orchestra.run.trial148.stdout:(6/35): python3-ceph-argparse-19.2.3-705.g04cf6 801 kB/s | 45 kB 00:00 2026-02-17T05:47:59.584 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:47:59.586 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:47:59.586 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repository Size 2026-02-17T05:47:59.586 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:47:59.586 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:47:59.586 INFO:teuthology.orchestra.run.trial016.stdout: ceph-radosgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 11 M 2026-02-17T05:47:59.586 INFO:teuthology.orchestra.run.trial016.stdout:Upgrading: 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: librados2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.4 M 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: librbd1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.2 M 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout:Installing dependencies: 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: ceph-base x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.5 M 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 22 M 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: ceph-selinux x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 25 k 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: libcephfs2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.0 M 2026-02-17T05:47:59.587 INFO:teuthology.orchestra.run.trial016.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: libradosstriper1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 503 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: librgw2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.4 M 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: python3-ceph-argparse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 45 k 2026-02-17T05:47:59.588 INFO:teuthology.orchestra.run.trial016.stdout: python3-ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 154 k 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: python3-cephfs x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 165 k 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: python3-rados x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 323 k 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: python3-rbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 303 k 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: python3-rgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 100 k 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout:Installing weak dependencies: 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-17T05:47:59.589 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:47:59.590 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:47:59.590 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:47:59.590 INFO:teuthology.orchestra.run.trial016.stdout:Install 33 Packages 2026-02-17T05:47:59.590 INFO:teuthology.orchestra.run.trial016.stdout:Upgrade 2 Packages 2026-02-17T05:47:59.590 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:47:59.590 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 63 M 2026-02-17T05:47:59.591 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:47:59.637 INFO:teuthology.orchestra.run.trial148.stdout:(7/35): python3-ceph-common-19.2.3-705.g04cf6dd 2.4 MB/s | 154 kB 00:00 2026-02-17T05:47:59.696 INFO:teuthology.orchestra.run.trial148.stdout:(8/35): ceph-common-19.2.3-705.g04cf6dd4.el9.x8 26 MB/s | 22 MB 00:00 2026-02-17T05:47:59.697 INFO:teuthology.orchestra.run.trial148.stdout:(9/35): python3-cephfs-19.2.3-705.g04cf6dd4.el9 2.7 MB/s | 165 kB 00:00 2026-02-17T05:47:59.710 INFO:teuthology.orchestra.run.trial148.stdout:(10/35): python3-rados-19.2.3-705.g04cf6dd4.el9 22 MB/s | 323 kB 00:00 2026-02-17T05:47:59.725 INFO:teuthology.orchestra.run.trial148.stdout:(11/35): python3-rbd-19.2.3-705.g04cf6dd4.el9.x 11 MB/s | 303 kB 00:00 2026-02-17T05:47:59.734 INFO:teuthology.orchestra.run.trial148.stdout:(12/35): librgw2-19.2.3-705.g04cf6dd4.el9.x86_6 24 MB/s | 5.4 MB 00:00 2026-02-17T05:47:59.734 INFO:teuthology.orchestra.run.trial148.stdout:(13/35): python3-rgw-19.2.3-705.g04cf6dd4.el9.x 4.1 MB/s | 100 kB 00:00 2026-02-17T05:47:59.811 INFO:teuthology.orchestra.run.trial148.stdout:(14/35): mailcap-2.1.49-5.el9.noarch.rpm 389 kB/s | 33 kB 00:00 2026-02-17T05:47:59.811 INFO:teuthology.orchestra.run.trial148.stdout:(15/35): pciutils-3.7.0-7.el9.x86_64.rpm 1.2 MB/s | 93 kB 00:00 2026-02-17T05:47:59.846 INFO:teuthology.orchestra.run.trial164.stdout:Extra Packages for Enterprise Linux 94 MB/s | 20 MB 00:00 2026-02-17T05:47:59.876 INFO:teuthology.orchestra.run.trial148.stdout:(16/35): nvme-cli-2.13-1.el9.x86_64.rpm 7.7 MB/s | 1.1 MB 00:00 2026-02-17T05:47:59.956 INFO:teuthology.orchestra.run.trial148.stdout:(17/35): boost-program-options-1.75.0-13.el9.x8 715 kB/s | 104 kB 00:00 2026-02-17T05:47:59.972 INFO:teuthology.orchestra.run.trial148.stdout:(18/35): libnbd-1.20.3-4.el9.x86_64.rpm 1.0 MB/s | 164 kB 00:00 2026-02-17T05:47:59.987 INFO:teuthology.orchestra.run.trial148.stdout:(19/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.4 MB/s | 45 kB 00:00 2026-02-17T05:48:00.016 INFO:teuthology.orchestra.run.trial148.stdout:(20/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2026-02-17T05:48:00.056 INFO:teuthology.orchestra.run.trial148.stdout:(21/35): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2026-02-17T05:48:00.057 INFO:teuthology.orchestra.run.trial148.stdout:(22/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 292 kB 00:00 2026-02-17T05:48:00.071 INFO:teuthology.orchestra.run.trial148.stdout:(23/35): qatlib-25.08.0-2.el9.x86_64.rpm 4.2 MB/s | 240 kB 00:00 2026-02-17T05:48:00.082 INFO:teuthology.orchestra.run.trial148.stdout:(24/35): qatlib-service-25.08.0-2.el9.x86_64.rp 1.4 MB/s | 37 kB 00:00 2026-02-17T05:48:00.085 INFO:teuthology.orchestra.run.trial148.stdout:(25/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.4 MB/s | 66 kB 00:00 2026-02-17T05:48:00.187 INFO:teuthology.orchestra.run.trial148.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 244 kB/s | 25 kB 00:00 2026-02-17T05:48:00.225 INFO:teuthology.orchestra.run.trial148.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2026-02-17T05:48:00.232 INFO:teuthology.orchestra.run.trial148.stdout:(28/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.9 MB/s | 308 kB 00:00 2026-02-17T05:48:00.245 INFO:teuthology.orchestra.run.trial148.stdout:(29/35): libunwind-1.6.2-1.el9.x86_64.rpm 3.3 MB/s | 67 kB 00:00 2026-02-17T05:48:00.273 INFO:teuthology.orchestra.run.trial148.stdout:(30/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 20 MB/s | 838 kB 00:00 2026-02-17T05:48:00.284 INFO:teuthology.orchestra.run.trial148.stdout:(31/35): re2-20211101-20.el9.x86_64.rpm 4.8 MB/s | 191 kB 00:00 2026-02-17T05:48:00.316 INFO:teuthology.orchestra.run.trial148.stdout:(32/35): thrift-0.15.0-4.el9.x86_64.rpm 37 MB/s | 1.6 MB 00:00 2026-02-17T05:48:00.414 INFO:teuthology.orchestra.run.trial128.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-17T05:48:00.489 INFO:teuthology.orchestra.run.trial016.stdout:(1/35): ceph-base-19.2.3-705.g04cf6dd4.el9.x86_ 15 MB/s | 5.5 MB 00:00 2026-02-17T05:48:00.538 INFO:teuthology.orchestra.run.trial016.stdout:(2/35): ceph-selinux-19.2.3-705.g04cf6dd4.el9.x 510 kB/s | 25 kB 00:00 2026-02-17T05:48:00.612 INFO:teuthology.orchestra.run.trial148.stdout:(33/35): librbd1-19.2.3-705.g04cf6dd4.el9.x86_6 11 MB/s | 3.2 MB 00:00 2026-02-17T05:48:00.640 INFO:teuthology.orchestra.run.trial016.stdout:(3/35): libcephfs2-19.2.3-705.g04cf6dd4.el9.x86 9.6 MB/s | 1.0 MB 00:00 2026-02-17T05:48:00.684 INFO:teuthology.orchestra.run.trial016.stdout:(4/35): libradosstriper1-19.2.3-705.g04cf6dd4.e 11 MB/s | 503 kB 00:00 2026-02-17T05:48:00.707 INFO:teuthology.orchestra.run.trial148.stdout:(34/35): librados2-19.2.3-705.g04cf6dd4.el9.x86 8.1 MB/s | 3.4 MB 00:00 2026-02-17T05:48:00.933 INFO:teuthology.orchestra.run.trial016.stdout:(5/35): librgw2-19.2.3-705.g04cf6dd4.el9.x86_64 22 MB/s | 5.4 MB 00:00 2026-02-17T05:48:00.950 INFO:teuthology.orchestra.run.trial016.stdout:(6/35): ceph-radosgw-19.2.3-705.g04cf6dd4.el9.x 13 MB/s | 11 MB 00:00 2026-02-17T05:48:00.951 INFO:teuthology.orchestra.run.trial016.stdout:(7/35): python3-ceph-argparse-19.2.3-705.g04cf6 2.5 MB/s | 45 kB 00:00 2026-02-17T05:48:00.962 INFO:teuthology.orchestra.run.trial016.stdout:(8/35): python3-ceph-common-19.2.3-705.g04cf6dd 12 MB/s | 154 kB 00:00 2026-02-17T05:48:00.971 INFO:teuthology.orchestra.run.trial016.stdout:(9/35): python3-cephfs-19.2.3-705.g04cf6dd4.el9 8.3 MB/s | 165 kB 00:00 2026-02-17T05:48:00.986 INFO:teuthology.orchestra.run.trial016.stdout:(10/35): python3-rados-19.2.3-705.g04cf6dd4.el9 13 MB/s | 323 kB 00:00 2026-02-17T05:48:00.995 INFO:teuthology.orchestra.run.trial016.stdout:(11/35): python3-rbd-19.2.3-705.g04cf6dd4.el9.x 12 MB/s | 303 kB 00:00 2026-02-17T05:48:01.006 INFO:teuthology.orchestra.run.trial016.stdout:(12/35): mailcap-2.1.49-5.el9.noarch.rpm 3.0 MB/s | 33 kB 00:00 2026-02-17T05:48:01.007 INFO:teuthology.orchestra.run.trial016.stdout:(13/35): python3-rgw-19.2.3-705.g04cf6dd4.el9.x 4.7 MB/s | 100 kB 00:00 2026-02-17T05:48:01.021 INFO:teuthology.orchestra.run.trial016.stdout:(14/35): pciutils-3.7.0-7.el9.x86_64.rpm 6.6 MB/s | 93 kB 00:00 2026-02-17T05:48:01.026 INFO:teuthology.orchestra.run.trial016.stdout:(15/35): nvme-cli-2.13-1.el9.x86_64.rpm 56 MB/s | 1.1 MB 00:00 2026-02-17T05:48:01.026 INFO:teuthology.orchestra.run.trial016.stdout:(16/35): boost-program-options-1.75.0-13.el9.x8 18 MB/s | 104 kB 00:00 2026-02-17T05:48:01.032 INFO:teuthology.orchestra.run.trial016.stdout:(17/35): libnbd-1.20.3-4.el9.x86_64.rpm 27 MB/s | 164 kB 00:00 2026-02-17T05:48:01.032 INFO:teuthology.orchestra.run.trial016.stdout:(18/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 27 MB/s | 160 kB 00:00 2026-02-17T05:48:01.036 INFO:teuthology.orchestra.run.trial016.stdout:(19/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 9.9 MB/s | 45 kB 00:00 2026-02-17T05:48:01.043 INFO:teuthology.orchestra.run.trial016.stdout:(20/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 50 MB/s | 292 kB 00:00 2026-02-17T05:48:01.049 INFO:teuthology.orchestra.run.trial016.stdout:(21/35): qatlib-25.08.0-2.el9.x86_64.rpm 38 MB/s | 240 kB 00:00 2026-02-17T05:48:01.052 INFO:teuthology.orchestra.run.trial016.stdout:(22/35): librdkafka-1.6.1-102.el9.x86_64.rpm 34 MB/s | 662 kB 00:00 2026-02-17T05:48:01.053 INFO:teuthology.orchestra.run.trial016.stdout:(23/35): qatlib-service-25.08.0-2.el9.x86_64.rp 8.5 MB/s | 37 kB 00:00 2026-02-17T05:48:01.057 INFO:teuthology.orchestra.run.trial016.stdout:(24/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 14 MB/s | 66 kB 00:00 2026-02-17T05:48:01.132 INFO:teuthology.orchestra.run.trial016.stdout:(25/35): ceph-common-19.2.3-705.g04cf6dd4.el9.x 22 MB/s | 22 MB 00:01 2026-02-17T05:48:01.547 INFO:teuthology.orchestra.run.trial016.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 60 kB/s | 25 kB 00:00 2026-02-17T05:48:01.548 INFO:teuthology.orchestra.run.trial148.stdout:(35/35): libarrow-9.0.0-15.el9.x86_64.rpm 3.0 MB/s | 4.4 MB 00:01 2026-02-17T05:48:01.552 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:01.552 INFO:teuthology.orchestra.run.trial148.stdout:Total 21 MB/s | 63 MB 00:02 2026-02-17T05:48:01.633 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:01.634 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:01.634 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repository Size 2026-02-17T05:48:01.634 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:01.634 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:01.634 INFO:teuthology.orchestra.run.trial128.stdout: ceph-radosgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 11 M 2026-02-17T05:48:01.634 INFO:teuthology.orchestra.run.trial128.stdout:Upgrading: 2026-02-17T05:48:01.634 INFO:teuthology.orchestra.run.trial128.stdout: librados2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.4 M 2026-02-17T05:48:01.634 INFO:teuthology.orchestra.run.trial128.stdout: librbd1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.2 M 2026-02-17T05:48:01.634 INFO:teuthology.orchestra.run.trial128.stdout:Installing dependencies: 2026-02-17T05:48:01.634 INFO:teuthology.orchestra.run.trial128.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: ceph-base x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.5 M 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 22 M 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: ceph-selinux x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 25 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: libcephfs2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.0 M 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: libradosstriper1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 503 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: librgw2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.4 M 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: python3-ceph-argparse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 45 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: python3-ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 154 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: python3-cephfs x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 165 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: python3-rados x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 323 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: python3-rbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 303 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: python3-rgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 100 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout:Installing weak dependencies: 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-17T05:48:01.635 INFO:teuthology.orchestra.run.trial128.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-17T05:48:01.636 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:01.636 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:01.636 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:01.636 INFO:teuthology.orchestra.run.trial128.stdout:Install 33 Packages 2026-02-17T05:48:01.636 INFO:teuthology.orchestra.run.trial128.stdout:Upgrade 2 Packages 2026-02-17T05:48:01.636 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:01.636 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 63 M 2026-02-17T05:48:01.636 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:01.646 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:01.664 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:01.664 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:01.711 INFO:teuthology.orchestra.run.trial016.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 298 kB/s | 49 kB 00:00 2026-02-17T05:48:01.796 INFO:teuthology.orchestra.run.trial016.stdout:(28/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 414 kB/s | 308 kB 00:00 2026-02-17T05:48:01.875 INFO:teuthology.orchestra.run.trial016.stdout:(29/35): libunwind-1.6.2-1.el9.x86_64.rpm 411 kB/s | 67 kB 00:00 2026-02-17T05:48:01.900 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:01.901 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:02.040 INFO:teuthology.orchestra.run.trial016.stdout:(30/35): re2-20211101-20.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2026-02-17T05:48:02.044 INFO:teuthology.orchestra.run.trial016.stdout:(31/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2026-02-17T05:48:02.050 INFO:teuthology.orchestra.run.trial016.stdout:(32/35): libarrow-9.0.0-15.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:00 2026-02-17T05:48:02.191 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:02.199 INFO:teuthology.orchestra.run.trial148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-17T05:48:02.201 INFO:teuthology.orchestra.run.trial148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-17T05:48:02.212 INFO:teuthology.orchestra.run.trial148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-17T05:48:02.253 INFO:teuthology.orchestra.run.trial128.stdout:(1/35): ceph-base-19.2.3-705.g04cf6dd4.el9.x86_ 16 MB/s | 5.5 MB 00:00 2026-02-17T05:48:02.309 INFO:teuthology.orchestra.run.trial128.stdout:(2/35): ceph-selinux-19.2.3-705.g04cf6dd4.el9.x 446 kB/s | 25 kB 00:00 2026-02-17T05:48:02.364 INFO:teuthology.orchestra.run.trial148.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-17T05:48:02.369 INFO:teuthology.orchestra.run.trial148.stdout: Upgrading : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:02.397 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:02.441 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:48:02.441 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:02.453 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:02.454 INFO:teuthology.orchestra.run.trial016.stdout:(33/35): thrift-0.15.0-4.el9.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2026-02-17T05:48:02.458 INFO:teuthology.orchestra.run.trial148.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-17T05:48:02.468 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 9/37 2026-02-17T05:48:02.492 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:48:02.492 INFO:teuthology.orchestra.run.trial016.stdout:(34/35): librados2-19.2.3-705.g04cf6dd4.el9.x86 7.6 MB/s | 3.4 MB 00:00 2026-02-17T05:48:02.493 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:02.504 INFO:teuthology.orchestra.run.trial016.stdout:(35/35): librbd1-19.2.3-705.g04cf6dd4.el9.x86_6 7.0 MB/s | 3.2 MB 00:00 2026-02-17T05:48:02.506 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:02.507 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:02.507 INFO:teuthology.orchestra.run.trial016.stdout:Total 22 MB/s | 63 MB 00:02 2026-02-17T05:48:02.538 INFO:teuthology.orchestra.run.trial148.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-17T05:48:02.541 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-17T05:48:02.549 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-17T05:48:02.551 INFO:teuthology.orchestra.run.trial148.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-17T05:48:02.554 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-17T05:48:02.559 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-17T05:48:02.589 INFO:teuthology.orchestra.run.trial128.stdout:(3/35): ceph-radosgw-19.2.3-705.g04cf6dd4.el9.x 16 MB/s | 11 MB 00:00 2026-02-17T05:48:02.605 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:02.616 INFO:teuthology.orchestra.run.trial128.stdout:(4/35): libradosstriper1-19.2.3-705.g04cf6dd4.e 19 MB/s | 503 kB 00:00 2026-02-17T05:48:02.623 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:02.623 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:02.703 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:48:02.704 INFO:teuthology.orchestra.run.trial148.stdout: Upgrading : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:48:02.729 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:48:02.732 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 20/37 2026-02-17T05:48:02.738 INFO:teuthology.orchestra.run.trial148.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-17T05:48:02.754 INFO:teuthology.orchestra.run.trial128.stdout:(5/35): libcephfs2-19.2.3-705.g04cf6dd4.el9.x86 2.2 MB/s | 1.0 MB 00:00 2026-02-17T05:48:02.783 INFO:teuthology.orchestra.run.trial128.stdout:(6/35): python3-ceph-argparse-19.2.3-705.g04cf6 1.5 MB/s | 45 kB 00:00 2026-02-17T05:48:02.816 INFO:teuthology.orchestra.run.trial128.stdout:(7/35): python3-ceph-common-19.2.3-705.g04cf6dd 4.7 MB/s | 154 kB 00:00 2026-02-17T05:48:02.833 INFO:teuthology.orchestra.run.trial128.stdout:(8/35): librgw2-19.2.3-705.g04cf6dd4.el9.x86_64 25 MB/s | 5.4 MB 00:00 2026-02-17T05:48:02.845 INFO:teuthology.orchestra.run.trial128.stdout:(9/35): python3-cephfs-19.2.3-705.g04cf6dd4.el9 5.6 MB/s | 165 kB 00:00 2026-02-17T05:48:02.857 INFO:teuthology.orchestra.run.trial128.stdout:(10/35): python3-rados-19.2.3-705.g04cf6dd4.el9 13 MB/s | 323 kB 00:00 2026-02-17T05:48:02.861 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:02.862 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:02.871 INFO:teuthology.orchestra.run.trial128.stdout:(11/35): python3-rbd-19.2.3-705.g04cf6dd4.el9.x 11 MB/s | 303 kB 00:00 2026-02-17T05:48:02.879 INFO:teuthology.orchestra.run.trial128.stdout:(12/35): python3-rgw-19.2.3-705.g04cf6dd4.el9.x 4.4 MB/s | 100 kB 00:00 2026-02-17T05:48:02.916 INFO:teuthology.orchestra.run.trial128.stdout:(13/35): ceph-common-19.2.3-705.g04cf6dd4.el9.x 22 MB/s | 22 MB 00:01 2026-02-17T05:48:02.917 INFO:teuthology.orchestra.run.trial128.stdout:(14/35): mailcap-2.1.49-5.el9.noarch.rpm 731 kB/s | 33 kB 00:00 2026-02-17T05:48:02.950 INFO:teuthology.orchestra.run.trial128.stdout:(15/35): pciutils-3.7.0-7.el9.x86_64.rpm 2.7 MB/s | 93 kB 00:00 2026-02-17T05:48:02.967 INFO:teuthology.orchestra.run.trial148.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-17T05:48:02.968 INFO:teuthology.orchestra.run.trial148.stdout: Installing : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:48:02.982 INFO:teuthology.orchestra.run.trial128.stdout:(16/35): nvme-cli-2.13-1.el9.x86_64.rpm 11 MB/s | 1.1 MB 00:00 2026-02-17T05:48:02.984 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:48:02.988 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 24/37 2026-02-17T05:48:02.995 INFO:teuthology.orchestra.run.trial148.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-17T05:48:02.996 INFO:teuthology.orchestra.run.trial164.stdout:lab-extras 2.8 MB/s | 50 kB 00:00 2026-02-17T05:48:02.997 INFO:teuthology.orchestra.run.trial148.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-17T05:48:02.998 INFO:teuthology.orchestra.run.trial148.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:48:03.016 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:48:03.128 INFO:teuthology.orchestra.run.trial148.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-17T05:48:03.131 INFO:teuthology.orchestra.run.trial148.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:48:03.159 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:03.168 INFO:teuthology.orchestra.run.trial016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-17T05:48:03.170 INFO:teuthology.orchestra.run.trial016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-17T05:48:03.182 INFO:teuthology.orchestra.run.trial016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-17T05:48:03.337 INFO:teuthology.orchestra.run.trial016.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-17T05:48:03.341 INFO:teuthology.orchestra.run.trial016.stdout: Upgrading : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:03.369 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:03.412 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:48:03.413 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:03.414 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:48:03.414 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T05:48:03.414 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:03.424 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:03.428 INFO:teuthology.orchestra.run.trial016.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-17T05:48:03.437 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 9/37 2026-02-17T05:48:03.453 INFO:teuthology.orchestra.run.trial148.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-17T05:48:03.454 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 31/37 2026-02-17T05:48:03.460 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:48:03.460 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:03.473 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:03.509 INFO:teuthology.orchestra.run.trial016.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-17T05:48:03.513 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-17T05:48:03.521 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-17T05:48:03.523 INFO:teuthology.orchestra.run.trial016.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-17T05:48:03.526 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-17T05:48:03.531 INFO:teuthology.orchestra.run.trial128.stdout:(17/35): libnbd-1.20.3-4.el9.x86_64.rpm 283 kB/s | 164 kB 00:00 2026-02-17T05:48:03.531 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-17T05:48:03.566 INFO:teuthology.orchestra.run.trial128.stdout:(18/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 274 kB/s | 160 kB 00:00 2026-02-17T05:48:03.617 INFO:teuthology.orchestra.run.trial128.stdout:(19/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 526 kB/s | 45 kB 00:00 2026-02-17T05:48:03.660 INFO:teuthology.orchestra.run.trial128.stdout:(20/35): boost-program-options-1.75.0-13.el9.x8 140 kB/s | 104 kB 00:00 2026-02-17T05:48:03.675 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:48:03.676 INFO:teuthology.orchestra.run.trial016.stdout: Upgrading : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:48:03.701 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:48:03.703 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 20/37 2026-02-17T05:48:03.710 INFO:teuthology.orchestra.run.trial016.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-17T05:48:03.786 INFO:teuthology.orchestra.run.trial128.stdout:(21/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2026-02-17T05:48:03.817 INFO:teuthology.orchestra.run.trial128.stdout:(22/35): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2026-02-17T05:48:03.872 INFO:teuthology.orchestra.run.trial128.stdout:(23/35): qatlib-service-25.08.0-2.el9.x86_64.rp 432 kB/s | 37 kB 00:00 2026-02-17T05:48:03.907 INFO:teuthology.orchestra.run.trial128.stdout:(24/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 739 kB/s | 66 kB 00:00 2026-02-17T05:48:03.939 INFO:teuthology.orchestra.run.trial016.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-17T05:48:03.940 INFO:teuthology.orchestra.run.trial016.stdout: Installing : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:48:03.956 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:48:03.959 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 24/37 2026-02-17T05:48:03.964 INFO:teuthology.orchestra.run.trial016.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-17T05:48:03.966 INFO:teuthology.orchestra.run.trial016.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-17T05:48:03.967 INFO:teuthology.orchestra.run.trial016.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:48:03.985 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:48:04.083 INFO:teuthology.orchestra.run.trial128.stdout:(25/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.4 MB/s | 308 kB 00:00 2026-02-17T05:48:04.097 INFO:teuthology.orchestra.run.trial016.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-17T05:48:04.099 INFO:teuthology.orchestra.run.trial016.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:48:04.105 INFO:teuthology.orchestra.run.trial128.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.1 MB/s | 25 kB 00:00 2026-02-17T05:48:04.128 INFO:teuthology.orchestra.run.trial128.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2026-02-17T05:48:04.150 INFO:teuthology.orchestra.run.trial128.stdout:(28/35): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2026-02-17T05:48:04.186 INFO:teuthology.orchestra.run.trial128.stdout:(29/35): libarrow-9.0.0-15.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2026-02-17T05:48:04.196 INFO:teuthology.orchestra.run.trial128.stdout:(30/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 18 MB/s | 838 kB 00:00 2026-02-17T05:48:04.212 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:04.213 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:04.213 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repository Size 2026-02-17T05:48:04.213 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: ceph-radosgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 11 M 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout:Upgrading: 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: librados2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.4 M 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: librbd1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.2 M 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout:Installing dependencies: 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: ceph-base x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.5 M 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 22 M 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: ceph-selinux x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 25 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: libcephfs2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.0 M 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: libradosstriper1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 503 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: librgw2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.4 M 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: python3-ceph-argparse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 45 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: python3-ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 154 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: python3-cephfs x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 165 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: python3-rados x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 323 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: python3-rbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 303 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: python3-rgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 100 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout:Installing weak dependencies: 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-17T05:48:04.214 INFO:teuthology.orchestra.run.trial164.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-17T05:48:04.215 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:04.215 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:04.215 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:04.215 INFO:teuthology.orchestra.run.trial164.stdout:Install 33 Packages 2026-02-17T05:48:04.215 INFO:teuthology.orchestra.run.trial164.stdout:Upgrade 2 Packages 2026-02-17T05:48:04.215 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:04.215 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 63 M 2026-02-17T05:48:04.215 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:04.216 INFO:teuthology.orchestra.run.trial128.stdout:(31/35): re2-20211101-20.el9.x86_64.rpm 6.3 MB/s | 191 kB 00:00 2026-02-17T05:48:04.243 INFO:teuthology.orchestra.run.trial128.stdout:(32/35): thrift-0.15.0-4.el9.x86_64.rpm 34 MB/s | 1.6 MB 00:00 2026-02-17T05:48:04.342 INFO:teuthology.orchestra.run.trial128.stdout:(33/35): librados2-19.2.3-705.g04cf6dd4.el9.x86 27 MB/s | 3.4 MB 00:00 2026-02-17T05:48:04.378 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:48:04.378 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T05:48:04.378 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:04.382 INFO:teuthology.orchestra.run.trial128.stdout:(34/35): librbd1-19.2.3-705.g04cf6dd4.el9.x86_6 23 MB/s | 3.2 MB 00:00 2026-02-17T05:48:04.417 INFO:teuthology.orchestra.run.trial016.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-17T05:48:04.419 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 31/37 2026-02-17T05:48:04.457 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:04.459 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:04.552 INFO:teuthology.orchestra.run.trial128.stdout:(35/35): qatlib-25.08.0-2.el9.x86_64.rpm 268 kB/s | 240 kB 00:00 2026-02-17T05:48:04.560 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:04.560 INFO:teuthology.orchestra.run.trial128.stdout:Total 22 MB/s | 63 MB 00:02 2026-02-17T05:48:04.663 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:04.681 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:04.681 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:04.746 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:04.749 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:48:04.791 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:48:04.791 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-17T05:48:04.791 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-17T05:48:04.791 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:04.795 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:04.875 INFO:teuthology.orchestra.run.trial164.stdout:(1/35): ceph-base-19.2.3-705.g04cf6dd4.el9.x86_ 12 MB/s | 5.5 MB 00:00 2026-02-17T05:48:04.919 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:04.919 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:04.924 INFO:teuthology.orchestra.run.trial164.stdout:(2/35): ceph-selinux-19.2.3-705.g04cf6dd4.el9.x 512 kB/s | 25 kB 00:00 2026-02-17T05:48:05.030 INFO:teuthology.orchestra.run.trial164.stdout:(3/35): ceph-radosgw-19.2.3-705.g04cf6dd4.el9.x 18 MB/s | 11 MB 00:00 2026-02-17T05:48:05.031 INFO:teuthology.orchestra.run.trial164.stdout:(4/35): libcephfs2-19.2.3-705.g04cf6dd4.el9.x86 9.1 MB/s | 1.0 MB 00:00 2026-02-17T05:48:05.069 INFO:teuthology.orchestra.run.trial164.stdout:(5/35): libradosstriper1-19.2.3-705.g04cf6dd4.e 13 MB/s | 503 kB 00:00 2026-02-17T05:48:05.119 INFO:teuthology.orchestra.run.trial164.stdout:(6/35): python3-ceph-argparse-19.2.3-705.g04cf6 911 kB/s | 45 kB 00:00 2026-02-17T05:48:05.173 INFO:teuthology.orchestra.run.trial164.stdout:(7/35): python3-ceph-common-19.2.3-705.g04cf6dd 2.8 MB/s | 154 kB 00:00 2026-02-17T05:48:05.207 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:05.215 INFO:teuthology.orchestra.run.trial128.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-17T05:48:05.218 INFO:teuthology.orchestra.run.trial128.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-17T05:48:05.229 INFO:teuthology.orchestra.run.trial128.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-17T05:48:05.233 INFO:teuthology.orchestra.run.trial164.stdout:(8/35): python3-cephfs-19.2.3-705.g04cf6dd4.el9 2.7 MB/s | 165 kB 00:00 2026-02-17T05:48:05.289 INFO:teuthology.orchestra.run.trial164.stdout:(9/35): ceph-common-19.2.3-705.g04cf6dd4.el9.x8 25 MB/s | 22 MB 00:00 2026-02-17T05:48:05.290 INFO:teuthology.orchestra.run.trial164.stdout:(10/35): python3-rados-19.2.3-705.g04cf6dd4.el9 5.5 MB/s | 323 kB 00:00 2026-02-17T05:48:05.297 INFO:teuthology.orchestra.run.trial164.stdout:(11/35): librgw2-19.2.3-705.g04cf6dd4.el9.x86_6 20 MB/s | 5.4 MB 00:00 2026-02-17T05:48:05.302 INFO:teuthology.orchestra.run.trial164.stdout:(12/35): python3-rbd-19.2.3-705.g04cf6dd4.el9.x 23 MB/s | 303 kB 00:00 2026-02-17T05:48:05.305 INFO:teuthology.orchestra.run.trial164.stdout:(13/35): python3-rgw-19.2.3-705.g04cf6dd4.el9.x 6.5 MB/s | 100 kB 00:00 2026-02-17T05:48:05.327 INFO:teuthology.orchestra.run.trial164.stdout:(14/35): mailcap-2.1.49-5.el9.noarch.rpm 1.1 MB/s | 33 kB 00:00 2026-02-17T05:48:05.345 INFO:teuthology.orchestra.run.trial164.stdout:(15/35): pciutils-3.7.0-7.el9.x86_64.rpm 2.3 MB/s | 93 kB 00:00 2026-02-17T05:48:05.367 INFO:teuthology.orchestra.run.trial164.stdout:(16/35): nvme-cli-2.13-1.el9.x86_64.rpm 17 MB/s | 1.1 MB 00:00 2026-02-17T05:48:05.378 INFO:teuthology.orchestra.run.trial128.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-17T05:48:05.383 INFO:teuthology.orchestra.run.trial128.stdout: Upgrading : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:05.413 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:05.418 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:05.420 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:05.457 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:48:05.458 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:05.469 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:05.473 INFO:teuthology.orchestra.run.trial128.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-17T05:48:05.482 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 9/37 2026-02-17T05:48:05.504 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:48:05.505 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:05.519 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:05.558 INFO:teuthology.orchestra.run.trial128.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-17T05:48:05.561 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-17T05:48:05.567 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-17T05:48:05.569 INFO:teuthology.orchestra.run.trial128.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-17T05:48:05.572 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-17T05:48:05.577 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-17T05:48:05.706 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:05.709 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:48:05.715 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:48:05.717 INFO:teuthology.orchestra.run.trial128.stdout: Upgrading : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:48:05.745 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:48:05.748 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 20/37 2026-02-17T05:48:05.754 INFO:teuthology.orchestra.run.trial128.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-17T05:48:05.754 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:48:05.754 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-17T05:48:05.754 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-17T05:48:05.754 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:05.757 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:05.946 INFO:teuthology.orchestra.run.trial164.stdout:(17/35): boost-program-options-1.75.0-13.el9.x8 168 kB/s | 104 kB 00:00 2026-02-17T05:48:05.978 INFO:teuthology.orchestra.run.trial128.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-17T05:48:05.979 INFO:teuthology.orchestra.run.trial128.stdout: Installing : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:48:05.996 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:48:05.999 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 24/37 2026-02-17T05:48:06.004 INFO:teuthology.orchestra.run.trial128.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-17T05:48:06.006 INFO:teuthology.orchestra.run.trial128.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-17T05:48:06.007 INFO:teuthology.orchestra.run.trial128.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:48:06.024 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:48:06.043 INFO:teuthology.orchestra.run.trial164.stdout:(18/35): libnbd-1.20.3-4.el9.x86_64.rpm 235 kB/s | 164 kB 00:00 2026-02-17T05:48:06.048 INFO:teuthology.orchestra.run.trial164.stdout:(19/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 442 kB/s | 45 kB 00:00 2026-02-17T05:48:06.135 INFO:teuthology.orchestra.run.trial128.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-17T05:48:06.137 INFO:teuthology.orchestra.run.trial128.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:48:06.232 INFO:teuthology.orchestra.run.trial164.stdout:(20/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2026-02-17T05:48:06.283 INFO:teuthology.orchestra.run.trial164.stdout:(21/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 175 kB/s | 160 kB 00:00 2026-02-17T05:48:06.336 INFO:teuthology.orchestra.run.trial164.stdout:(22/35): qatlib-25.08.0-2.el9.x86_64.rpm 2.3 MB/s | 240 kB 00:00 2026-02-17T05:48:06.414 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:48:06.414 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T05:48:06.414 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:06.438 INFO:teuthology.orchestra.run.trial164.stdout:(23/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 650 kB/s | 66 kB 00:00 2026-02-17T05:48:06.451 INFO:teuthology.orchestra.run.trial128.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-17T05:48:06.453 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 31/37 2026-02-17T05:48:06.465 INFO:teuthology.orchestra.run.trial164.stdout:(24/35): qatlib-service-25.08.0-2.el9.x86_64.rp 204 kB/s | 37 kB 00:00 2026-02-17T05:48:06.642 INFO:teuthology.orchestra.run.trial164.stdout:(25/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.5 MB/s | 308 kB 00:00 2026-02-17T05:48:06.676 INFO:teuthology.orchestra.run.trial164.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 736 kB/s | 25 kB 00:00 2026-02-17T05:48:06.710 INFO:teuthology.orchestra.run.trial164.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2026-02-17T05:48:06.744 INFO:teuthology.orchestra.run.trial164.stdout:(28/35): libunwind-1.6.2-1.el9.x86_64.rpm 1.9 MB/s | 67 kB 00:00 2026-02-17T05:48:06.803 INFO:teuthology.orchestra.run.trial164.stdout:(29/35): libarrow-9.0.0-15.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2026-02-17T05:48:06.812 INFO:teuthology.orchestra.run.trial164.stdout:(30/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2026-02-17T05:48:06.833 INFO:teuthology.orchestra.run.trial164.stdout:(31/35): re2-20211101-20.el9.x86_64.rpm 6.1 MB/s | 191 kB 00:00 2026-02-17T05:48:06.879 INFO:teuthology.orchestra.run.trial164.stdout:(32/35): librdkafka-1.6.1-102.el9.x86_64.rpm 792 kB/s | 662 kB 00:00 2026-02-17T05:48:06.935 INFO:teuthology.orchestra.run.trial164.stdout:(33/35): librados2-19.2.3-705.g04cf6dd4.el9.x86 34 MB/s | 3.4 MB 00:00 2026-02-17T05:48:06.937 INFO:teuthology.orchestra.run.trial164.stdout:(34/35): thrift-0.15.0-4.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2026-02-17T05:48:07.003 INFO:teuthology.orchestra.run.trial164.stdout:(35/35): librbd1-19.2.3-705.g04cf6dd4.el9.x86_6 26 MB/s | 3.2 MB 00:00 2026-02-17T05:48:07.006 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:07.006 INFO:teuthology.orchestra.run.trial164.stdout:Total 23 MB/s | 63 MB 00:02 2026-02-17T05:48:07.105 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:07.122 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:07.122 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:07.359 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:07.360 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:07.438 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:07.441 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:07.651 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:07.659 INFO:teuthology.orchestra.run.trial164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-17T05:48:07.661 INFO:teuthology.orchestra.run.trial164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-17T05:48:07.673 INFO:teuthology.orchestra.run.trial164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-17T05:48:07.726 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:07.729 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:48:07.769 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:48:07.769 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-17T05:48:07.769 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-17T05:48:07.770 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:07.773 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:07.823 INFO:teuthology.orchestra.run.trial164.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-17T05:48:07.827 INFO:teuthology.orchestra.run.trial164.stdout: Upgrading : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:07.854 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:07.896 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:48:07.897 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:07.909 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:07.913 INFO:teuthology.orchestra.run.trial164.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-17T05:48:07.923 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 9/37 2026-02-17T05:48:07.947 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:48:07.952 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:07.960 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:07.992 INFO:teuthology.orchestra.run.trial164.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-17T05:48:07.995 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-17T05:48:08.002 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-17T05:48:08.004 INFO:teuthology.orchestra.run.trial164.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-17T05:48:08.007 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-17T05:48:08.012 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-17T05:48:08.151 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:48:08.152 INFO:teuthology.orchestra.run.trial164.stdout: Upgrading : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:48:08.177 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:48:08.181 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 20/37 2026-02-17T05:48:08.187 INFO:teuthology.orchestra.run.trial164.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-17T05:48:08.411 INFO:teuthology.orchestra.run.trial164.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-17T05:48:08.412 INFO:teuthology.orchestra.run.trial164.stdout: Installing : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:48:08.430 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:48:08.433 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 24/37 2026-02-17T05:48:08.438 INFO:teuthology.orchestra.run.trial164.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-17T05:48:08.440 INFO:teuthology.orchestra.run.trial164.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-17T05:48:08.441 INFO:teuthology.orchestra.run.trial164.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:48:08.460 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:48:08.571 INFO:teuthology.orchestra.run.trial164.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-17T05:48:08.573 INFO:teuthology.orchestra.run.trial164.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:48:08.856 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:48:08.856 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T05:48:08.857 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:08.897 INFO:teuthology.orchestra.run.trial164.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-17T05:48:08.899 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 31/37 2026-02-17T05:48:09.315 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:09.315 INFO:teuthology.orchestra.run.trial148.stdout:skipping the directory /sys 2026-02-17T05:48:09.315 INFO:teuthology.orchestra.run.trial148.stdout:skipping the directory /proc 2026-02-17T05:48:09.315 INFO:teuthology.orchestra.run.trial148.stdout:skipping the directory /mnt 2026-02-17T05:48:09.315 INFO:teuthology.orchestra.run.trial148.stdout:skipping the directory /var/tmp 2026-02-17T05:48:09.315 INFO:teuthology.orchestra.run.trial148.stdout:skipping the directory /home 2026-02-17T05:48:09.315 INFO:teuthology.orchestra.run.trial148.stdout:skipping the directory /root 2026-02-17T05:48:09.315 INFO:teuthology.orchestra.run.trial148.stdout:skipping the directory /tmp 2026-02-17T05:48:09.315 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:09.775 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:48:09.798 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:48:09.798 INFO:teuthology.orchestra.run.trial148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:09.798 INFO:teuthology.orchestra.run.trial148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-17T05:48:09.798 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:48:09.798 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:48:09.798 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:09.799 INFO:teuthology.orchestra.run.trial148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:48:09.811 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:48:09.811 INFO:teuthology.orchestra.run.trial148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:09.864 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:09.865 INFO:teuthology.orchestra.run.trial148.stdout:Creating group 'qat' with GID 992. 2026-02-17T05:48:09.865 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:09.884 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:09.887 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:10.135 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/37 2026-02-17T05:48:10.136 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/37 2026-02-17T05:48:10.136 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/37 2026-02-17T05:48:10.136 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/37 2026-02-17T05:48:10.136 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:10.136 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:48:10.136 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:10.136 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 8/37 2026-02-17T05:48:10.136 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 9/37 2026-02-17T05:48:10.136 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:48:10.136 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 12/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 13/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 15/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-17T05:48:10.137 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-17T05:48:10.138 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 36/37 2026-02-17T05:48:10.169 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:48:10.173 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:48:10.195 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:10.195 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:10.195 INFO:teuthology.orchestra.run.trial148.stdout:Upgraded: 2026-02-17T05:48:10.195 INFO:teuthology.orchestra.run.trial148.stdout: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.196 INFO:teuthology.orchestra.run.trial148.stdout: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.196 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:10.196 INFO:teuthology.orchestra.run.trial148.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-17T05:48:10.196 INFO:teuthology.orchestra.run.trial148.stdout: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.196 INFO:teuthology.orchestra.run.trial148.stdout: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.196 INFO:teuthology.orchestra.run.trial148.stdout: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.197 INFO:teuthology.orchestra.run.trial148.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.198 INFO:teuthology.orchestra.run.trial148.stdout: python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.199 INFO:teuthology.orchestra.run.trial148.stdout: python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:10.199 INFO:teuthology.orchestra.run.trial148.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-17T05:48:10.199 INFO:teuthology.orchestra.run.trial148.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-17T05:48:10.199 INFO:teuthology.orchestra.run.trial148.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-17T05:48:10.199 INFO:teuthology.orchestra.run.trial148.stdout: re2-1:20211101-20.el9.x86_64 2026-02-17T05:48:10.199 INFO:teuthology.orchestra.run.trial148.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-17T05:48:10.199 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:10.199 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:10.214 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:48:10.214 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-17T05:48:10.214 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-17T05:48:10.214 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:10.217 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:10.236 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-test 2026-02-17T05:48:10.262 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:10.262 INFO:teuthology.orchestra.run.trial016.stdout:skipping the directory /sys 2026-02-17T05:48:10.262 INFO:teuthology.orchestra.run.trial016.stdout:skipping the directory /proc 2026-02-17T05:48:10.262 INFO:teuthology.orchestra.run.trial016.stdout:skipping the directory /mnt 2026-02-17T05:48:10.262 INFO:teuthology.orchestra.run.trial016.stdout:skipping the directory /var/tmp 2026-02-17T05:48:10.262 INFO:teuthology.orchestra.run.trial016.stdout:skipping the directory /home 2026-02-17T05:48:10.262 INFO:teuthology.orchestra.run.trial016.stdout:skipping the directory /root 2026-02-17T05:48:10.262 INFO:teuthology.orchestra.run.trial016.stdout:skipping the directory /tmp 2026-02-17T05:48:10.263 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:10.551 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:13 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:10.638 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:10.638 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repository Size 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout: ceph-test x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 50 M 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout:Installing dependencies: 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout: libcephsqlite x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 163 k 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout:Install 5 Packages 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 50 M 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 212 M 2026-02-17T05:48:10.639 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:10.722 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:48:10.746 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:48:10.746 INFO:teuthology.orchestra.run.trial016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:10.747 INFO:teuthology.orchestra.run.trial016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-17T05:48:10.747 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:48:10.747 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:48:10.747 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:10.748 INFO:teuthology.orchestra.run.trial016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:48:10.762 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:48:10.762 INFO:teuthology.orchestra.run.trial016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:10.815 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:10.815 INFO:teuthology.orchestra.run.trial016.stdout:Creating group 'qat' with GID 992. 2026-02-17T05:48:10.815 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:10.827 INFO:teuthology.orchestra.run.trial148.stdout:(1/5): libcephsqlite-19.2.3-705.g04cf6dd4.el9.x 1.6 MB/s | 163 kB 00:00 2026-02-17T05:48:11.083 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/37 2026-02-17T05:48:11.083 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/37 2026-02-17T05:48:11.083 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/37 2026-02-17T05:48:11.083 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 8/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 9/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 12/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 13/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 15/37 2026-02-17T05:48:11.084 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-17T05:48:11.085 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-17T05:48:11.086 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-17T05:48:11.086 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-17T05:48:11.086 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-17T05:48:11.086 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-17T05:48:11.086 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-17T05:48:11.086 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-17T05:48:11.086 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:11.086 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-17T05:48:11.086 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 36/37 2026-02-17T05:48:11.144 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:11.145 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:11.145 INFO:teuthology.orchestra.run.trial016.stdout:Upgraded: 2026-02-17T05:48:11.145 INFO:teuthology.orchestra.run.trial016.stdout: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.145 INFO:teuthology.orchestra.run.trial016.stdout: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.145 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:11.145 INFO:teuthology.orchestra.run.trial016.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-17T05:48:11.145 INFO:teuthology.orchestra.run.trial016.stdout: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.146 INFO:teuthology.orchestra.run.trial016.stdout: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.146 INFO:teuthology.orchestra.run.trial016.stdout: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.146 INFO:teuthology.orchestra.run.trial016.stdout: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.146 INFO:teuthology.orchestra.run.trial016.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-17T05:48:11.146 INFO:teuthology.orchestra.run.trial016.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-17T05:48:11.146 INFO:teuthology.orchestra.run.trial016.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-17T05:48:11.146 INFO:teuthology.orchestra.run.trial016.stdout: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.147 INFO:teuthology.orchestra.run.trial016.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-17T05:48:11.147 INFO:teuthology.orchestra.run.trial016.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-17T05:48:11.147 INFO:teuthology.orchestra.run.trial016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-17T05:48:11.147 INFO:teuthology.orchestra.run.trial016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-17T05:48:11.147 INFO:teuthology.orchestra.run.trial016.stdout: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.147 INFO:teuthology.orchestra.run.trial016.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-17T05:48:11.147 INFO:teuthology.orchestra.run.trial016.stdout: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.147 INFO:teuthology.orchestra.run.trial016.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-17T05:48:11.147 INFO:teuthology.orchestra.run.trial016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-17T05:48:11.148 INFO:teuthology.orchestra.run.trial016.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-17T05:48:11.148 INFO:teuthology.orchestra.run.trial016.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-17T05:48:11.148 INFO:teuthology.orchestra.run.trial016.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-17T05:48:11.148 INFO:teuthology.orchestra.run.trial016.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-17T05:48:11.148 INFO:teuthology.orchestra.run.trial016.stdout: python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.148 INFO:teuthology.orchestra.run.trial016.stdout: python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.148 INFO:teuthology.orchestra.run.trial016.stdout: python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.148 INFO:teuthology.orchestra.run.trial016.stdout: python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.148 INFO:teuthology.orchestra.run.trial016.stdout: python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.149 INFO:teuthology.orchestra.run.trial016.stdout: python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:11.149 INFO:teuthology.orchestra.run.trial016.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-17T05:48:11.149 INFO:teuthology.orchestra.run.trial016.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-17T05:48:11.149 INFO:teuthology.orchestra.run.trial016.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-17T05:48:11.149 INFO:teuthology.orchestra.run.trial016.stdout: re2-1:20211101-20.el9.x86_64 2026-02-17T05:48:11.149 INFO:teuthology.orchestra.run.trial016.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-17T05:48:11.149 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:11.149 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:11.186 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-test 2026-02-17T05:48:11.492 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:13 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repository Size 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout: ceph-test x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 50 M 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout:Installing dependencies: 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout: libcephsqlite x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 163 k 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-17T05:48:11.574 INFO:teuthology.orchestra.run.trial016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-17T05:48:11.575 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:11.575 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:11.575 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:11.575 INFO:teuthology.orchestra.run.trial016.stdout:Install 5 Packages 2026-02-17T05:48:11.575 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:11.575 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 50 M 2026-02-17T05:48:11.575 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 212 M 2026-02-17T05:48:11.575 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:11.627 INFO:teuthology.orchestra.run.trial148.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 259 kB/s | 233 kB 00:00 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial016.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 4.6 MB/s | 233 kB 00:00 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial016.stdout:(2/5): socat-1.7.4.1-8.el9.x86_64.rpm 17 MB/s | 303 kB 00:00 2026-02-17T05:48:11.738 INFO:teuthology.orchestra.run.trial016.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 7.1 MB/s | 64 kB 00:00 2026-02-17T05:48:11.754 INFO:teuthology.orchestra.run.trial148.stdout:(3/5): ceph-test-19.2.3-705.g04cf6dd4.el9.x86_6 48 MB/s | 50 MB 00:01 2026-02-17T05:48:11.758 INFO:teuthology.orchestra.run.trial016.stdout:(4/5): libcephsqlite-19.2.3-705.g04cf6dd4.el9.x 1.6 MB/s | 163 kB 00:00 2026-02-17T05:48:11.813 INFO:teuthology.orchestra.run.trial148.stdout:(4/5): socat-1.7.4.1-8.el9.x86_64.rpm 308 kB/s | 303 kB 00:00 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial148.stdout:(5/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 219 kB/s | 64 kB 00:00 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial148.stdout:Total 39 MB/s | 50 MB 00:01 2026-02-17T05:48:11.950 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:11.959 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:11.960 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:12.146 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:12.146 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:12.224 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:12.225 INFO:teuthology.orchestra.run.trial128.stdout:skipping the directory /sys 2026-02-17T05:48:12.225 INFO:teuthology.orchestra.run.trial128.stdout:skipping the directory /proc 2026-02-17T05:48:12.225 INFO:teuthology.orchestra.run.trial128.stdout:skipping the directory /mnt 2026-02-17T05:48:12.225 INFO:teuthology.orchestra.run.trial128.stdout:skipping the directory /var/tmp 2026-02-17T05:48:12.225 INFO:teuthology.orchestra.run.trial128.stdout:skipping the directory /home 2026-02-17T05:48:12.225 INFO:teuthology.orchestra.run.trial128.stdout:skipping the directory /root 2026-02-17T05:48:12.225 INFO:teuthology.orchestra.run.trial128.stdout:skipping the directory /tmp 2026-02-17T05:48:12.225 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:12.357 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:12.363 INFO:teuthology.orchestra.run.trial148.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-17T05:48:12.367 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-17T05:48:12.375 INFO:teuthology.orchestra.run.trial148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-17T05:48:12.376 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:48:12.680 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:48:12.682 INFO:teuthology.orchestra.run.trial016.stdout:(5/5): ceph-test-19.2.3-705.g04cf6dd4.el9.x86_6 49 MB/s | 50 MB 00:01 2026-02-17T05:48:12.683 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:12.683 INFO:teuthology.orchestra.run.trial016.stdout:Total 45 MB/s | 50 MB 00:01 2026-02-17T05:48:12.702 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:48:12.703 INFO:teuthology.orchestra.run.trial128.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:12.703 INFO:teuthology.orchestra.run.trial128.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-17T05:48:12.703 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:48:12.703 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:48:12.703 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:12.704 INFO:teuthology.orchestra.run.trial128.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:48:12.709 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:12.718 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:48:12.718 INFO:teuthology.orchestra.run.trial128.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:12.718 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:12.718 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:12.775 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:12.775 INFO:teuthology.orchestra.run.trial128.stdout:Creating group 'qat' with GID 992. 2026-02-17T05:48:12.776 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:12.905 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:12.905 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:13.045 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/37 2026-02-17T05:48:13.045 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/37 2026-02-17T05:48:13.045 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/37 2026-02-17T05:48:13.045 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/37 2026-02-17T05:48:13.045 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:13.045 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:48:13.045 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:13.045 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 8/37 2026-02-17T05:48:13.045 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 9/37 2026-02-17T05:48:13.045 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 12/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 13/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 15/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-17T05:48:13.046 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 36/37 2026-02-17T05:48:13.107 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:13.107 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:13.107 INFO:teuthology.orchestra.run.trial128.stdout:Upgraded: 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-17T05:48:13.108 INFO:teuthology.orchestra.run.trial128.stdout: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-17T05:48:13.109 INFO:teuthology.orchestra.run.trial128.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: re2-1:20211101-20.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-17T05:48:13.110 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:13.111 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:13.119 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:13.125 INFO:teuthology.orchestra.run.trial016.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:48:13.148 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-test 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:13 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repository Size 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial128.stdout: ceph-test x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 50 M 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout:Installing dependencies: 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout: libcephsqlite x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 163 k 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout:Install 5 Packages 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 50 M 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 212 M 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial128.stdout:(1/5): libcephsqlite-19.2.3-705.g04cf6dd4.el9.x 1.7 MB/s | 163 kB 00:00 2026-02-17T05:48:14.102 INFO:teuthology.orchestra.run.trial128.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 466 kB/s | 233 kB 00:00 2026-02-17T05:48:14.196 INFO:teuthology.orchestra.run.trial128.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 681 kB/s | 64 kB 00:00 2026-02-17T05:48:14.223 INFO:teuthology.orchestra.run.trial128.stdout:(4/5): socat-1.7.4.1-8.el9.x86_64.rpm 577 kB/s | 303 kB 00:00 2026-02-17T05:48:14.609 INFO:teuthology.orchestra.run.trial128.stdout:(5/5): ceph-test-19.2.3-705.g04cf6dd4.el9.x86_6 49 MB/s | 50 MB 00:01 2026-02-17T05:48:14.610 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:14.610 INFO:teuthology.orchestra.run.trial128.stdout:Total 47 MB/s | 50 MB 00:01 2026-02-17T05:48:14.632 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:48:14.635 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:14.645 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:14.645 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:14.651 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:48:14.755 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:14.756 INFO:teuthology.orchestra.run.trial164.stdout:skipping the directory /sys 2026-02-17T05:48:14.756 INFO:teuthology.orchestra.run.trial164.stdout:skipping the directory /proc 2026-02-17T05:48:14.756 INFO:teuthology.orchestra.run.trial164.stdout:skipping the directory /mnt 2026-02-17T05:48:14.756 INFO:teuthology.orchestra.run.trial164.stdout:skipping the directory /var/tmp 2026-02-17T05:48:14.756 INFO:teuthology.orchestra.run.trial164.stdout:skipping the directory /home 2026-02-17T05:48:14.756 INFO:teuthology.orchestra.run.trial164.stdout:skipping the directory /root 2026-02-17T05:48:14.756 INFO:teuthology.orchestra.run.trial164.stdout:skipping the directory /tmp 2026-02-17T05:48:14.756 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:14.831 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:14.831 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:14.884 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:48:14.884 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/5 2026-02-17T05:48:14.885 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/5 2026-02-17T05:48:14.885 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-17T05:48:14.885 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-17T05:48:14.939 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-17T05:48:14.939 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:14.939 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:14.939 INFO:teuthology.orchestra.run.trial148.stdout: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:14.939 INFO:teuthology.orchestra.run.trial148.stdout: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:14.940 INFO:teuthology.orchestra.run.trial148.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-17T05:48:14.940 INFO:teuthology.orchestra.run.trial148.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-17T05:48:14.940 INFO:teuthology.orchestra.run.trial148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-17T05:48:14.940 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:14.940 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:14.974 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph 2026-02-17T05:48:15.043 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:15.049 INFO:teuthology.orchestra.run.trial128.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-17T05:48:15.053 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-17T05:48:15.060 INFO:teuthology.orchestra.run.trial128.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-17T05:48:15.061 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:48:15.210 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:48:15.233 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:48:15.233 INFO:teuthology.orchestra.run.trial164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:15.233 INFO:teuthology.orchestra.run.trial164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-17T05:48:15.233 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:48:15.233 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:48:15.233 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:15.235 INFO:teuthology.orchestra.run.trial164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:48:15.248 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:48:15.248 INFO:teuthology.orchestra.run.trial164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:15.276 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:18 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:15.303 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:15.303 INFO:teuthology.orchestra.run.trial164.stdout:Creating group 'qat' with GID 992. 2026-02-17T05:48:15.303 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:15.363 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:15.366 INFO:teuthology.orchestra.run.trial148.stdout:====================================================================================== 2026-02-17T05:48:15.367 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repository Size 2026-02-17T05:48:15.367 INFO:teuthology.orchestra.run.trial148.stdout:====================================================================================== 2026-02-17T05:48:15.367 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:15.367 INFO:teuthology.orchestra.run.trial148.stdout: ceph x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.5 k 2026-02-17T05:48:15.367 INFO:teuthology.orchestra.run.trial148.stdout:Installing dependencies: 2026-02-17T05:48:15.367 INFO:teuthology.orchestra.run.trial148.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-17T05:48:15.367 INFO:teuthology.orchestra.run.trial148.stdout: ceph-grafana-dashboards noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 31 k 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mds x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 2.4 M 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.1 M 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-modules-core noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 253 k 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mon x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 4.7 M 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: ceph-osd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 17 M 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: ceph-prometheus-alerts noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 17 k 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 769 k 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-17T05:48:15.368 INFO:teuthology.orchestra.run.trial148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-17T05:48:15.369 INFO:teuthology.orchestra.run.trial148.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-17T05:48:15.370 INFO:teuthology.orchestra.run.trial148.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-17T05:48:15.371 INFO:teuthology.orchestra.run.trial148.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-17T05:48:15.372 INFO:teuthology.orchestra.run.trial148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-17T05:48:15.373 INFO:teuthology.orchestra.run.trial148.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-17T05:48:15.374 INFO:teuthology.orchestra.run.trial148.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout:Installing weak dependencies: 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 150 k 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-dashboard noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 3.8 M 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 7.4 M 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 22 k 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-rook noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 49 k 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout: ceph-volume noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 299 k 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:15.375 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:15.376 INFO:teuthology.orchestra.run.trial148.stdout:====================================================================================== 2026-02-17T05:48:15.376 INFO:teuthology.orchestra.run.trial148.stdout:Install 99 Packages 2026-02-17T05:48:15.376 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:15.377 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 95 M 2026-02-17T05:48:15.377 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 520 M 2026-02-17T05:48:15.377 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:15.436 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:48:15.454 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:48:15.570 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 8/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 9/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 12/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 13/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 15/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-17T05:48:15.571 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 36/37 2026-02-17T05:48:15.633 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:48:15.633 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:15.633 INFO:teuthology.orchestra.run.trial164.stdout:Upgraded: 2026-02-17T05:48:15.633 INFO:teuthology.orchestra.run.trial164.stdout: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.633 INFO:teuthology.orchestra.run.trial164.stdout: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.633 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:15.633 INFO:teuthology.orchestra.run.trial164.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-17T05:48:15.633 INFO:teuthology.orchestra.run.trial164.stdout: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.633 INFO:teuthology.orchestra.run.trial164.stdout: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.634 INFO:teuthology.orchestra.run.trial164.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.635 INFO:teuthology.orchestra.run.trial164.stdout: python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.636 INFO:teuthology.orchestra.run.trial164.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-17T05:48:15.636 INFO:teuthology.orchestra.run.trial164.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-17T05:48:15.636 INFO:teuthology.orchestra.run.trial164.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-17T05:48:15.636 INFO:teuthology.orchestra.run.trial164.stdout: re2-1:20211101-20.el9.x86_64 2026-02-17T05:48:15.636 INFO:teuthology.orchestra.run.trial164.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-17T05:48:15.636 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:15.636 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:15.666 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:48:15.666 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/5 2026-02-17T05:48:15.666 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/5 2026-02-17T05:48:15.666 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-17T05:48:15.666 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-17T05:48:15.674 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-test 2026-02-17T05:48:15.719 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-17T05:48:15.719 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:15.719 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:15.719 INFO:teuthology.orchestra.run.trial016.stdout: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.719 INFO:teuthology.orchestra.run.trial016.stdout: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:15.719 INFO:teuthology.orchestra.run.trial016.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-17T05:48:15.719 INFO:teuthology.orchestra.run.trial016.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-17T05:48:15.719 INFO:teuthology.orchestra.run.trial016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-17T05:48:15.719 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:15.719 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:15.721 INFO:teuthology.orchestra.run.trial148.stdout:(1/99): ceph-19.2.3-705.g04cf6dd4.el9.x86_64.rp 121 kB/s | 6.5 kB 00:00 2026-02-17T05:48:15.754 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph 2026-02-17T05:48:15.784 INFO:teuthology.orchestra.run.trial148.stdout:(2/99): ceph-mgr-19.2.3-705.g04cf6dd4.el9.x86_6 9.3 MB/s | 1.1 MB 00:00 2026-02-17T05:48:15.850 INFO:teuthology.orchestra.run.trial148.stdout:(3/99): ceph-mds-19.2.3-705.g04cf6dd4.el9.x86_6 13 MB/s | 2.4 MB 00:00 2026-02-17T05:48:15.885 INFO:teuthology.orchestra.run.trial148.stdout:(4/99): ceph-grafana-dashboards-19.2.3-705.g04c 877 kB/s | 31 kB 00:00 2026-02-17T05:48:15.925 INFO:teuthology.orchestra.run.trial148.stdout:(5/99): ceph-mgr-cephadm-19.2.3-705.g04cf6dd4.e 3.7 MB/s | 150 kB 00:00 2026-02-17T05:48:15.986 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:13 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:16.020 INFO:teuthology.orchestra.run.trial148.stdout:(6/99): ceph-mon-19.2.3-705.g04cf6dd4.el9.x86_6 16 MB/s | 4.7 MB 00:00 2026-02-17T05:48:16.063 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:18 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:16.069 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:16.069 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:16.069 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repository Size 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout: ceph-test x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 50 M 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout:Installing dependencies: 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout: libcephsqlite x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 163 k 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:16.070 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:16.071 INFO:teuthology.orchestra.run.trial164.stdout:Install 5 Packages 2026-02-17T05:48:16.071 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:16.071 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 50 M 2026-02-17T05:48:16.071 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 212 M 2026-02-17T05:48:16.071 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:16.112 INFO:teuthology.orchestra.run.trial148.stdout:(7/99): ceph-mgr-dashboard-19.2.3-705.g04cf6dd4 20 MB/s | 3.8 MB 00:00 2026-02-17T05:48:16.148 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:16.151 INFO:teuthology.orchestra.run.trial016.stdout:====================================================================================== 2026-02-17T05:48:16.151 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repository Size 2026-02-17T05:48:16.152 INFO:teuthology.orchestra.run.trial016.stdout:====================================================================================== 2026-02-17T05:48:16.152 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:16.152 INFO:teuthology.orchestra.run.trial016.stdout: ceph x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.5 k 2026-02-17T05:48:16.152 INFO:teuthology.orchestra.run.trial016.stdout:Installing dependencies: 2026-02-17T05:48:16.152 INFO:teuthology.orchestra.run.trial016.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-17T05:48:16.152 INFO:teuthology.orchestra.run.trial016.stdout: ceph-grafana-dashboards noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 31 k 2026-02-17T05:48:16.152 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mds x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 2.4 M 2026-02-17T05:48:16.152 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.1 M 2026-02-17T05:48:16.152 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-modules-core noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 253 k 2026-02-17T05:48:16.152 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mon x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 4.7 M 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: ceph-osd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 17 M 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: ceph-prometheus-alerts noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 17 k 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 769 k 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-17T05:48:16.153 INFO:teuthology.orchestra.run.trial016.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-17T05:48:16.154 INFO:teuthology.orchestra.run.trial016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-17T05:48:16.155 INFO:teuthology.orchestra.run.trial016.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-17T05:48:16.156 INFO:teuthology.orchestra.run.trial016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-17T05:48:16.157 INFO:teuthology.orchestra.run.trial016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-17T05:48:16.157 INFO:teuthology.orchestra.run.trial016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-17T05:48:16.157 INFO:teuthology.orchestra.run.trial016.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-17T05:48:16.157 INFO:teuthology.orchestra.run.trial016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-17T05:48:16.157 INFO:teuthology.orchestra.run.trial016.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-17T05:48:16.157 INFO:teuthology.orchestra.run.trial016.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-17T05:48:16.157 INFO:teuthology.orchestra.run.trial016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-17T05:48:16.157 INFO:teuthology.orchestra.run.trial016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-17T05:48:16.157 INFO:teuthology.orchestra.run.trial016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-17T05:48:16.157 INFO:teuthology.orchestra.run.trial016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-17T05:48:16.158 INFO:teuthology.orchestra.run.trial016.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-17T05:48:16.158 INFO:teuthology.orchestra.run.trial016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-17T05:48:16.158 INFO:teuthology.orchestra.run.trial016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-17T05:48:16.158 INFO:teuthology.orchestra.run.trial016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-17T05:48:16.158 INFO:teuthology.orchestra.run.trial016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-17T05:48:16.158 INFO:teuthology.orchestra.run.trial016.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-17T05:48:16.158 INFO:teuthology.orchestra.run.trial016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-17T05:48:16.158 INFO:teuthology.orchestra.run.trial016.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-17T05:48:16.158 INFO:teuthology.orchestra.run.trial016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-17T05:48:16.158 INFO:teuthology.orchestra.run.trial016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-17T05:48:16.159 INFO:teuthology.orchestra.run.trial016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-17T05:48:16.159 INFO:teuthology.orchestra.run.trial016.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-17T05:48:16.159 INFO:teuthology.orchestra.run.trial016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-17T05:48:16.159 INFO:teuthology.orchestra.run.trial016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-17T05:48:16.159 INFO:teuthology.orchestra.run.trial016.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-17T05:48:16.159 INFO:teuthology.orchestra.run.trial016.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-17T05:48:16.159 INFO:teuthology.orchestra.run.trial016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-17T05:48:16.159 INFO:teuthology.orchestra.run.trial016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-17T05:48:16.159 INFO:teuthology.orchestra.run.trial016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-17T05:48:16.159 INFO:teuthology.orchestra.run.trial016.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-17T05:48:16.160 INFO:teuthology.orchestra.run.trial016.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-17T05:48:16.160 INFO:teuthology.orchestra.run.trial016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-17T05:48:16.160 INFO:teuthology.orchestra.run.trial016.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-17T05:48:16.160 INFO:teuthology.orchestra.run.trial016.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-17T05:48:16.160 INFO:teuthology.orchestra.run.trial016.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-17T05:48:16.160 INFO:teuthology.orchestra.run.trial016.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-17T05:48:16.160 INFO:teuthology.orchestra.run.trial016.stdout:Installing weak dependencies: 2026-02-17T05:48:16.160 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 150 k 2026-02-17T05:48:16.160 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-dashboard noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 3.8 M 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 7.4 M 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 22 k 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-rook noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 49 k 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout: ceph-volume noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 299 k 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout:====================================================================================== 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout:Install 99 Packages 2026-02-17T05:48:16.161 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:16.163 INFO:teuthology.orchestra.run.trial148.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-705.g04cf6dd4 486 kB/s | 22 kB 00:00 2026-02-17T05:48:16.163 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 95 M 2026-02-17T05:48:16.163 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 520 M 2026-02-17T05:48:16.163 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:16.209 INFO:teuthology.orchestra.run.trial148.stdout:(9/99): ceph-mgr-modules-core-19.2.3-705.g04cf6 4.8 MB/s | 253 kB 00:00 2026-02-17T05:48:16.260 INFO:teuthology.orchestra.run.trial148.stdout:(10/99): ceph-mgr-rook-19.2.3-705.g04cf6dd4.el9 974 kB/s | 49 kB 00:00 2026-02-17T05:48:16.312 INFO:teuthology.orchestra.run.trial148.stdout:(11/99): ceph-prometheus-alerts-19.2.3-705.g04c 322 kB/s | 17 kB 00:00 2026-02-17T05:48:16.387 INFO:teuthology.orchestra.run.trial148.stdout:(12/99): ceph-osd-19.2.3-705.g04cf6dd4.el9.x86_ 28 MB/s | 17 MB 00:00 2026-02-17T05:48:16.389 INFO:teuthology.orchestra.run.trial148.stdout:(13/99): ceph-volume-19.2.3-705.g04cf6dd4.el9.n 3.8 MB/s | 299 kB 00:00 2026-02-17T05:48:16.398 INFO:teuthology.orchestra.run.trial148.stdout:(14/99): ceph-mgr-diskprediction-local-19.2.3-7 20 MB/s | 7.4 MB 00:00 2026-02-17T05:48:16.403 INFO:teuthology.orchestra.run.trial148.stdout:(15/99): cephadm-19.2.3-705.g04cf6dd4.el9.noarc 49 MB/s | 769 kB 00:00 2026-02-17T05:48:16.468 INFO:teuthology.orchestra.run.trial164.stdout:(1/5): libcephsqlite-19.2.3-705.g04cf6dd4.el9.x 1.8 MB/s | 163 kB 00:00 2026-02-17T05:48:16.535 INFO:teuthology.orchestra.run.trial164.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 1.5 MB/s | 233 kB 00:00 2026-02-17T05:48:16.564 INFO:teuthology.orchestra.run.trial164.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.2 MB/s | 64 kB 00:00 2026-02-17T05:48:16.612 INFO:teuthology.orchestra.run.trial016.stdout:(1/99): ceph-19.2.3-705.g04cf6dd4.el9.x86_64.rp 51 kB/s | 6.5 kB 00:00 2026-02-17T05:48:16.630 INFO:teuthology.orchestra.run.trial164.stdout:(4/5): socat-1.7.4.1-8.el9.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2026-02-17T05:48:16.891 INFO:teuthology.orchestra.run.trial148.stdout:(16/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 82 kB/s | 40 kB 00:00 2026-02-17T05:48:17.014 INFO:teuthology.orchestra.run.trial148.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 118 kB/s | 72 kB 00:00 2026-02-17T05:48:17.261 INFO:teuthology.orchestra.run.trial016.stdout:(2/99): ceph-mgr-19.2.3-705.g04cf6dd4.el9.x86_6 1.4 MB/s | 1.1 MB 00:00 2026-02-17T05:48:17.312 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:48:17.331 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:48:17.332 INFO:teuthology.orchestra.run.trial148.stdout:(18/99): cryptsetup-2.8.1-2.el9.x86_64.rpm 371 kB/s | 350 kB 00:00 2026-02-17T05:48:17.453 INFO:teuthology.orchestra.run.trial164.stdout:(5/5): ceph-test-19.2.3-705.g04cf6dd4.el9.x86_6 46 MB/s | 50 MB 00:01 2026-02-17T05:48:17.455 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:17.455 INFO:teuthology.orchestra.run.trial164.stdout:Total 36 MB/s | 50 MB 00:01 2026-02-17T05:48:17.456 INFO:teuthology.orchestra.run.trial016.stdout:(3/99): ceph-mds-19.2.3-705.g04cf6dd4.el9.x86_6 2.5 MB/s | 2.4 MB 00:00 2026-02-17T05:48:17.468 INFO:teuthology.orchestra.run.trial016.stdout:(4/99): ceph-grafana-dashboards-19.2.3-705.g04c 2.7 MB/s | 31 kB 00:00 2026-02-17T05:48:17.479 INFO:teuthology.orchestra.run.trial016.stdout:(5/99): ceph-mgr-cephadm-19.2.3-705.g04cf6dd4.e 13 MB/s | 150 kB 00:00 2026-02-17T05:48:17.481 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:17.490 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:17.490 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:17.542 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:48:17.542 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/5 2026-02-17T05:48:17.542 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/5 2026-02-17T05:48:17.542 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-17T05:48:17.542 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-17T05:48:17.594 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-17T05:48:17.594 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:17.594 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:17.594 INFO:teuthology.orchestra.run.trial128.stdout: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:17.595 INFO:teuthology.orchestra.run.trial128.stdout: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:17.595 INFO:teuthology.orchestra.run.trial128.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-17T05:48:17.595 INFO:teuthology.orchestra.run.trial128.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-17T05:48:17.595 INFO:teuthology.orchestra.run.trial128.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-17T05:48:17.595 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:17.595 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:17.627 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph 2026-02-17T05:48:17.665 INFO:teuthology.orchestra.run.trial016.stdout:(6/99): ceph-mgr-dashboard-19.2.3-705.g04cf6dd4 21 MB/s | 3.8 MB 00:00 2026-02-17T05:48:17.677 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:17.677 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:17.680 INFO:teuthology.orchestra.run.trial148.stdout:(19/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 728 kB/s | 253 kB 00:00 2026-02-17T05:48:17.696 INFO:teuthology.orchestra.run.trial016.stdout:(7/99): ceph-mon-19.2.3-705.g04cf6dd4.el9.x86_6 4.4 MB/s | 4.7 MB 00:01 2026-02-17T05:48:17.711 INFO:teuthology.orchestra.run.trial016.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-705.g04cf6dd4 1.5 MB/s | 22 kB 00:00 2026-02-17T05:48:17.731 INFO:teuthology.orchestra.run.trial016.stdout:(9/99): ceph-mgr-modules-core-19.2.3-705.g04cf6 12 MB/s | 253 kB 00:00 2026-02-17T05:48:17.749 INFO:teuthology.orchestra.run.trial016.stdout:(10/99): ceph-mgr-rook-19.2.3-705.g04cf6dd4.el9 2.7 MB/s | 49 kB 00:00 2026-02-17T05:48:17.761 INFO:teuthology.orchestra.run.trial148.stdout:(20/99): libquadmath-11.5.0-14.el9.x86_64.rpm 247 kB/s | 184 kB 00:00 2026-02-17T05:48:17.767 INFO:teuthology.orchestra.run.trial016.stdout:(11/99): ceph-prometheus-alerts-19.2.3-705.g04c 917 kB/s | 17 kB 00:00 2026-02-17T05:48:17.811 INFO:teuthology.orchestra.run.trial016.stdout:(12/99): ceph-volume-19.2.3-705.g04cf6dd4.el9.n 6.8 MB/s | 299 kB 00:00 2026-02-17T05:48:17.888 INFO:teuthology.orchestra.run.trial016.stdout:(13/99): cephadm-19.2.3-705.g04cf6dd4.el9.noarc 9.7 MB/s | 769 kB 00:00 2026-02-17T05:48:17.889 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:17.894 INFO:teuthology.orchestra.run.trial164.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-17T05:48:17.898 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-17T05:48:17.907 INFO:teuthology.orchestra.run.trial164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-17T05:48:17.907 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:48:17.934 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:17 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:17.943 INFO:teuthology.orchestra.run.trial016.stdout:(14/99): cryptsetup-2.8.1-2.el9.x86_64.rpm 6.3 MB/s | 350 kB 00:00 2026-02-17T05:48:17.968 INFO:teuthology.orchestra.run.trial016.stdout:(15/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.6 MB/s | 40 kB 00:00 2026-02-17T05:48:17.972 INFO:teuthology.orchestra.run.trial148.stdout:(21/99): libgfortran-11.5.0-14.el9.x86_64.rpm 735 kB/s | 794 kB 00:01 2026-02-17T05:48:17.983 INFO:teuthology.orchestra.run.trial016.stdout:(16/99): ceph-mgr-diskprediction-local-19.2.3-7 23 MB/s | 7.4 MB 00:00 2026-02-17T05:48:17.993 INFO:teuthology.orchestra.run.trial016.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 2.8 MB/s | 72 kB 00:00 2026-02-17T05:48:18.019 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout:====================================================================================== 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repository Size 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout:====================================================================================== 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: ceph x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.5 k 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout:Installing dependencies: 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: ceph-grafana-dashboards noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 31 k 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mds x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 2.4 M 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.1 M 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-modules-core noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 253 k 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mon x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 4.7 M 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: ceph-osd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 17 M 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: ceph-prometheus-alerts noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 17 k 2026-02-17T05:48:18.022 INFO:teuthology.orchestra.run.trial128.stdout: cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 769 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-17T05:48:18.023 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-17T05:48:18.024 INFO:teuthology.orchestra.run.trial128.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout:Installing weak dependencies: 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 150 k 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-dashboard noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 3.8 M 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 7.4 M 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 22 k 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-rook noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 49 k 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: ceph-volume noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 299 k 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout:====================================================================================== 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout:Install 99 Packages 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 95 M 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 520 M 2026-02-17T05:48:18.025 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:18.030 INFO:teuthology.orchestra.run.trial016.stdout:(18/99): ceph-osd-19.2.3-705.g04cf6dd4.el9.x86_ 22 MB/s | 17 MB 00:00 2026-02-17T05:48:18.031 INFO:teuthology.orchestra.run.trial016.stdout:(19/99): libgfortran-11.5.0-14.el9.x86_64.rpm 16 MB/s | 794 kB 00:00 2026-02-17T05:48:18.031 INFO:teuthology.orchestra.run.trial016.stdout:(20/99): libquadmath-11.5.0-14.el9.x86_64.rpm 4.7 MB/s | 184 kB 00:00 2026-02-17T05:48:18.057 INFO:teuthology.orchestra.run.trial016.stdout:(21/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 9.3 MB/s | 253 kB 00:00 2026-02-17T05:48:18.057 INFO:teuthology.orchestra.run.trial016.stdout:(22/99): python3-ply-3.11-14.el9.noarch.rpm 4.1 MB/s | 106 kB 00:00 2026-02-17T05:48:18.074 INFO:teuthology.orchestra.run.trial016.stdout:(23/99): python3-cryptography-36.0.1-5.el9.x86_ 30 MB/s | 1.2 MB 00:00 2026-02-17T05:48:18.081 INFO:teuthology.orchestra.run.trial016.stdout:(24/99): python3-pycparser-2.20-6.el9.noarch.rp 5.6 MB/s | 135 kB 00:00 2026-02-17T05:48:18.086 INFO:teuthology.orchestra.run.trial016.stdout:(25/99): python3-pyparsing-2.4.7-9.el9.noarch.r 5.2 MB/s | 150 kB 00:00 2026-02-17T05:48:18.100 INFO:teuthology.orchestra.run.trial016.stdout:(26/99): python3-requests-2.25.1-10.el9.noarch. 4.6 MB/s | 126 kB 00:00 2026-02-17T05:48:18.105 INFO:teuthology.orchestra.run.trial148.stdout:(22/99): python3-ply-3.11-14.el9.noarch.rpm 310 kB/s | 106 kB 00:00 2026-02-17T05:48:18.107 INFO:teuthology.orchestra.run.trial016.stdout:(27/99): python3-urllib3-1.26.5-7.el9.noarch.rp 8.1 MB/s | 218 kB 00:00 2026-02-17T05:48:18.113 INFO:teuthology.orchestra.run.trial016.stdout:(28/99): unzip-6.0-59.el9.x86_64.rpm 6.5 MB/s | 182 kB 00:00 2026-02-17T05:48:18.121 INFO:teuthology.orchestra.run.trial016.stdout:(29/99): zip-3.0-35.el9.x86_64.rpm 13 MB/s | 266 kB 00:00 2026-02-17T05:48:18.150 INFO:teuthology.orchestra.run.trial016.stdout:(30/99): flexiblas-3.0.4-9.el9.x86_64.rpm 696 kB/s | 30 kB 00:00 2026-02-17T05:48:18.157 INFO:teuthology.orchestra.run.trial148.stdout:(23/99): python3-pycparser-2.20-6.el9.noarch.rp 731 kB/s | 135 kB 00:00 2026-02-17T05:48:18.163 INFO:teuthology.orchestra.run.trial016.stdout:(31/99): flexiblas-openblas-openmp-3.0.4-9.el9. 354 kB/s | 15 kB 00:00 2026-02-17T05:48:18.215 INFO:teuthology.orchestra.run.trial016.stdout:(32/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 3.7 MB/s | 246 kB 00:00 2026-02-17T05:48:18.227 INFO:teuthology.orchestra.run.trial016.stdout:(33/99): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 188 kB 00:00 2026-02-17T05:48:18.237 INFO:teuthology.orchestra.run.trial016.stdout:(34/99): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-17T05:48:18.285 INFO:teuthology.orchestra.run.trial016.stdout:(35/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 17 MB/s | 3.0 MB 00:00 2026-02-17T05:48:18.298 INFO:teuthology.orchestra.run.trial016.stdout:(36/99): protobuf-3.14.0-17.el9.x86_64.rpm 16 MB/s | 1.0 MB 00:00 2026-02-17T05:48:18.323 INFO:teuthology.orchestra.run.trial016.stdout:(37/99): python3-devel-3.9.25-2.el9.x86_64.rpm 9.7 MB/s | 244 kB 00:00 2026-02-17T05:48:18.342 INFO:teuthology.orchestra.run.trial148.stdout:(24/99): python3-requests-2.25.1-10.el9.noarch. 684 kB/s | 126 kB 00:00 2026-02-17T05:48:18.343 INFO:teuthology.orchestra.run.trial016.stdout:(38/99): python3-babel-2.9.1-2.el9.noarch.rpm 103 MB/s | 6.0 MB 00:00 2026-02-17T05:48:18.349 INFO:teuthology.orchestra.run.trial016.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 9.1 MB/s | 249 kB 00:00 2026-02-17T05:48:18.367 INFO:teuthology.orchestra.run.trial016.stdout:(40/99): python3-libstoragemgmt-1.10.1-1.el9.x8 7.1 MB/s | 177 kB 00:00 2026-02-17T05:48:18.377 INFO:teuthology.orchestra.run.trial016.stdout:(41/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 45 MB/s | 1.2 MB 00:00 2026-02-17T05:48:18.381 INFO:teuthology.orchestra.run.trial128.stdout:(1/99): ceph-19.2.3-705.g04cf6dd4.el9.x86_64.rp 127 kB/s | 6.5 kB 00:00 2026-02-17T05:48:18.392 INFO:teuthology.orchestra.run.trial016.stdout:(42/99): python3-mako-1.1.4-6.el9.noarch.rpm 6.9 MB/s | 172 kB 00:00 2026-02-17T05:48:18.399 INFO:teuthology.orchestra.run.trial016.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 1.5 MB/s | 35 kB 00:00 2026-02-17T05:48:18.420 INFO:teuthology.orchestra.run.trial016.stdout:(44/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 27 MB/s | 5.3 MB 00:00 2026-02-17T05:48:18.424 INFO:teuthology.orchestra.run.trial016.stdout:(45/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 18 MB/s | 442 kB 00:00 2026-02-17T05:48:18.443 INFO:teuthology.orchestra.run.trial016.stdout:(46/99): python3-packaging-20.9-5.el9.noarch.rp 3.2 MB/s | 77 kB 00:00 2026-02-17T05:48:18.448 INFO:teuthology.orchestra.run.trial016.stdout:(47/99): python3-protobuf-3.14.0-17.el9.noarch. 11 MB/s | 267 kB 00:00 2026-02-17T05:48:18.466 INFO:teuthology.orchestra.run.trial016.stdout:(48/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 82 MB/s | 6.1 MB 00:00 2026-02-17T05:48:18.468 INFO:teuthology.orchestra.run.trial016.stdout:(49/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 6.3 MB/s | 159 kB 00:00 2026-02-17T05:48:18.473 INFO:teuthology.orchestra.run.trial016.stdout:(50/99): python3-pyasn1-modules-0.4.8-6.el9.noa 11 MB/s | 279 kB 00:00 2026-02-17T05:48:18.477 INFO:teuthology.orchestra.run.trial128.stdout:(2/99): ceph-mgr-19.2.3-705.g04cf6dd4.el9.x86_6 7.3 MB/s | 1.1 MB 00:00 2026-02-17T05:48:18.489 INFO:teuthology.orchestra.run.trial016.stdout:(51/99): python3-requests-oauthlib-1.3.0-12.el9 2.3 MB/s | 54 kB 00:00 2026-02-17T05:48:18.490 INFO:teuthology.orchestra.run.trial128.stdout:(3/99): ceph-mds-19.2.3-705.g04cf6dd4.el9.x86_6 15 MB/s | 2.4 MB 00:00 2026-02-17T05:48:18.495 INFO:teuthology.orchestra.run.trial016.stdout:(52/99): python3-toml-0.10.2-6.el9.noarch.rpm 1.8 MB/s | 42 kB 00:00 2026-02-17T05:48:18.512 INFO:teuthology.orchestra.run.trial016.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 8.0 MB/s | 189 kB 00:00 2026-02-17T05:48:18.513 INFO:teuthology.orchestra.run.trial128.stdout:(4/99): ceph-grafana-dashboards-19.2.3-705.g04c 1.4 MB/s | 31 kB 00:00 2026-02-17T05:48:18.518 INFO:teuthology.orchestra.run.trial016.stdout:(54/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.9 MB/s | 90 kB 00:00 2026-02-17T05:48:18.530 INFO:teuthology.orchestra.run.trial148.stdout:(25/99): python3-pyparsing-2.4.7-9.el9.noarch.r 354 kB/s | 150 kB 00:00 2026-02-17T05:48:18.539 INFO:teuthology.orchestra.run.trial128.stdout:(5/99): ceph-mgr-cephadm-19.2.3-705.g04cf6dd4.e 5.6 MB/s | 150 kB 00:00 2026-02-17T05:48:18.589 INFO:teuthology.orchestra.run.trial016.stdout:(55/99): lua-devel-5.4.4-4.el9.x86_64.rpm 291 kB/s | 22 kB 00:00 2026-02-17T05:48:18.629 INFO:teuthology.orchestra.run.trial016.stdout:(56/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 14 MB/s | 551 kB 00:00 2026-02-17T05:48:18.633 INFO:teuthology.orchestra.run.trial016.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 4.3 MB/s | 19 kB 00:00 2026-02-17T05:48:18.640 INFO:teuthology.orchestra.run.trial016.stdout:(58/99): luarocks-3.9.2-5.el9.noarch.rpm 25 MB/s | 151 kB 00:00 2026-02-17T05:48:18.648 INFO:teuthology.orchestra.run.trial016.stdout:(59/99): python3-asyncssh-2.13.2-5.el9.noarch.r 64 MB/s | 548 kB 00:00 2026-02-17T05:48:18.653 INFO:teuthology.orchestra.run.trial016.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 5.9 MB/s | 29 kB 00:00 2026-02-17T05:48:18.658 INFO:teuthology.orchestra.run.trial016.stdout:(61/99): python3-backports-tarfile-1.2.0-1.el9. 11 MB/s | 60 kB 00:00 2026-02-17T05:48:18.664 INFO:teuthology.orchestra.run.trial016.stdout:(62/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 7.5 MB/s | 43 kB 00:00 2026-02-17T05:48:18.670 INFO:teuthology.orchestra.run.trial016.stdout:(63/99): python3-cachetools-4.2.4-1.el9.noarch. 4.7 MB/s | 32 kB 00:00 2026-02-17T05:48:18.675 INFO:teuthology.orchestra.run.trial016.stdout:(64/99): python3-certifi-2023.05.07-4.el9.noarc 3.0 MB/s | 14 kB 00:00 2026-02-17T05:48:18.682 INFO:teuthology.orchestra.run.trial016.stdout:(65/99): python3-cheroot-10.0.1-4.el9.noarch.rp 25 MB/s | 173 kB 00:00 2026-02-17T05:48:18.685 INFO:teuthology.orchestra.run.trial128.stdout:(6/99): ceph-mgr-dashboard-19.2.3-705.g04cf6dd4 26 MB/s | 3.8 MB 00:00 2026-02-17T05:48:18.689 INFO:teuthology.orchestra.run.trial148.stdout:(26/99): python3-urllib3-1.26.5-7.el9.noarch.rp 627 kB/s | 218 kB 00:00 2026-02-17T05:48:18.695 INFO:teuthology.orchestra.run.trial016.stdout:(66/99): python3-cherrypy-18.6.1-2.el9.noarch.r 29 MB/s | 358 kB 00:00 2026-02-17T05:48:18.702 INFO:teuthology.orchestra.run.trial016.stdout:(67/99): python3-google-auth-2.45.0-1.el9.noarc 34 MB/s | 254 kB 00:00 2026-02-17T05:48:18.711 INFO:teuthology.orchestra.run.trial128.stdout:(7/99): ceph-mon-19.2.3-705.g04cf6dd4.el9.x86_6 14 MB/s | 4.7 MB 00:00 2026-02-17T05:48:18.748 INFO:teuthology.orchestra.run.trial016.stdout:(68/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 44 MB/s | 2.0 MB 00:00 2026-02-17T05:48:18.755 INFO:teuthology.orchestra.run.trial128.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-705.g04cf6dd4 504 kB/s | 22 kB 00:00 2026-02-17T05:48:18.758 INFO:teuthology.orchestra.run.trial016.stdout:(69/99): python3-grpcio-tools-1.46.7-10.el9.x86 15 MB/s | 144 kB 00:00 2026-02-17T05:48:18.765 INFO:teuthology.orchestra.run.trial016.stdout:(70/99): python3-influxdb-5.3.1-1.el9.noarch.rp 18 MB/s | 139 kB 00:00 2026-02-17T05:48:18.771 INFO:teuthology.orchestra.run.trial016.stdout:(71/99): python3-isodate-0.6.1-3.el9.noarch.rpm 10 MB/s | 56 kB 00:00 2026-02-17T05:48:18.776 INFO:teuthology.orchestra.run.trial016.stdout:(72/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 2.2 MB/s | 11 kB 00:00 2026-02-17T05:48:18.780 INFO:teuthology.orchestra.run.trial016.stdout:(73/99): python3-jaraco-classes-3.2.1-5.el9.noa 3.6 MB/s | 18 kB 00:00 2026-02-17T05:48:18.785 INFO:teuthology.orchestra.run.trial016.stdout:(74/99): python3-jaraco-collections-3.0.0-8.el9 4.6 MB/s | 23 kB 00:00 2026-02-17T05:48:18.790 INFO:teuthology.orchestra.run.trial016.stdout:(75/99): python3-jaraco-context-6.0.1-3.el9.noa 4.0 MB/s | 20 kB 00:00 2026-02-17T05:48:18.795 INFO:teuthology.orchestra.run.trial016.stdout:(76/99): python3-jaraco-functools-3.5.0-2.el9.n 4.0 MB/s | 19 kB 00:00 2026-02-17T05:48:18.800 INFO:teuthology.orchestra.run.trial016.stdout:(77/99): python3-jaraco-text-4.0.0-2.el9.noarch 5.2 MB/s | 26 kB 00:00 2026-02-17T05:48:18.809 INFO:teuthology.orchestra.run.trial128.stdout:(9/99): ceph-mgr-modules-core-19.2.3-705.g04cf6 4.6 MB/s | 253 kB 00:00 2026-02-17T05:48:18.813 INFO:teuthology.orchestra.run.trial016.stdout:(78/99): python3-kubernetes-26.1.0-3.el9.noarch 83 MB/s | 1.0 MB 00:00 2026-02-17T05:48:18.818 INFO:teuthology.orchestra.run.trial016.stdout:(79/99): python3-logutils-0.3.5-21.el9.noarch.r 8.6 MB/s | 46 kB 00:00 2026-02-17T05:48:18.822 INFO:teuthology.orchestra.run.trial016.stdout:(80/99): protobuf-compiler-3.14.0-17.el9.x86_64 2.8 MB/s | 862 kB 00:00 2026-02-17T05:48:18.825 INFO:teuthology.orchestra.run.trial016.stdout:(81/99): python3-more-itertools-8.12.0-2.el9.no 12 MB/s | 79 kB 00:00 2026-02-17T05:48:18.830 INFO:teuthology.orchestra.run.trial016.stdout:(82/99): python3-natsort-7.1.1-5.el9.noarch.rpm 11 MB/s | 58 kB 00:00 2026-02-17T05:48:18.837 INFO:teuthology.orchestra.run.trial016.stdout:(83/99): python3-pecan-1.4.2-3.el9.noarch.rpm 36 MB/s | 272 kB 00:00 2026-02-17T05:48:18.840 INFO:teuthology.orchestra.run.trial016.stdout:(84/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 4.7 MB/s | 86 kB 00:00 2026-02-17T05:48:18.842 INFO:teuthology.orchestra.run.trial016.stdout:(85/99): python3-portend-3.1.0-2.el9.noarch.rpm 3.4 MB/s | 16 kB 00:00 2026-02-17T05:48:18.846 INFO:teuthology.orchestra.run.trial016.stdout:(86/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 14 MB/s | 90 kB 00:00 2026-02-17T05:48:18.847 INFO:teuthology.orchestra.run.trial016.stdout:(87/99): python3-repoze-lru-0.7-16.el9.noarch.r 6.1 MB/s | 31 kB 00:00 2026-02-17T05:48:18.854 INFO:teuthology.orchestra.run.trial016.stdout:(88/99): python3-routes-2.5.1-5.el9.noarch.rpm 25 MB/s | 188 kB 00:00 2026-02-17T05:48:18.854 INFO:teuthology.orchestra.run.trial016.stdout:(89/99): python3-rsa-4.9-2.el9.noarch.rpm 8.0 MB/s | 59 kB 00:00 2026-02-17T05:48:18.860 INFO:teuthology.orchestra.run.trial016.stdout:(90/99): python3-tempora-5.0.0-2.el9.noarch.rpm 7.1 MB/s | 36 kB 00:00 2026-02-17T05:48:18.860 INFO:teuthology.orchestra.run.trial128.stdout:(10/99): ceph-mgr-rook-19.2.3-705.g04cf6dd4.el9 972 kB/s | 49 kB 00:00 2026-02-17T05:48:18.861 INFO:teuthology.orchestra.run.trial016.stdout:(91/99): python3-saml-1.16.0-1.el9.noarch.rpm 18 MB/s | 125 kB 00:00 2026-02-17T05:48:18.865 INFO:teuthology.orchestra.run.trial016.stdout:(92/99): python3-typing-extensions-4.15.0-1.el9 15 MB/s | 86 kB 00:00 2026-02-17T05:48:18.871 INFO:teuthology.orchestra.run.trial016.stdout:(93/99): python3-webob-1.8.8-2.el9.noarch.rpm 23 MB/s | 230 kB 00:00 2026-02-17T05:48:18.871 INFO:teuthology.orchestra.run.trial016.stdout:(94/99): python3-websocket-client-1.2.3-2.el9.n 16 MB/s | 90 kB 00:00 2026-02-17T05:48:18.881 INFO:teuthology.orchestra.run.trial016.stdout:(95/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 4.9 MB/s | 48 kB 00:00 2026-02-17T05:48:18.906 INFO:teuthology.orchestra.run.trial016.stdout:(96/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 44 MB/s | 19 MB 00:00 2026-02-17T05:48:18.907 INFO:teuthology.orchestra.run.trial016.stdout:(97/99): python3-xmltodict-0.12.0-15.el9.noarch 862 kB/s | 22 kB 00:00 2026-02-17T05:48:18.908 INFO:teuthology.orchestra.run.trial016.stdout:(98/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 11 MB/s | 427 kB 00:00 2026-02-17T05:48:18.919 INFO:teuthology.orchestra.run.trial016.stdout:(99/99): python3-zc-lockfile-2.0-10.el9.noarch. 1.6 MB/s | 20 kB 00:00 2026-02-17T05:48:18.919 INFO:teuthology.orchestra.run.trial128.stdout:(11/99): ceph-mgr-diskprediction-local-19.2.3-7 32 MB/s | 7.4 MB 00:00 2026-02-17T05:48:18.919 INFO:teuthology.orchestra.run.trial128.stdout:(12/99): ceph-prometheus-alerts-19.2.3-705.g04c 282 kB/s | 17 kB 00:00 2026-02-17T05:48:18.920 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:18.920 INFO:teuthology.orchestra.run.trial016.stdout:Total 34 MB/s | 95 MB 00:02 2026-02-17T05:48:18.950 INFO:teuthology.orchestra.run.trial128.stdout:(13/99): ceph-volume-19.2.3-705.g04cf6dd4.el9.n 9.5 MB/s | 299 kB 00:00 2026-02-17T05:48:18.955 INFO:teuthology.orchestra.run.trial148.stdout:(27/99): unzip-6.0-59.el9.x86_64.rpm 428 kB/s | 182 kB 00:00 2026-02-17T05:48:18.980 INFO:teuthology.orchestra.run.trial128.stdout:(14/99): cephadm-19.2.3-705.g04cf6dd4.el9.noarc 12 MB/s | 769 kB 00:00 2026-02-17T05:48:19.022 INFO:teuthology.orchestra.run.trial148.stdout:(28/99): flexiblas-3.0.4-9.el9.x86_64.rpm 443 kB/s | 30 kB 00:00 2026-02-17T05:48:19.037 INFO:teuthology.orchestra.run.trial148.stdout:(29/99): zip-3.0-35.el9.x86_64.rpm 765 kB/s | 266 kB 00:00 2026-02-17T05:48:19.090 INFO:teuthology.orchestra.run.trial128.stdout:(15/99): ceph-osd-19.2.3-705.g04cf6dd4.el9.x86_ 28 MB/s | 17 MB 00:00 2026-02-17T05:48:19.104 INFO:teuthology.orchestra.run.trial148.stdout:(30/99): flexiblas-openblas-openmp-3.0.4-9.el9. 222 kB/s | 15 kB 00:00 2026-02-17T05:48:19.156 INFO:teuthology.orchestra.run.trial148.stdout:(31/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 22 MB/s | 3.0 MB 00:00 2026-02-17T05:48:19.178 INFO:teuthology.orchestra.run.trial148.stdout:(32/99): lua-5.4.4-4.el9.x86_64.rpm 8.5 MB/s | 188 kB 00:00 2026-02-17T05:48:19.193 INFO:teuthology.orchestra.run.trial148.stdout:(33/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 2.7 MB/s | 246 kB 00:00 2026-02-17T05:48:19.198 INFO:teuthology.orchestra.run.trial148.stdout:(34/99): openblas-0.3.29-1.el9.x86_64.rpm 2.1 MB/s | 42 kB 00:00 2026-02-17T05:48:19.221 INFO:teuthology.orchestra.run.trial148.stdout:(35/99): protobuf-3.14.0-17.el9.x86_64.rpm 43 MB/s | 1.0 MB 00:00 2026-02-17T05:48:19.230 INFO:teuthology.orchestra.run.trial128.stdout:(16/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 162 kB/s | 40 kB 00:00 2026-02-17T05:48:19.251 INFO:teuthology.orchestra.run.trial148.stdout:(36/99): python3-cryptography-36.0.1-5.el9.x86_ 812 kB/s | 1.2 MB 00:01 2026-02-17T05:48:19.275 INFO:teuthology.orchestra.run.trial148.stdout:(37/99): python3-babel-2.9.1-2.el9.noarch.rpm 110 MB/s | 6.0 MB 00:00 2026-02-17T05:48:19.299 INFO:teuthology.orchestra.run.trial148.stdout:(38/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 10 MB/s | 249 kB 00:00 2026-02-17T05:48:19.320 INFO:teuthology.orchestra.run.trial128.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 314 kB/s | 72 kB 00:00 2026-02-17T05:48:19.321 INFO:teuthology.orchestra.run.trial148.stdout:(39/99): python3-libstoragemgmt-1.10.1-1.el9.x8 7.8 MB/s | 177 kB 00:00 2026-02-17T05:48:19.328 INFO:teuthology.orchestra.run.trial128.stdout:(18/99): cryptsetup-2.8.1-2.el9.x86_64.rpm 926 kB/s | 350 kB 00:00 2026-02-17T05:48:19.346 INFO:teuthology.orchestra.run.trial148.stdout:(40/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 48 MB/s | 1.2 MB 00:00 2026-02-17T05:48:19.347 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:19.371 INFO:teuthology.orchestra.run.trial148.stdout:(41/99): python3-mako-1.1.4-6.el9.noarch.rpm 6.9 MB/s | 172 kB 00:00 2026-02-17T05:48:19.382 INFO:teuthology.orchestra.run.trial148.stdout:(42/99): python3-devel-3.9.25-2.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2026-02-17T05:48:19.384 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:19.384 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:19.394 INFO:teuthology.orchestra.run.trial148.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 1.5 MB/s | 35 kB 00:00 2026-02-17T05:48:19.416 INFO:teuthology.orchestra.run.trial148.stdout:(44/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 20 MB/s | 442 kB 00:00 2026-02-17T05:48:19.439 INFO:teuthology.orchestra.run.trial148.stdout:(45/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 22 MB/s | 5.3 MB 00:00 2026-02-17T05:48:19.439 INFO:teuthology.orchestra.run.trial148.stdout:(46/99): python3-packaging-20.9-5.el9.noarch.rp 3.2 MB/s | 77 kB 00:00 2026-02-17T05:48:19.460 INFO:teuthology.orchestra.run.trial148.stdout:(47/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 7.4 MB/s | 159 kB 00:00 2026-02-17T05:48:19.466 INFO:teuthology.orchestra.run.trial148.stdout:(48/99): python3-protobuf-3.14.0-17.el9.noarch. 9.8 MB/s | 267 kB 00:00 2026-02-17T05:48:19.484 INFO:teuthology.orchestra.run.trial148.stdout:(49/99): python3-pyasn1-modules-0.4.8-6.el9.noa 12 MB/s | 279 kB 00:00 2026-02-17T05:48:19.490 INFO:teuthology.orchestra.run.trial148.stdout:(50/99): python3-requests-oauthlib-1.3.0-12.el9 2.2 MB/s | 54 kB 00:00 2026-02-17T05:48:19.514 INFO:teuthology.orchestra.run.trial148.stdout:(51/99): python3-toml-0.10.2-6.el9.noarch.rpm 1.7 MB/s | 42 kB 00:00 2026-02-17T05:48:19.540 INFO:teuthology.orchestra.run.trial148.stdout:(52/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 7.0 MB/s | 189 kB 00:00 2026-02-17T05:48:19.566 INFO:teuthology.orchestra.run.trial148.stdout:(53/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.5 MB/s | 90 kB 00:00 2026-02-17T05:48:19.598 INFO:teuthology.orchestra.run.trial128.stdout:(19/99): libquadmath-11.5.0-14.el9.x86_64.rpm 663 kB/s | 184 kB 00:00 2026-02-17T05:48:19.650 INFO:teuthology.orchestra.run.trial128.stdout:(20/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 787 kB/s | 253 kB 00:00 2026-02-17T05:48:19.653 INFO:teuthology.orchestra.run.trial148.stdout:(54/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 113 MB/s | 19 MB 00:00 2026-02-17T05:48:19.662 INFO:teuthology.orchestra.run.trial148.stdout:(55/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 22 MB/s | 6.1 MB 00:00 2026-02-17T05:48:19.733 INFO:teuthology.orchestra.run.trial128.stdout:(21/99): libgfortran-11.5.0-14.el9.x86_64.rpm 1.5 MB/s | 794 kB 00:00 2026-02-17T05:48:19.802 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:19.802 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:19.820 INFO:teuthology.orchestra.run.trial148.stdout:(56/99): lua-devel-5.4.4-4.el9.x86_64.rpm 88 kB/s | 22 kB 00:00 2026-02-17T05:48:19.923 INFO:teuthology.orchestra.run.trial128.stdout:(22/99): python3-ply-3.11-14.el9.noarch.rpm 391 kB/s | 106 kB 00:00 2026-02-17T05:48:20.016 INFO:teuthology.orchestra.run.trial128.stdout:(23/99): python3-pycparser-2.20-6.el9.noarch.rp 476 kB/s | 135 kB 00:00 2026-02-17T05:48:20.189 INFO:teuthology.orchestra.run.trial128.stdout:(24/99): python3-cryptography-36.0.1-5.el9.x86_ 2.1 MB/s | 1.2 MB 00:00 2026-02-17T05:48:20.195 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:48:20.201 INFO:teuthology.orchestra.run.trial128.stdout:(25/99): python3-pyparsing-2.4.7-9.el9.noarch.r 541 kB/s | 150 kB 00:00 2026-02-17T05:48:20.213 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:48:20.232 INFO:teuthology.orchestra.run.trial148.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 47 kB/s | 19 kB 00:00 2026-02-17T05:48:20.250 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:20.252 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-17T05:48:20.283 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-17T05:48:20.288 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-17T05:48:20.292 INFO:teuthology.orchestra.run.trial128.stdout:(26/99): python3-requests-2.25.1-10.el9.noarch. 459 kB/s | 126 kB 00:00 2026-02-17T05:48:20.295 INFO:teuthology.orchestra.run.trial148.stdout:(58/99): protobuf-compiler-3.14.0-17.el9.x86_64 1.3 MB/s | 862 kB 00:00 2026-02-17T05:48:20.309 INFO:teuthology.orchestra.run.trial016.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-17T05:48:20.317 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-17T05:48:20.350 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-17T05:48:20.359 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-17T05:48:20.364 INFO:teuthology.orchestra.run.trial016.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-17T05:48:20.367 INFO:teuthology.orchestra.run.trial016.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-17T05:48:20.371 INFO:teuthology.orchestra.run.trial016.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-17T05:48:20.395 INFO:teuthology.orchestra.run.trial016.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-17T05:48:20.407 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-17T05:48:20.410 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-17T05:48:20.416 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-17T05:48:20.418 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-17T05:48:20.429 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-17T05:48:20.435 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-17T05:48:20.454 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:48:20.454 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/5 2026-02-17T05:48:20.454 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/5 2026-02-17T05:48:20.455 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-17T05:48:20.455 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-17T05:48:20.459 INFO:teuthology.orchestra.run.trial016.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-17T05:48:20.462 INFO:teuthology.orchestra.run.trial016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-17T05:48:20.470 INFO:teuthology.orchestra.run.trial016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-17T05:48:20.494 INFO:teuthology.orchestra.run.trial016.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-17T05:48:20.507 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-17T05:48:20.507 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:20.507 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:20.507 INFO:teuthology.orchestra.run.trial164.stdout: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:20.507 INFO:teuthology.orchestra.run.trial164.stdout: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:20.508 INFO:teuthology.orchestra.run.trial164.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-17T05:48:20.508 INFO:teuthology.orchestra.run.trial164.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-17T05:48:20.508 INFO:teuthology.orchestra.run.trial164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-17T05:48:20.508 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:20.508 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:20.508 INFO:teuthology.orchestra.run.trial128.stdout:(27/99): python3-urllib3-1.26.5-7.el9.noarch.rp 682 kB/s | 218 kB 00:00 2026-02-17T05:48:20.522 INFO:teuthology.orchestra.run.trial128.stdout:(28/99): unzip-6.0-59.el9.x86_64.rpm 567 kB/s | 182 kB 00:00 2026-02-17T05:48:20.541 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph 2026-02-17T05:48:20.545 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-17T05:48:20.557 INFO:teuthology.orchestra.run.trial148.stdout:(59/99): luarocks-3.9.2-5.el9.noarch.rpm 467 kB/s | 151 kB 00:00 2026-02-17T05:48:20.558 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-17T05:48:20.560 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-17T05:48:20.565 INFO:teuthology.orchestra.run.trial016.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-17T05:48:20.570 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-17T05:48:20.575 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-17T05:48:20.575 INFO:teuthology.orchestra.run.trial128.stdout:(29/99): flexiblas-3.0.4-9.el9.x86_64.rpm 444 kB/s | 30 kB 00:00 2026-02-17T05:48:20.578 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-17T05:48:20.592 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-17T05:48:20.598 INFO:teuthology.orchestra.run.trial128.stdout:(30/99): flexiblas-openblas-openmp-3.0.4-9.el9. 643 kB/s | 15 kB 00:00 2026-02-17T05:48:20.613 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-17T05:48:20.618 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-17T05:48:20.619 INFO:teuthology.orchestra.run.trial128.stdout:(31/99): zip-3.0-35.el9.x86_64.rpm 811 kB/s | 266 kB 00:00 2026-02-17T05:48:20.623 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-17T05:48:20.634 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-17T05:48:20.638 INFO:teuthology.orchestra.run.trial148.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 361 kB/s | 29 kB 00:00 2026-02-17T05:48:20.643 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-17T05:48:20.649 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-17T05:48:20.669 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-17T05:48:20.678 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-17T05:48:20.687 INFO:teuthology.orchestra.run.trial128.stdout:(32/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 2.7 MB/s | 246 kB 00:00 2026-02-17T05:48:20.703 INFO:teuthology.orchestra.run.trial128.stdout:(33/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 17 MB/s | 3.0 MB 00:00 2026-02-17T05:48:20.711 INFO:teuthology.orchestra.run.trial128.stdout:(34/99): openblas-0.3.29-1.el9.x86_64.rpm 1.7 MB/s | 42 kB 00:00 2026-02-17T05:48:20.721 INFO:teuthology.orchestra.run.trial148.stdout:(61/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 521 kB/s | 551 kB 00:01 2026-02-17T05:48:20.721 INFO:teuthology.orchestra.run.trial148.stdout:(62/99): python3-backports-tarfile-1.2.0-1.el9. 729 kB/s | 60 kB 00:00 2026-02-17T05:48:20.730 INFO:teuthology.orchestra.run.trial128.stdout:(35/99): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2026-02-17T05:48:20.732 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-17T05:48:20.744 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-17T05:48:20.756 INFO:teuthology.orchestra.run.trial128.stdout:(36/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 99 MB/s | 5.3 MB 00:00 2026-02-17T05:48:20.760 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-17T05:48:20.765 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-17T05:48:20.772 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-17T05:48:20.780 INFO:teuthology.orchestra.run.trial128.stdout:(37/99): python3-devel-3.9.25-2.el9.x86_64.rpm 10 MB/s | 244 kB 00:00 2026-02-17T05:48:20.794 INFO:teuthology.orchestra.run.trial128.stdout:(38/99): protobuf-3.14.0-17.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2026-02-17T05:48:20.802 INFO:teuthology.orchestra.run.trial148.stdout:(63/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 532 kB/s | 43 kB 00:00 2026-02-17T05:48:20.803 INFO:teuthology.orchestra.run.trial148.stdout:(64/99): python3-cachetools-4.2.4-1.el9.noarch. 394 kB/s | 32 kB 00:00 2026-02-17T05:48:20.803 INFO:teuthology.orchestra.run.trial128.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 11 MB/s | 249 kB 00:00 2026-02-17T05:48:20.813 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-17T05:48:20.819 INFO:teuthology.orchestra.run.trial128.stdout:(40/99): python3-libstoragemgmt-1.10.1-1.el9.x8 6.8 MB/s | 177 kB 00:00 2026-02-17T05:48:20.829 INFO:teuthology.orchestra.run.trial128.stdout:(41/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 47 MB/s | 1.2 MB 00:00 2026-02-17T05:48:20.846 INFO:teuthology.orchestra.run.trial128.stdout:(42/99): python3-mako-1.1.4-6.el9.noarch.rpm 6.4 MB/s | 172 kB 00:00 2026-02-17T05:48:20.851 INFO:teuthology.orchestra.run.trial128.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 1.6 MB/s | 35 kB 00:00 2026-02-17T05:48:20.856 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:18 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:20.874 INFO:teuthology.orchestra.run.trial128.stdout:(44/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 18 MB/s | 442 kB 00:00 2026-02-17T05:48:20.884 INFO:teuthology.orchestra.run.trial148.stdout:(65/99): python3-certifi-2023.05.07-4.el9.noarc 173 kB/s | 14 kB 00:00 2026-02-17T05:48:20.897 INFO:teuthology.orchestra.run.trial128.stdout:(45/99): python3-packaging-20.9-5.el9.noarch.rp 3.3 MB/s | 77 kB 00:00 2026-02-17T05:48:20.920 INFO:teuthology.orchestra.run.trial128.stdout:(46/99): python3-protobuf-3.14.0-17.el9.noarch. 11 MB/s | 267 kB 00:00 2026-02-17T05:48:20.928 INFO:teuthology.orchestra.run.trial128.stdout:(47/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 75 MB/s | 6.1 MB 00:00 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial128.stdout:(48/99): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 6.0 MB 00:00 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial164.stdout:====================================================================================== 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repository Size 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial164.stdout:====================================================================================== 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial164.stdout: ceph x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.5 k 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial164.stdout:Installing dependencies: 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial164.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial164.stdout: ceph-grafana-dashboards noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 31 k 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mds x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 2.4 M 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.1 M 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-modules-core noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 253 k 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mon x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 4.7 M 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: ceph-osd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 17 M 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: ceph-prometheus-alerts noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 17 k 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 769 k 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial164.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial164.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial164.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial164.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial164.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial164.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial164.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial164.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial164.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout:Installing weak dependencies: 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 150 k 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-dashboard noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 3.8 M 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 7.4 M 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 22 k 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-rook noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 49 k 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout: ceph-volume noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 299 k 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:20.953 INFO:teuthology.orchestra.run.trial164.stdout:====================================================================================== 2026-02-17T05:48:20.954 INFO:teuthology.orchestra.run.trial164.stdout:Install 99 Packages 2026-02-17T05:48:20.954 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:20.955 INFO:teuthology.orchestra.run.trial128.stdout:(49/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 6.5 MB/s | 159 kB 00:00 2026-02-17T05:48:20.955 INFO:teuthology.orchestra.run.trial128.stdout:(50/99): python3-pyasn1-modules-0.4.8-6.el9.noa 10 MB/s | 279 kB 00:00 2026-02-17T05:48:20.955 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 95 M 2026-02-17T05:48:20.955 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 520 M 2026-02-17T05:48:20.955 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:20.967 INFO:teuthology.orchestra.run.trial128.stdout:(51/99): python3-requests-oauthlib-1.3.0-12.el9 2.0 MB/s | 54 kB 00:00 2026-02-17T05:48:20.977 INFO:teuthology.orchestra.run.trial128.stdout:(52/99): python3-toml-0.10.2-6.el9.noarch.rpm 1.7 MB/s | 42 kB 00:00 2026-02-17T05:48:20.993 INFO:teuthology.orchestra.run.trial128.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 7.2 MB/s | 189 kB 00:00 2026-02-17T05:48:21.001 INFO:teuthology.orchestra.run.trial128.stdout:(54/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.7 MB/s | 90 kB 00:00 2026-02-17T05:48:21.047 INFO:teuthology.orchestra.run.trial148.stdout:(66/99): python3-cheroot-10.0.1-4.el9.noarch.rp 712 kB/s | 173 kB 00:00 2026-02-17T05:48:21.101 INFO:teuthology.orchestra.run.trial128.stdout:(55/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 123 MB/s | 19 MB 00:00 2026-02-17T05:48:21.139 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-17T05:48:21.149 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-17T05:48:21.153 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-17T05:48:21.158 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-17T05:48:21.160 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-17T05:48:21.188 INFO:teuthology.orchestra.run.trial016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-17T05:48:21.232 INFO:teuthology.orchestra.run.trial016.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-17T05:48:21.238 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-17T05:48:21.242 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-17T05:48:21.248 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-17T05:48:21.252 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-17T05:48:21.258 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-17T05:48:21.261 INFO:teuthology.orchestra.run.trial128.stdout:(56/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 3.4 MB/s | 551 kB 00:00 2026-02-17T05:48:21.262 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-17T05:48:21.280 INFO:teuthology.orchestra.run.trial128.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 986 kB/s | 19 kB 00:00 2026-02-17T05:48:21.290 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-17T05:48:21.300 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-17T05:48:21.301 INFO:teuthology.orchestra.run.trial128.stdout:(58/99): luarocks-3.9.2-5.el9.noarch.rpm 7.3 MB/s | 151 kB 00:00 2026-02-17T05:48:21.335 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-17T05:48:21.340 INFO:teuthology.orchestra.run.trial128.stdout:(59/99): python3-asyncssh-2.13.2-5.el9.noarch.r 14 MB/s | 548 kB 00:00 2026-02-17T05:48:21.344 INFO:teuthology.orchestra.run.trial164.stdout:(1/99): ceph-19.2.3-705.g04cf6dd4.el9.x86_64.rp 118 kB/s | 6.5 kB 00:00 2026-02-17T05:48:21.360 INFO:teuthology.orchestra.run.trial128.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 1.4 MB/s | 29 kB 00:00 2026-02-17T05:48:21.362 INFO:teuthology.orchestra.run.trial148.stdout:(67/99): python3-asyncssh-2.13.2-5.el9.noarch.r 513 kB/s | 548 kB 00:01 2026-02-17T05:48:21.370 INFO:teuthology.orchestra.run.trial148.stdout:(68/99): python3-cherrypy-18.6.1-2.el9.noarch.r 737 kB/s | 358 kB 00:00 2026-02-17T05:48:21.371 INFO:teuthology.orchestra.run.trial148.stdout:(69/99): python3-google-auth-2.45.0-1.el9.noarc 783 kB/s | 254 kB 00:00 2026-02-17T05:48:21.380 INFO:teuthology.orchestra.run.trial128.stdout:(61/99): python3-backports-tarfile-1.2.0-1.el9. 3.0 MB/s | 60 kB 00:00 2026-02-17T05:48:21.418 INFO:teuthology.orchestra.run.trial164.stdout:(2/99): ceph-mgr-19.2.3-705.g04cf6dd4.el9.x86_6 8.3 MB/s | 1.1 MB 00:00 2026-02-17T05:48:21.424 INFO:teuthology.orchestra.run.trial128.stdout:(62/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 987 kB/s | 43 kB 00:00 2026-02-17T05:48:21.443 INFO:teuthology.orchestra.run.trial164.stdout:(3/99): ceph-mds-19.2.3-705.g04cf6dd4.el9.x86_6 16 MB/s | 2.4 MB 00:00 2026-02-17T05:48:21.444 INFO:teuthology.orchestra.run.trial128.stdout:(63/99): python3-cachetools-4.2.4-1.el9.noarch. 1.6 MB/s | 32 kB 00:00 2026-02-17T05:48:21.464 INFO:teuthology.orchestra.run.trial128.stdout:(64/99): python3-certifi-2023.05.07-4.el9.noarc 721 kB/s | 14 kB 00:00 2026-02-17T05:48:21.473 INFO:teuthology.orchestra.run.trial164.stdout:(4/99): ceph-grafana-dashboards-19.2.3-705.g04c 1.0 MB/s | 31 kB 00:00 2026-02-17T05:48:21.484 INFO:teuthology.orchestra.run.trial128.stdout:(65/99): python3-cheroot-10.0.1-4.el9.noarch.rp 8.3 MB/s | 173 kB 00:00 2026-02-17T05:48:21.506 INFO:teuthology.orchestra.run.trial128.stdout:(66/99): python3-cherrypy-18.6.1-2.el9.noarch.r 16 MB/s | 358 kB 00:00 2026-02-17T05:48:21.506 INFO:teuthology.orchestra.run.trial164.stdout:(5/99): ceph-mgr-cephadm-19.2.3-705.g04cf6dd4.e 4.5 MB/s | 150 kB 00:00 2026-02-17T05:48:21.526 INFO:teuthology.orchestra.run.trial128.stdout:(67/99): python3-google-auth-2.45.0-1.el9.noarc 12 MB/s | 254 kB 00:00 2026-02-17T05:48:21.534 INFO:teuthology.orchestra.run.trial148.stdout:(70/99): python3-influxdb-5.3.1-1.el9.noarch.rp 854 kB/s | 139 kB 00:00 2026-02-17T05:48:21.560 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 60/99 2026-02-17T05:48:21.570 INFO:teuthology.orchestra.run.trial128.stdout:(68/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 46 MB/s | 2.0 MB 00:00 2026-02-17T05:48:21.573 INFO:teuthology.orchestra.run.trial128.stdout:(69/99): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2026-02-17T05:48:21.584 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-17T05:48:21.588 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-17T05:48:21.591 INFO:teuthology.orchestra.run.trial128.stdout:(70/99): python3-grpcio-tools-1.46.7-10.el9.x86 7.0 MB/s | 144 kB 00:00 2026-02-17T05:48:21.596 INFO:teuthology.orchestra.run.trial128.stdout:(71/99): python3-influxdb-5.3.1-1.el9.noarch.rp 5.7 MB/s | 139 kB 00:00 2026-02-17T05:48:21.610 INFO:teuthology.orchestra.run.trial128.stdout:(72/99): python3-isodate-0.6.1-3.el9.noarch.rpm 2.8 MB/s | 56 kB 00:00 2026-02-17T05:48:21.613 INFO:teuthology.orchestra.run.trial148.stdout:(71/99): python3-grpcio-tools-1.46.7-10.el9.x86 594 kB/s | 144 kB 00:00 2026-02-17T05:48:21.616 INFO:teuthology.orchestra.run.trial128.stdout:(73/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 554 kB/s | 11 kB 00:00 2026-02-17T05:48:21.616 INFO:teuthology.orchestra.run.trial148.stdout:(72/99): python3-isodate-0.6.1-3.el9.noarch.rpm 683 kB/s | 56 kB 00:00 2026-02-17T05:48:21.630 INFO:teuthology.orchestra.run.trial128.stdout:(74/99): python3-jaraco-classes-3.2.1-5.el9.noa 907 kB/s | 18 kB 00:00 2026-02-17T05:48:21.635 INFO:teuthology.orchestra.run.trial128.stdout:(75/99): python3-jaraco-collections-3.0.0-8.el9 1.1 MB/s | 23 kB 00:00 2026-02-17T05:48:21.646 INFO:teuthology.orchestra.run.trial016.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-17T05:48:21.648 INFO:teuthology.orchestra.run.trial016.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-17T05:48:21.654 INFO:teuthology.orchestra.run.trial128.stdout:(76/99): python3-jaraco-context-6.0.1-3.el9.noa 843 kB/s | 20 kB 00:00 2026-02-17T05:48:21.655 INFO:teuthology.orchestra.run.trial128.stdout:(77/99): python3-jaraco-functools-3.5.0-2.el9.n 997 kB/s | 19 kB 00:00 2026-02-17T05:48:21.669 INFO:teuthology.orchestra.run.trial016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-17T05:48:21.673 INFO:teuthology.orchestra.run.trial128.stdout:(78/99): python3-jaraco-text-4.0.0-2.el9.noarch 1.3 MB/s | 26 kB 00:00 2026-02-17T05:48:21.674 INFO:teuthology.orchestra.run.trial164.stdout:(6/99): ceph-mgr-dashboard-19.2.3-705.g04cf6dd4 23 MB/s | 3.8 MB 00:00 2026-02-17T05:48:21.680 INFO:teuthology.orchestra.run.trial128.stdout:(79/99): python3-kubernetes-26.1.0-3.el9.noarch 41 MB/s | 1.0 MB 00:00 2026-02-17T05:48:21.695 INFO:teuthology.orchestra.run.trial128.stdout:(80/99): python3-logutils-0.3.5-21.el9.noarch.r 2.1 MB/s | 46 kB 00:00 2026-02-17T05:48:21.695 INFO:teuthology.orchestra.run.trial148.stdout:(73/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 131 kB/s | 11 kB 00:00 2026-02-17T05:48:21.698 INFO:teuthology.orchestra.run.trial148.stdout:(74/99): python3-jaraco-classes-3.2.1-5.el9.noa 216 kB/s | 18 kB 00:00 2026-02-17T05:48:21.699 INFO:teuthology.orchestra.run.trial164.stdout:(7/99): ceph-mon-19.2.3-705.g04cf6dd4.el9.x86_6 13 MB/s | 4.7 MB 00:00 2026-02-17T05:48:21.718 INFO:teuthology.orchestra.run.trial128.stdout:(81/99): python3-more-itertools-8.12.0-2.el9.no 2.0 MB/s | 79 kB 00:00 2026-02-17T05:48:21.719 INFO:teuthology.orchestra.run.trial128.stdout:(82/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 3.5 MB/s | 86 kB 00:00 2026-02-17T05:48:21.738 INFO:teuthology.orchestra.run.trial128.stdout:(83/99): python3-natsort-7.1.1-5.el9.noarch.rpm 2.9 MB/s | 58 kB 00:00 2026-02-17T05:48:21.742 INFO:teuthology.orchestra.run.trial128.stdout:(84/99): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2026-02-17T05:48:21.748 INFO:teuthology.orchestra.run.trial164.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-705.g04cf6dd4 449 kB/s | 22 kB 00:00 2026-02-17T05:48:21.758 INFO:teuthology.orchestra.run.trial128.stdout:(85/99): python3-portend-3.1.0-2.el9.noarch.rpm 841 kB/s | 16 kB 00:00 2026-02-17T05:48:21.762 INFO:teuthology.orchestra.run.trial128.stdout:(86/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 4.4 MB/s | 90 kB 00:00 2026-02-17T05:48:21.777 INFO:teuthology.orchestra.run.trial148.stdout:(75/99): python3-jaraco-collections-3.0.0-8.el9 283 kB/s | 23 kB 00:00 2026-02-17T05:48:21.778 INFO:teuthology.orchestra.run.trial128.stdout:(87/99): python3-repoze-lru-0.7-16.el9.noarch.r 1.5 MB/s | 31 kB 00:00 2026-02-17T05:48:21.780 INFO:teuthology.orchestra.run.trial148.stdout:(76/99): python3-jaraco-context-6.0.1-3.el9.noa 239 kB/s | 20 kB 00:00 2026-02-17T05:48:21.783 INFO:teuthology.orchestra.run.trial128.stdout:(88/99): python3-routes-2.5.1-5.el9.noarch.rpm 9.1 MB/s | 188 kB 00:00 2026-02-17T05:48:21.797 INFO:teuthology.orchestra.run.trial128.stdout:(89/99): python3-rsa-4.9-2.el9.noarch.rpm 2.9 MB/s | 59 kB 00:00 2026-02-17T05:48:21.803 INFO:teuthology.orchestra.run.trial128.stdout:(90/99): python3-saml-1.16.0-1.el9.noarch.rpm 6.1 MB/s | 125 kB 00:00 2026-02-17T05:48:21.805 INFO:teuthology.orchestra.run.trial164.stdout:(9/99): ceph-mgr-modules-core-19.2.3-705.g04cf6 4.3 MB/s | 253 kB 00:00 2026-02-17T05:48:21.817 INFO:teuthology.orchestra.run.trial128.stdout:(91/99): python3-tempora-5.0.0-2.el9.noarch.rpm 1.8 MB/s | 36 kB 00:00 2026-02-17T05:48:21.823 INFO:teuthology.orchestra.run.trial128.stdout:(92/99): python3-typing-extensions-4.15.0-1.el9 4.2 MB/s | 86 kB 00:00 2026-02-17T05:48:21.841 INFO:teuthology.orchestra.run.trial128.stdout:(93/99): python3-webob-1.8.8-2.el9.noarch.rpm 9.6 MB/s | 230 kB 00:00 2026-02-17T05:48:21.843 INFO:teuthology.orchestra.run.trial128.stdout:(94/99): python3-websocket-client-1.2.3-2.el9.n 4.4 MB/s | 90 kB 00:00 2026-02-17T05:48:21.859 INFO:teuthology.orchestra.run.trial148.stdout:(77/99): python3-jaraco-functools-3.5.0-2.el9.n 236 kB/s | 19 kB 00:00 2026-02-17T05:48:21.862 INFO:teuthology.orchestra.run.trial148.stdout:(78/99): python3-jaraco-text-4.0.0-2.el9.noarch 324 kB/s | 26 kB 00:00 2026-02-17T05:48:21.862 INFO:teuthology.orchestra.run.trial164.stdout:(10/99): ceph-mgr-rook-19.2.3-705.g04cf6dd4.el9 871 kB/s | 49 kB 00:00 2026-02-17T05:48:21.863 INFO:teuthology.orchestra.run.trial128.stdout:(95/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 19 MB/s | 427 kB 00:00 2026-02-17T05:48:21.863 INFO:teuthology.orchestra.run.trial128.stdout:(96/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 2.3 MB/s | 48 kB 00:00 2026-02-17T05:48:21.882 INFO:teuthology.orchestra.run.trial128.stdout:(97/99): python3-xmltodict-0.12.0-15.el9.noarch 1.1 MB/s | 22 kB 00:00 2026-02-17T05:48:21.883 INFO:teuthology.orchestra.run.trial128.stdout:(98/99): python3-zc-lockfile-2.0-10.el9.noarch. 996 kB/s | 20 kB 00:00 2026-02-17T05:48:21.919 INFO:teuthology.orchestra.run.trial164.stdout:(11/99): ceph-prometheus-alerts-19.2.3-705.g04c 293 kB/s | 17 kB 00:00 2026-02-17T05:48:21.944 INFO:teuthology.orchestra.run.trial148.stdout:(79/99): python3-logutils-0.3.5-21.el9.noarch.r 566 kB/s | 46 kB 00:00 2026-02-17T05:48:21.979 INFO:teuthology.orchestra.run.trial164.stdout:(12/99): ceph-mgr-diskprediction-local-19.2.3-7 24 MB/s | 7.4 MB 00:00 2026-02-17T05:48:21.980 INFO:teuthology.orchestra.run.trial164.stdout:(13/99): ceph-volume-19.2.3-705.g04cf6dd4.el9.n 4.8 MB/s | 299 kB 00:00 2026-02-17T05:48:21.996 INFO:teuthology.orchestra.run.trial016.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-17T05:48:22.022 INFO:teuthology.orchestra.run.trial164.stdout:(14/99): cephadm-19.2.3-705.g04cf6dd4.el9.noarc 18 MB/s | 769 kB 00:00 2026-02-17T05:48:22.044 INFO:teuthology.orchestra.run.trial164.stdout:(15/99): ceph-osd-19.2.3-705.g04cf6dd4.el9.x86_ 27 MB/s | 17 MB 00:00 2026-02-17T05:48:22.069 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-17T05:48:22.091 INFO:teuthology.orchestra.run.trial164.stdout:(16/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 586 kB/s | 40 kB 00:00 2026-02-17T05:48:22.107 INFO:teuthology.orchestra.run.trial148.stdout:(80/99): python3-more-itertools-8.12.0-2.el9.no 484 kB/s | 79 kB 00:00 2026-02-17T05:48:22.107 INFO:teuthology.orchestra.run.trial164.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 1.1 MB/s | 72 kB 00:00 2026-02-17T05:48:22.126 INFO:teuthology.orchestra.run.trial164.stdout:(18/99): cryptsetup-2.8.1-2.el9.x86_64.rpm 2.3 MB/s | 350 kB 00:00 2026-02-17T05:48:22.151 INFO:teuthology.orchestra.run.trial164.stdout:(19/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 10 MB/s | 253 kB 00:00 2026-02-17T05:48:22.151 INFO:teuthology.orchestra.run.trial164.stdout:(20/99): libquadmath-11.5.0-14.el9.x86_64.rpm 4.1 MB/s | 184 kB 00:00 2026-02-17T05:48:22.175 INFO:teuthology.orchestra.run.trial164.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 4.3 MB/s | 106 kB 00:00 2026-02-17T05:48:22.185 INFO:teuthology.orchestra.run.trial164.stdout:(22/99): libgfortran-11.5.0-14.el9.x86_64.rpm 8.2 MB/s | 794 kB 00:00 2026-02-17T05:48:22.199 INFO:teuthology.orchestra.run.trial164.stdout:(23/99): python3-pycparser-2.20-6.el9.noarch.rp 5.6 MB/s | 135 kB 00:00 2026-02-17T05:48:22.211 INFO:teuthology.orchestra.run.trial164.stdout:(24/99): python3-pyparsing-2.4.7-9.el9.noarch.r 5.7 MB/s | 150 kB 00:00 2026-02-17T05:48:22.215 INFO:teuthology.orchestra.run.trial164.stdout:(25/99): python3-cryptography-36.0.1-5.el9.x86_ 19 MB/s | 1.2 MB 00:00 2026-02-17T05:48:22.223 INFO:teuthology.orchestra.run.trial164.stdout:(26/99): python3-requests-2.25.1-10.el9.noarch. 5.2 MB/s | 126 kB 00:00 2026-02-17T05:48:22.238 INFO:teuthology.orchestra.run.trial164.stdout:(27/99): python3-urllib3-1.26.5-7.el9.noarch.rp 7.9 MB/s | 218 kB 00:00 2026-02-17T05:48:22.239 INFO:teuthology.orchestra.run.trial164.stdout:(28/99): unzip-6.0-59.el9.x86_64.rpm 7.5 MB/s | 182 kB 00:00 2026-02-17T05:48:22.247 INFO:teuthology.orchestra.run.trial164.stdout:(29/99): zip-3.0-35.el9.x86_64.rpm 11 MB/s | 266 kB 00:00 2026-02-17T05:48:22.270 INFO:teuthology.orchestra.run.trial148.stdout:(81/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 530 kB/s | 86 kB 00:00 2026-02-17T05:48:22.271 INFO:teuthology.orchestra.run.trial164.stdout:(30/99): flexiblas-openblas-openmp-3.0.4-9.el9. 642 kB/s | 15 kB 00:00 2026-02-17T05:48:22.280 INFO:teuthology.orchestra.run.trial164.stdout:(31/99): flexiblas-3.0.4-9.el9.x86_64.rpm 710 kB/s | 30 kB 00:00 2026-02-17T05:48:22.305 INFO:teuthology.orchestra.run.trial164.stdout:(32/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 7.1 MB/s | 246 kB 00:00 2026-02-17T05:48:22.314 INFO:teuthology.orchestra.run.trial164.stdout:(33/99): openblas-0.3.29-1.el9.x86_64.rpm 4.5 MB/s | 42 kB 00:00 2026-02-17T05:48:22.317 INFO:teuthology.orchestra.run.trial164.stdout:(34/99): lua-5.4.4-4.el9.x86_64.rpm 5.0 MB/s | 188 kB 00:00 2026-02-17T05:48:22.324 INFO:teuthology.orchestra.run.trial164.stdout:(35/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 35 MB/s | 3.0 MB 00:00 2026-02-17T05:48:22.352 INFO:teuthology.orchestra.run.trial148.stdout:(82/99): python3-natsort-7.1.1-5.el9.noarch.rpm 705 kB/s | 58 kB 00:00 2026-02-17T05:48:22.355 INFO:teuthology.orchestra.run.trial164.stdout:(36/99): python3-babel-2.9.1-2.el9.noarch.rpm 195 MB/s | 6.0 MB 00:00 2026-02-17T05:48:22.357 INFO:teuthology.orchestra.run.trial164.stdout:(37/99): protobuf-3.14.0-17.el9.x86_64.rpm 25 MB/s | 1.0 MB 00:00 2026-02-17T05:48:22.366 INFO:teuthology.orchestra.run.trial164.stdout:(38/99): python3-devel-3.9.25-2.el9.x86_64.rpm 22 MB/s | 244 kB 00:00 2026-02-17T05:48:22.371 INFO:teuthology.orchestra.run.trial164.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 17 MB/s | 249 kB 00:00 2026-02-17T05:48:22.377 INFO:teuthology.orchestra.run.trial164.stdout:(40/99): python3-libstoragemgmt-1.10.1-1.el9.x8 16 MB/s | 177 kB 00:00 2026-02-17T05:48:22.388 INFO:teuthology.orchestra.run.trial164.stdout:(41/99): python3-mako-1.1.4-6.el9.noarch.rpm 16 MB/s | 172 kB 00:00 2026-02-17T05:48:22.397 INFO:teuthology.orchestra.run.trial164.stdout:(42/99): python3-markupsafe-1.1.1-12.el9.x86_64 3.9 MB/s | 35 kB 00:00 2026-02-17T05:48:22.421 INFO:teuthology.orchestra.run.trial164.stdout:(43/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 25 MB/s | 1.2 MB 00:00 2026-02-17T05:48:22.437 INFO:teuthology.orchestra.run.trial164.stdout:(44/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 155 MB/s | 6.1 MB 00:00 2026-02-17T05:48:22.438 INFO:teuthology.orchestra.run.trial164.stdout:(45/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 25 MB/s | 442 kB 00:00 2026-02-17T05:48:22.445 INFO:teuthology.orchestra.run.trial164.stdout:(46/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 40 MB/s | 5.3 MB 00:00 2026-02-17T05:48:22.446 INFO:teuthology.orchestra.run.trial164.stdout:(47/99): python3-packaging-20.9-5.el9.noarch.rp 7.9 MB/s | 77 kB 00:00 2026-02-17T05:48:22.453 INFO:teuthology.orchestra.run.trial164.stdout:(48/99): python3-protobuf-3.14.0-17.el9.noarch. 17 MB/s | 267 kB 00:00 2026-02-17T05:48:22.455 INFO:teuthology.orchestra.run.trial164.stdout:(49/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 15 MB/s | 159 kB 00:00 2026-02-17T05:48:22.456 INFO:teuthology.orchestra.run.trial164.stdout:(50/99): python3-pyasn1-modules-0.4.8-6.el9.noa 27 MB/s | 279 kB 00:00 2026-02-17T05:48:22.465 INFO:teuthology.orchestra.run.trial164.stdout:(51/99): python3-toml-0.10.2-6.el9.noarch.rpm 4.6 MB/s | 42 kB 00:00 2026-02-17T05:48:22.466 INFO:teuthology.orchestra.run.trial164.stdout:(52/99): python3-requests-oauthlib-1.3.0-12.el9 4.1 MB/s | 54 kB 00:00 2026-02-17T05:48:22.474 INFO:teuthology.orchestra.run.trial164.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 20 MB/s | 189 kB 00:00 2026-02-17T05:48:22.477 INFO:teuthology.orchestra.run.trial164.stdout:(54/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 8.3 MB/s | 90 kB 00:00 2026-02-17T05:48:22.660 INFO:teuthology.orchestra.run.trial128.stdout:(99/99): protobuf-compiler-3.14.0-17.el9.x86_64 520 kB/s | 862 kB 00:01 2026-02-17T05:48:22.663 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:22.663 INFO:teuthology.orchestra.run.trial128.stdout:Total 21 MB/s | 95 MB 00:04 2026-02-17T05:48:22.677 INFO:teuthology.orchestra.run.trial148.stdout:(83/99): python3-pecan-1.4.2-3.el9.noarch.rpm 838 kB/s | 272 kB 00:00 2026-02-17T05:48:22.745 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-17T05:48:22.758 INFO:teuthology.orchestra.run.trial148.stdout:(84/99): python3-portend-3.1.0-2.el9.noarch.rpm 201 kB/s | 16 kB 00:00 2026-02-17T05:48:22.763 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-17T05:48:22.773 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-17T05:48:22.780 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-17T05:48:22.793 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-17T05:48:22.809 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-17T05:48:22.860 INFO:teuthology.orchestra.run.trial164.stdout:(55/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 48 MB/s | 19 MB 00:00 2026-02-17T05:48:22.885 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-17T05:48:22.891 INFO:teuthology.orchestra.run.trial164.stdout:(56/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 17 MB/s | 551 kB 00:00 2026-02-17T05:48:22.894 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-17T05:48:22.897 INFO:teuthology.orchestra.run.trial164.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 3.4 MB/s | 19 kB 00:00 2026-02-17T05:48:22.904 INFO:teuthology.orchestra.run.trial164.stdout:(58/99): luarocks-3.9.2-5.el9.noarch.rpm 22 MB/s | 151 kB 00:00 2026-02-17T05:48:22.913 INFO:teuthology.orchestra.run.trial164.stdout:(59/99): python3-asyncssh-2.13.2-5.el9.noarch.r 59 MB/s | 548 kB 00:00 2026-02-17T05:48:22.918 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-17T05:48:22.919 INFO:teuthology.orchestra.run.trial164.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 5.4 MB/s | 29 kB 00:00 2026-02-17T05:48:22.921 INFO:teuthology.orchestra.run.trial148.stdout:(85/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 553 kB/s | 90 kB 00:00 2026-02-17T05:48:22.924 INFO:teuthology.orchestra.run.trial164.stdout:(61/99): python3-backports-tarfile-1.2.0-1.el9. 10 MB/s | 60 kB 00:00 2026-02-17T05:48:22.930 INFO:teuthology.orchestra.run.trial164.stdout:(62/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 7.6 MB/s | 43 kB 00:00 2026-02-17T05:48:22.936 INFO:teuthology.orchestra.run.trial164.stdout:(63/99): python3-cachetools-4.2.4-1.el9.noarch. 5.5 MB/s | 32 kB 00:00 2026-02-17T05:48:22.941 INFO:teuthology.orchestra.run.trial164.stdout:(64/99): python3-certifi-2023.05.07-4.el9.noarc 2.8 MB/s | 14 kB 00:00 2026-02-17T05:48:22.948 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-17T05:48:22.949 INFO:teuthology.orchestra.run.trial164.stdout:(65/99): python3-cheroot-10.0.1-4.el9.noarch.rp 21 MB/s | 173 kB 00:00 2026-02-17T05:48:22.962 INFO:teuthology.orchestra.run.trial164.stdout:(66/99): python3-cherrypy-18.6.1-2.el9.noarch.r 27 MB/s | 358 kB 00:00 2026-02-17T05:48:22.970 INFO:teuthology.orchestra.run.trial164.stdout:(67/99): python3-google-auth-2.45.0-1.el9.noarc 32 MB/s | 254 kB 00:00 2026-02-17T05:48:22.999 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-17T05:48:23.003 INFO:teuthology.orchestra.run.trial148.stdout:(86/99): python3-repoze-lru-0.7-16.el9.noarch.r 374 kB/s | 31 kB 00:00 2026-02-17T05:48:23.006 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-17T05:48:23.009 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-17T05:48:23.010 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-17T05:48:23.028 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:23.028 INFO:teuthology.orchestra.run.trial016.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-17T05:48:23.028 INFO:teuthology.orchestra.run.trial016.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-17T05:48:23.028 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:23.034 INFO:teuthology.orchestra.run.trial164.stdout:(68/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 32 MB/s | 2.0 MB 00:00 2026-02-17T05:48:23.036 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:23.042 INFO:teuthology.orchestra.run.trial164.stdout:(69/99): python3-grpcio-tools-1.46.7-10.el9.x86 19 MB/s | 144 kB 00:00 2026-02-17T05:48:23.050 INFO:teuthology.orchestra.run.trial164.stdout:(70/99): python3-influxdb-5.3.1-1.el9.noarch.rp 17 MB/s | 139 kB 00:00 2026-02-17T05:48:23.055 INFO:teuthology.orchestra.run.trial164.stdout:(71/99): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2026-02-17T05:48:23.056 INFO:teuthology.orchestra.run.trial164.stdout:(72/99): python3-isodate-0.6.1-3.el9.noarch.rpm 8.4 MB/s | 56 kB 00:00 2026-02-17T05:48:23.060 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:23.061 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-17T05:48:23.061 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:23.063 INFO:teuthology.orchestra.run.trial164.stdout:(73/99): python3-jaraco-classes-3.2.1-5.el9.noa 2.8 MB/s | 18 kB 00:00 2026-02-17T05:48:23.064 INFO:teuthology.orchestra.run.trial164.stdout:(74/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 1.2 MB/s | 11 kB 00:00 2026-02-17T05:48:23.068 INFO:teuthology.orchestra.run.trial164.stdout:(75/99): python3-jaraco-collections-3.0.0-8.el9 4.3 MB/s | 23 kB 00:00 2026-02-17T05:48:23.070 INFO:teuthology.orchestra.run.trial164.stdout:(76/99): python3-jaraco-context-6.0.1-3.el9.noa 3.4 MB/s | 20 kB 00:00 2026-02-17T05:48:23.073 INFO:teuthology.orchestra.run.trial148.stdout:(87/99): python3-kubernetes-26.1.0-3.el9.noarch 861 kB/s | 1.0 MB 00:01 2026-02-17T05:48:23.074 INFO:teuthology.orchestra.run.trial164.stdout:(77/99): python3-jaraco-functools-3.5.0-2.el9.n 3.7 MB/s | 19 kB 00:00 2026-02-17T05:48:23.076 INFO:teuthology.orchestra.run.trial164.stdout:(78/99): python3-jaraco-text-4.0.0-2.el9.noarch 4.2 MB/s | 26 kB 00:00 2026-02-17T05:48:23.083 INFO:teuthology.orchestra.run.trial164.stdout:(79/99): python3-logutils-0.3.5-21.el9.noarch.r 6.6 MB/s | 46 kB 00:00 2026-02-17T05:48:23.085 INFO:teuthology.orchestra.run.trial164.stdout:(80/99): python3-kubernetes-26.1.0-3.el9.noarch 85 MB/s | 1.0 MB 00:00 2026-02-17T05:48:23.090 INFO:teuthology.orchestra.run.trial164.stdout:(81/99): python3-more-itertools-8.12.0-2.el9.no 11 MB/s | 79 kB 00:00 2026-02-17T05:48:23.091 INFO:teuthology.orchestra.run.trial164.stdout:(82/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 13 MB/s | 86 kB 00:00 2026-02-17T05:48:23.093 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:23.094 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-17T05:48:23.096 INFO:teuthology.orchestra.run.trial164.stdout:(83/99): python3-natsort-7.1.1-5.el9.noarch.rpm 9.9 MB/s | 58 kB 00:00 2026-02-17T05:48:23.098 INFO:teuthology.orchestra.run.trial164.stdout:(84/99): python3-pecan-1.4.2-3.el9.noarch.rpm 41 MB/s | 272 kB 00:00 2026-02-17T05:48:23.101 INFO:teuthology.orchestra.run.trial164.stdout:(85/99): python3-portend-3.1.0-2.el9.noarch.rpm 3.2 MB/s | 16 kB 00:00 2026-02-17T05:48:23.104 INFO:teuthology.orchestra.run.trial164.stdout:(86/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 14 MB/s | 90 kB 00:00 2026-02-17T05:48:23.109 INFO:teuthology.orchestra.run.trial164.stdout:(87/99): python3-repoze-lru-0.7-16.el9.noarch.r 3.9 MB/s | 31 kB 00:00 2026-02-17T05:48:23.110 INFO:teuthology.orchestra.run.trial164.stdout:(88/99): python3-routes-2.5.1-5.el9.noarch.rpm 31 MB/s | 188 kB 00:00 2026-02-17T05:48:23.115 INFO:teuthology.orchestra.run.trial164.stdout:(89/99): python3-rsa-4.9-2.el9.noarch.rpm 9.9 MB/s | 59 kB 00:00 2026-02-17T05:48:23.117 INFO:teuthology.orchestra.run.trial164.stdout:(90/99): python3-saml-1.16.0-1.el9.noarch.rpm 18 MB/s | 125 kB 00:00 2026-02-17T05:48:23.120 INFO:teuthology.orchestra.run.trial164.stdout:(91/99): python3-tempora-5.0.0-2.el9.noarch.rpm 6.6 MB/s | 36 kB 00:00 2026-02-17T05:48:23.124 INFO:teuthology.orchestra.run.trial164.stdout:(92/99): python3-typing-extensions-4.15.0-1.el9 13 MB/s | 86 kB 00:00 2026-02-17T05:48:23.130 INFO:teuthology.orchestra.run.trial164.stdout:(93/99): python3-webob-1.8.8-2.el9.noarch.rpm 23 MB/s | 230 kB 00:00 2026-02-17T05:48:23.131 INFO:teuthology.orchestra.run.trial164.stdout:(94/99): python3-websocket-client-1.2.3-2.el9.n 13 MB/s | 90 kB 00:00 2026-02-17T05:48:23.132 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:23.133 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:23.140 INFO:teuthology.orchestra.run.trial164.stdout:(95/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 5.3 MB/s | 48 kB 00:00 2026-02-17T05:48:23.145 INFO:teuthology.orchestra.run.trial164.stdout:(96/99): python3-xmltodict-0.12.0-15.el9.noarch 4.5 MB/s | 22 kB 00:00 2026-02-17T05:48:23.147 INFO:teuthology.orchestra.run.trial164.stdout:(97/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 25 MB/s | 427 kB 00:00 2026-02-17T05:48:23.150 INFO:teuthology.orchestra.run.trial164.stdout:(98/99): python3-zc-lockfile-2.0-10.el9.noarch. 3.9 MB/s | 20 kB 00:00 2026-02-17T05:48:23.154 INFO:teuthology.orchestra.run.trial148.stdout:(88/99): python3-rsa-4.9-2.el9.noarch.rpm 723 kB/s | 59 kB 00:00 2026-02-17T05:48:23.159 INFO:teuthology.orchestra.run.trial016.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 84/99 2026-02-17T05:48:23.161 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:48:23.173 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:48:23.174 INFO:teuthology.orchestra.run.trial016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:23.174 INFO:teuthology.orchestra.run.trial016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-17T05:48:23.174 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:23.248 INFO:teuthology.orchestra.run.trial148.stdout:(89/99): python3-routes-2.5.1-5.el9.noarch.rpm 773 kB/s | 188 kB 00:00 2026-02-17T05:48:23.318 INFO:teuthology.orchestra.run.trial148.stdout:(90/99): python3-saml-1.16.0-1.el9.noarch.rpm 764 kB/s | 125 kB 00:00 2026-02-17T05:48:23.329 INFO:teuthology.orchestra.run.trial148.stdout:(91/99): python3-tempora-5.0.0-2.el9.noarch.rpm 437 kB/s | 36 kB 00:00 2026-02-17T05:48:23.480 INFO:teuthology.orchestra.run.trial148.stdout:(92/99): python3-typing-extensions-4.15.0-1.el9 533 kB/s | 86 kB 00:00 2026-02-17T05:48:23.548 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:23.549 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:23.643 INFO:teuthology.orchestra.run.trial148.stdout:(93/99): python3-websocket-client-1.2.3-2.el9.n 552 kB/s | 90 kB 00:00 2026-02-17T05:48:23.654 INFO:teuthology.orchestra.run.trial148.stdout:(94/99): python3-webob-1.8.8-2.el9.noarch.rpm 709 kB/s | 230 kB 00:00 2026-02-17T05:48:23.736 INFO:teuthology.orchestra.run.trial148.stdout:(95/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 592 kB/s | 48 kB 00:00 2026-02-17T05:48:23.818 INFO:teuthology.orchestra.run.trial148.stdout:(96/99): python3-xmltodict-0.12.0-15.el9.noarch 268 kB/s | 22 kB 00:00 2026-02-17T05:48:23.849 INFO:teuthology.orchestra.run.trial148.stdout:(97/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 839 kB/s | 2.0 MB 00:02 2026-02-17T05:48:23.873 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:48:23.899 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:48:23.899 INFO:teuthology.orchestra.run.trial016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:23.899 INFO:teuthology.orchestra.run.trial016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-17T05:48:23.899 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:48:23.899 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:48:23.899 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:23.902 INFO:teuthology.orchestra.run.trial148.stdout:(98/99): python3-zc-lockfile-2.0-10.el9.noarch. 240 kB/s | 20 kB 00:00 2026-02-17T05:48:23.955 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:48:23.957 INFO:teuthology.orchestra.run.trial016.stdout: Installing : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:48:23.962 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 88/99 2026-02-17T05:48:23.981 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 89/99 2026-02-17T05:48:23.983 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:48:23.992 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:23.994 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-17T05:48:24.022 INFO:teuthology.orchestra.run.trial164.stdout:(99/99): protobuf-compiler-3.14.0-17.el9.x86_64 558 kB/s | 862 kB 00:01 2026-02-17T05:48:24.025 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:24.026 INFO:teuthology.orchestra.run.trial164.stdout:Total 31 MB/s | 95 MB 00:03 2026-02-17T05:48:24.026 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-17T05:48:24.030 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-17T05:48:24.052 INFO:teuthology.orchestra.run.trial128.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-17T05:48:24.058 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-17T05:48:24.091 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-17T05:48:24.100 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-17T05:48:24.105 INFO:teuthology.orchestra.run.trial128.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-17T05:48:24.108 INFO:teuthology.orchestra.run.trial128.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-17T05:48:24.112 INFO:teuthology.orchestra.run.trial128.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-17T05:48:24.136 INFO:teuthology.orchestra.run.trial128.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-17T05:48:24.148 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-17T05:48:24.151 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-17T05:48:24.157 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-17T05:48:24.159 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-17T05:48:24.170 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-17T05:48:24.176 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-17T05:48:24.200 INFO:teuthology.orchestra.run.trial128.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-17T05:48:24.203 INFO:teuthology.orchestra.run.trial128.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-17T05:48:24.210 INFO:teuthology.orchestra.run.trial148.stdout:(99/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 753 kB/s | 427 kB 00:00 2026-02-17T05:48:24.211 INFO:teuthology.orchestra.run.trial128.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-17T05:48:24.215 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:24.215 INFO:teuthology.orchestra.run.trial148.stdout:Total 11 MB/s | 95 MB 00:08 2026-02-17T05:48:24.235 INFO:teuthology.orchestra.run.trial128.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-17T05:48:24.286 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-17T05:48:24.299 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-17T05:48:24.301 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-17T05:48:24.306 INFO:teuthology.orchestra.run.trial128.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-17T05:48:24.311 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-17T05:48:24.316 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-17T05:48:24.319 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-17T05:48:24.334 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-17T05:48:24.355 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-17T05:48:24.360 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-17T05:48:24.365 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-17T05:48:24.376 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-17T05:48:24.385 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-17T05:48:24.391 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-17T05:48:24.411 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-17T05:48:24.420 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-17T05:48:24.463 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:24.474 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-17T05:48:24.477 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:48:24.481 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:48:24.486 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-17T05:48:24.499 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:24.499 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:24.502 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-17T05:48:24.507 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-17T05:48:24.514 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-17T05:48:24.555 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-17T05:48:24.638 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:24.675 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:24.675 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:24.881 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-17T05:48:24.891 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-17T05:48:24.895 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-17T05:48:24.901 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-17T05:48:24.902 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-17T05:48:24.914 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:24.915 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:24.930 INFO:teuthology.orchestra.run.trial128.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-17T05:48:24.932 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:48:24.933 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:48:24.946 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:48:24.947 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:48:24.974 INFO:teuthology.orchestra.run.trial128.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-17T05:48:24.981 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-17T05:48:24.985 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-17T05:48:24.990 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-17T05:48:24.994 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-17T05:48:25.001 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-17T05:48:25.002 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:48:25.004 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-17T05:48:25.033 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-17T05:48:25.042 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-17T05:48:25.051 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 94/99 2026-02-17T05:48:25.052 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:48:25.075 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:48:25.075 INFO:teuthology.orchestra.run.trial016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:25.075 INFO:teuthology.orchestra.run.trial016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-17T05:48:25.075 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:48:25.076 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:48:25.076 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:25.077 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-17T05:48:25.089 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:48:25.095 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:25.095 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:25.303 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 60/99 2026-02-17T05:48:25.306 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:48:25.307 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:48:25.327 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-17T05:48:25.329 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:48:25.329 INFO:teuthology.orchestra.run.trial016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:25.329 INFO:teuthology.orchestra.run.trial016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-17T05:48:25.329 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:48:25.329 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:48:25.329 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:25.331 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-17T05:48:25.355 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:25.357 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-17T05:48:25.388 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-17T05:48:25.389 INFO:teuthology.orchestra.run.trial128.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-17T05:48:25.391 INFO:teuthology.orchestra.run.trial128.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-17T05:48:25.392 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-17T05:48:25.412 INFO:teuthology.orchestra.run.trial128.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-17T05:48:25.413 INFO:teuthology.orchestra.run.trial164.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-17T05:48:25.419 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-17T05:48:25.430 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:48:25.450 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:48:25.450 INFO:teuthology.orchestra.run.trial016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:25.451 INFO:teuthology.orchestra.run.trial016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-17T05:48:25.451 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:48:25.451 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:48:25.451 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:25.452 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-17T05:48:25.461 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-17T05:48:25.466 INFO:teuthology.orchestra.run.trial164.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-17T05:48:25.468 INFO:teuthology.orchestra.run.trial164.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-17T05:48:25.469 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:48:25.473 INFO:teuthology.orchestra.run.trial164.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-17T05:48:25.496 INFO:teuthology.orchestra.run.trial164.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-17T05:48:25.508 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-17T05:48:25.512 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-17T05:48:25.518 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-17T05:48:25.520 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-17T05:48:25.531 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-17T05:48:25.537 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-17T05:48:25.542 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:25.544 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-17T05:48:25.560 INFO:teuthology.orchestra.run.trial164.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-17T05:48:25.564 INFO:teuthology.orchestra.run.trial164.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-17T05:48:25.570 INFO:teuthology.orchestra.run.trial164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-17T05:48:25.575 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-17T05:48:25.580 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-17T05:48:25.594 INFO:teuthology.orchestra.run.trial164.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-17T05:48:25.602 INFO:teuthology.orchestra.run.trial148.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-17T05:48:25.608 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-17T05:48:25.642 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-17T05:48:25.645 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-17T05:48:25.651 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-17T05:48:25.656 INFO:teuthology.orchestra.run.trial148.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-17T05:48:25.658 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-17T05:48:25.659 INFO:teuthology.orchestra.run.trial148.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-17T05:48:25.660 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-17T05:48:25.663 INFO:teuthology.orchestra.run.trial148.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-17T05:48:25.664 INFO:teuthology.orchestra.run.trial164.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-17T05:48:25.670 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-17T05:48:25.674 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-17T05:48:25.678 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-17T05:48:25.686 INFO:teuthology.orchestra.run.trial148.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-17T05:48:25.692 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-17T05:48:25.699 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-17T05:48:25.702 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-17T05:48:25.708 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-17T05:48:25.710 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-17T05:48:25.712 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-17T05:48:25.717 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-17T05:48:25.721 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-17T05:48:25.722 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-17T05:48:25.727 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-17T05:48:25.733 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-17T05:48:25.739 INFO:teuthology.orchestra.run.trial128.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-17T05:48:25.742 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-17T05:48:25.748 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-17T05:48:25.751 INFO:teuthology.orchestra.run.trial148.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-17T05:48:25.755 INFO:teuthology.orchestra.run.trial148.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-17T05:48:25.762 INFO:teuthology.orchestra.run.trial148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-17T05:48:25.768 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-17T05:48:25.776 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-17T05:48:25.786 INFO:teuthology.orchestra.run.trial148.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-17T05:48:25.812 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-17T05:48:25.830 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-17T05:48:25.838 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-17T05:48:25.843 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-17T05:48:25.851 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-17T05:48:25.854 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-17T05:48:25.859 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-17T05:48:25.860 INFO:teuthology.orchestra.run.trial148.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-17T05:48:25.863 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-17T05:48:25.865 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-17T05:48:25.870 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-17T05:48:25.871 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-17T05:48:25.873 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-17T05:48:25.888 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-17T05:48:25.909 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-17T05:48:25.911 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-17T05:48:25.914 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-17T05:48:25.919 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-17T05:48:25.930 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-17T05:48:25.939 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-17T05:48:25.945 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-17T05:48:25.965 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-17T05:48:25.974 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-17T05:48:26.029 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-17T05:48:26.041 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-17T05:48:26.057 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-17T05:48:26.062 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-17T05:48:26.069 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-17T05:48:26.112 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-17T05:48:26.147 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 6/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 7/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 8/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 9/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 10/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 11/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 12/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 13/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 14/99 2026-02-17T05:48:26.148 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 15/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 16/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-17T05:48:26.149 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 39/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-17T05:48:26.150 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-17T05:48:26.151 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-17T05:48:26.152 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-17T05:48:26.214 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: lua-5.4.4-4.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-17T05:48:26.215 INFO:teuthology.orchestra.run.trial016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-ply-3.11-14.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-17T05:48:26.216 INFO:teuthology.orchestra.run.trial016.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: unzip-6.0-59.el9.x86_64 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: zip-3.0-35.el9.x86_64 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:26.217 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:26.234 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-17T05:48:26.244 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-17T05:48:26.248 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-17T05:48:26.254 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-17T05:48:26.256 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-17T05:48:26.272 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-base 2026-02-17T05:48:26.283 INFO:teuthology.orchestra.run.trial164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-17T05:48:26.327 INFO:teuthology.orchestra.run.trial164.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-17T05:48:26.334 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-17T05:48:26.337 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-17T05:48:26.343 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-17T05:48:26.346 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-17T05:48:26.353 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-17T05:48:26.356 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-17T05:48:26.384 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-17T05:48:26.394 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-17T05:48:26.429 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-17T05:48:26.440 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-17T05:48:26.450 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-17T05:48:26.454 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-17T05:48:26.460 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-17T05:48:26.462 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-17T05:48:26.486 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-17T05:48:26.490 INFO:teuthology.orchestra.run.trial148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-17T05:48:26.504 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-17T05:48:26.514 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-17T05:48:26.520 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-17T05:48:26.534 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-17T05:48:26.534 INFO:teuthology.orchestra.run.trial148.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-17T05:48:26.540 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-17T05:48:26.544 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-17T05:48:26.549 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-17T05:48:26.550 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-17T05:48:26.553 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-17T05:48:26.560 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-17T05:48:26.564 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-17T05:48:26.586 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:26.592 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-17T05:48:26.602 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-17T05:48:26.625 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-17T05:48:26.634 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-17T05:48:26.637 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-17T05:48:26.650 INFO:teuthology.orchestra.run.trial016.stdout:Package ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:26.652 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 60/99 2026-02-17T05:48:26.657 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-17T05:48:26.667 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:26.668 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:26.668 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:26.675 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-17T05:48:26.679 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-17T05:48:26.687 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-17T05:48:26.689 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install cephadm 2026-02-17T05:48:26.737 INFO:teuthology.orchestra.run.trial164.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-17T05:48:26.738 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-17T05:48:26.739 INFO:teuthology.orchestra.run.trial164.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-17T05:48:26.744 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-17T05:48:26.748 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-17T05:48:26.749 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-17T05:48:26.760 INFO:teuthology.orchestra.run.trial164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-17T05:48:26.768 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:26.768 INFO:teuthology.orchestra.run.trial128.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-17T05:48:26.768 INFO:teuthology.orchestra.run.trial128.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-17T05:48:26.768 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:26.776 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:26.803 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:26.803 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-17T05:48:26.803 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:26.840 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-17T05:48:26.863 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 60/99 2026-02-17T05:48:26.887 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-17T05:48:26.891 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-17T05:48:26.904 INFO:teuthology.orchestra.run.trial128.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 84/99 2026-02-17T05:48:26.906 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:48:26.920 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:48:26.920 INFO:teuthology.orchestra.run.trial128.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:26.920 INFO:teuthology.orchestra.run.trial128.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-17T05:48:26.920 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:26.950 INFO:teuthology.orchestra.run.trial148.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-17T05:48:26.952 INFO:teuthology.orchestra.run.trial148.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-17T05:48:26.973 INFO:teuthology.orchestra.run.trial148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-17T05:48:26.999 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:27.066 INFO:teuthology.orchestra.run.trial016.stdout:Package cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:27.082 INFO:teuthology.orchestra.run.trial164.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-17T05:48:27.083 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:27.084 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:27.084 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:27.105 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T05:48:27.155 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-17T05:48:27.302 INFO:teuthology.orchestra.run.trial148.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-17T05:48:27.376 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-17T05:48:27.410 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:29 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:27.491 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:27.492 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:27.492 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:48:27.492 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:27.492 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:27.493 INFO:teuthology.orchestra.run.trial016.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 145 k 2026-02-17T05:48:27.493 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:27.493 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:27.493 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:27.493 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:48:27.493 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:27.493 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 145 k 2026-02-17T05:48:27.493 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 431 k 2026-02-17T05:48:27.493 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:27.590 INFO:teuthology.orchestra.run.trial016.stdout:ceph-immutable-object-cache-19.2.3-705.g04cf6dd 1.5 MB/s | 145 kB 00:00 2026-02-17T05:48:27.591 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:27.591 INFO:teuthology.orchestra.run.trial016.stdout:Total 1.4 MB/s | 145 kB 00:00 2026-02-17T05:48:27.591 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:27.594 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:27.594 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:27.616 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:48:27.625 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:27.625 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:27.642 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:48:27.642 INFO:teuthology.orchestra.run.trial128.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:27.642 INFO:teuthology.orchestra.run.trial128.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-17T05:48:27.642 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:48:27.642 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:48:27.642 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:27.679 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:27.680 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:27.699 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:48:27.701 INFO:teuthology.orchestra.run.trial128.stdout: Installing : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:48:27.702 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:27.703 INFO:teuthology.orchestra.run.trial016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:27.703 INFO:teuthology.orchestra.run.trial016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-17T05:48:27.703 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:27.706 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 88/99 2026-02-17T05:48:27.726 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 89/99 2026-02-17T05:48:27.727 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:48:27.821 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-17T05:48:27.839 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-17T05:48:27.849 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-17T05:48:27.856 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-17T05:48:27.869 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-17T05:48:27.884 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-17T05:48:27.959 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-17T05:48:27.967 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-17T05:48:27.991 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-17T05:48:28.014 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:28.014 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:28.014 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:28.014 INFO:teuthology.orchestra.run.trial016.stdout: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:28.014 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:28.014 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:28.020 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-17T05:48:28.040 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-mgr 2026-02-17T05:48:28.053 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-17T05:48:28.070 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-17T05:48:28.071 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-17T05:48:28.077 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-17T05:48:28.080 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-17T05:48:28.080 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-17T05:48:28.081 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-17T05:48:28.087 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-17T05:48:28.100 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-17T05:48:28.101 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:28.101 INFO:teuthology.orchestra.run.trial164.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-17T05:48:28.101 INFO:teuthology.orchestra.run.trial164.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-17T05:48:28.101 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:28.109 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:28.116 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-17T05:48:28.137 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:28.137 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-17T05:48:28.137 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:28.172 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-17T05:48:28.192 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-17T05:48:28.201 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-17T05:48:28.220 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:48:28.223 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:48:28.224 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-17T05:48:28.235 INFO:teuthology.orchestra.run.trial164.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 84/99 2026-02-17T05:48:28.237 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:48:28.251 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:48:28.251 INFO:teuthology.orchestra.run.trial164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:28.252 INFO:teuthology.orchestra.run.trial164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-17T05:48:28.252 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:28.254 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-17T05:48:28.305 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-17T05:48:28.312 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-17T05:48:28.315 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-17T05:48:28.316 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-17T05:48:28.333 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:28.334 INFO:teuthology.orchestra.run.trial148.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-17T05:48:28.334 INFO:teuthology.orchestra.run.trial148.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-17T05:48:28.334 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:28.341 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:28.351 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:28.366 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:48:28.366 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-17T05:48:28.366 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:28.400 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-17T05:48:28.415 INFO:teuthology.orchestra.run.trial016.stdout:Package ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:28.432 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:28.433 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:28.433 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:28.454 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T05:48:28.464 INFO:teuthology.orchestra.run.trial148.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 84/99 2026-02-17T05:48:28.466 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:48:28.481 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:48:28.481 INFO:teuthology.orchestra.run.trial148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:28.481 INFO:teuthology.orchestra.run.trial148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-17T05:48:28.481 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:28.675 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:48:28.676 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:48:28.688 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:48:28.689 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:48:28.747 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:48:28.766 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:28.796 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 94/99 2026-02-17T05:48:28.797 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:48:28.819 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:48:28.819 INFO:teuthology.orchestra.run.trial128.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:28.819 INFO:teuthology.orchestra.run.trial128.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-17T05:48:28.819 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:48:28.819 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:48:28.819 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:28.830 INFO:teuthology.orchestra.run.trial016.stdout:Package ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:28.830 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:48:28.847 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:28.848 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:28.848 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:28.868 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T05:48:28.967 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:48:28.993 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:48:28.993 INFO:teuthology.orchestra.run.trial164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:28.994 INFO:teuthology.orchestra.run.trial164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-17T05:48:28.994 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:48:28.994 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:48:28.994 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:29.046 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:48:29.048 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:48:29.050 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:48:29.052 INFO:teuthology.orchestra.run.trial164.stdout: Installing : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:48:29.057 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 88/99 2026-02-17T05:48:29.069 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:48:29.069 INFO:teuthology.orchestra.run.trial128.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:29.069 INFO:teuthology.orchestra.run.trial128.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-17T05:48:29.069 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:48:29.070 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:48:29.070 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:29.077 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 89/99 2026-02-17T05:48:29.078 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:48:29.171 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:48:29.175 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:48:29.176 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:31 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:29.192 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:48:29.192 INFO:teuthology.orchestra.run.trial128.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:29.192 INFO:teuthology.orchestra.run.trial128.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-17T05:48:29.192 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:48:29.192 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:48:29.192 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:29.201 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:48:29.201 INFO:teuthology.orchestra.run.trial148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:29.201 INFO:teuthology.orchestra.run.trial148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-17T05:48:29.201 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:48:29.201 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:48:29.201 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:29.211 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:48:29.239 INFO:teuthology.orchestra.run.trial016.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:29.257 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:48:29.258 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:29.258 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:29.258 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:29.259 INFO:teuthology.orchestra.run.trial148.stdout: Installing : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:48:29.265 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 88/99 2026-02-17T05:48:29.279 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-mgr-rook 2026-02-17T05:48:29.285 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 89/99 2026-02-17T05:48:29.287 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:48:29.577 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:48:29.580 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:48:29.584 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:31 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:29.648 INFO:teuthology.orchestra.run.trial016.stdout:Package ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:29.665 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:29.666 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:29.666 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:29.686 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T05:48:29.781 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:48:29.784 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:48:29.908 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:48:29.909 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/99 2026-02-17T05:48:29.909 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/99 2026-02-17T05:48:29.909 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/99 2026-02-17T05:48:29.909 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/99 2026-02-17T05:48:29.909 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/99 2026-02-17T05:48:29.909 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 6/99 2026-02-17T05:48:29.909 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 7/99 2026-02-17T05:48:29.909 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 8/99 2026-02-17T05:48:29.909 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 9/99 2026-02-17T05:48:29.909 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 10/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 11/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 12/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 13/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 14/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 15/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 16/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-17T05:48:29.910 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-17T05:48:29.911 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 39/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-17T05:48:29.912 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-17T05:48:29.913 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-17T05:48:29.914 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-17T05:48:29.915 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-17T05:48:29.916 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-17T05:48:29.972 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-17T05:48:29.972 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:29.972 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:29.973 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-17T05:48:29.974 INFO:teuthology.orchestra.run.trial128.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: lua-5.4.4-4.el9.x86_64 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-17T05:48:29.975 INFO:teuthology.orchestra.run.trial128.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-17T05:48:29.976 INFO:teuthology.orchestra.run.trial128.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-17T05:48:29.977 INFO:teuthology.orchestra.run.trial128.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-ply-3.11-14.el9.noarch 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-17T05:48:29.978 INFO:teuthology.orchestra.run.trial128.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-17T05:48:29.979 INFO:teuthology.orchestra.run.trial128.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: unzip-6.0-59.el9.x86_64 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: zip-3.0-35.el9.x86_64 2026-02-17T05:48:29.980 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:29.981 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:29.995 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:31 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:30.026 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-base 2026-02-17T05:48:30.046 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:48:30.047 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:48:30.059 INFO:teuthology.orchestra.run.trial016.stdout:Package ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:30.061 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:48:30.062 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:48:30.077 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:30.077 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:30.077 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:30.098 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-fuse 2026-02-17T05:48:30.115 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:48:30.167 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 94/99 2026-02-17T05:48:30.168 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:48:30.188 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:48:30.189 INFO:teuthology.orchestra.run.trial164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:30.189 INFO:teuthology.orchestra.run.trial164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-17T05:48:30.189 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:48:30.189 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:48:30.189 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:30.200 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:48:30.239 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:48:30.240 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:48:30.253 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:48:30.253 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:48:30.310 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:48:30.339 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 94/99 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:48:30.381 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:48:30.381 INFO:teuthology.orchestra.run.trial148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:30.381 INFO:teuthology.orchestra.run.trial148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-17T05:48:30.381 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:48:30.382 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:48:30.382 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:30.392 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:48:30.403 INFO:teuthology.orchestra.run.trial128.stdout:Package ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:30.409 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:32 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:30.420 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:30.421 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:30.421 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:30.425 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:48:30.427 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:48:30.443 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install cephadm 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:30.491 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:30.492 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:30.493 INFO:teuthology.orchestra.run.trial016.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:30.493 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:30.493 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:30.493 INFO:teuthology.orchestra.run.trial016.stdout: ceph-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.2 M 2026-02-17T05:48:30.493 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:30.493 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:30.493 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:30.493 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:48:30.493 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:30.493 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 1.2 M 2026-02-17T05:48:30.494 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 3.6 M 2026-02-17T05:48:30.494 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:30.554 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:48:30.574 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:48:30.574 INFO:teuthology.orchestra.run.trial164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:30.574 INFO:teuthology.orchestra.run.trial164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-17T05:48:30.574 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:48:30.575 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:48:30.575 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:30.593 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:48:30.606 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:48:30.608 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:48:30.612 INFO:teuthology.orchestra.run.trial016.stdout:ceph-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 9.6 MB/s | 1.2 MB 00:00 2026-02-17T05:48:30.613 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:30.613 INFO:teuthology.orchestra.run.trial016.stdout:Total 9.5 MB/s | 1.2 MB 00:00 2026-02-17T05:48:30.613 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:30.617 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:30.617 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:30.630 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:48:30.631 INFO:teuthology.orchestra.run.trial148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:30.631 INFO:teuthology.orchestra.run.trial148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-17T05:48:30.631 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:48:30.631 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:48:30.631 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:30.651 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:30.651 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:30.732 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:48:30.751 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:30.753 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:48:30.753 INFO:teuthology.orchestra.run.trial148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:30.753 INFO:teuthology.orchestra.run.trial148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-17T05:48:30.753 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:48:30.754 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:48:30.754 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:30.756 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:30.768 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:30.772 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:48:30.820 INFO:teuthology.orchestra.run.trial128.stdout:Package cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:30.837 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:30.838 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:30.838 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:30.860 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T05:48:31.008 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:31.066 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:31.066 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:31.066 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:31.066 INFO:teuthology.orchestra.run.trial016.stdout: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.066 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:31.066 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:31.093 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-volume 2026-02-17T05:48:31.170 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:31 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:31.247 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:48:31.248 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/99 2026-02-17T05:48:31.248 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/99 2026-02-17T05:48:31.248 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/99 2026-02-17T05:48:31.248 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/99 2026-02-17T05:48:31.248 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/99 2026-02-17T05:48:31.248 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 6/99 2026-02-17T05:48:31.248 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 7/99 2026-02-17T05:48:31.248 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 8/99 2026-02-17T05:48:31.248 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 9/99 2026-02-17T05:48:31.248 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 10/99 2026-02-17T05:48:31.249 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 11/99 2026-02-17T05:48:31.249 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 12/99 2026-02-17T05:48:31.249 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 13/99 2026-02-17T05:48:31.249 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 14/99 2026-02-17T05:48:31.249 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 15/99 2026-02-17T05:48:31.249 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 16/99 2026-02-17T05:48:31.249 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-17T05:48:31.249 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-17T05:48:31.249 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-17T05:48:31.249 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-17T05:48:31.250 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-17T05:48:31.251 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-17T05:48:31.251 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-17T05:48:31.251 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-17T05:48:31.251 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-17T05:48:31.251 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-17T05:48:31.251 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-17T05:48:31.251 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-17T05:48:31.251 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 39/99 2026-02-17T05:48:31.251 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-17T05:48:31.251 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-17T05:48:31.252 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-17T05:48:31.253 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-17T05:48:31.254 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-17T05:48:31.255 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-17T05:48:31.256 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-17T05:48:31.257 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:31.258 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:31.258 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:48:31.258 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:31.258 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:31.258 INFO:teuthology.orchestra.run.trial128.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 145 k 2026-02-17T05:48:31.258 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:31.258 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:31.258 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:31.258 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:48:31.259 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:31.259 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 145 k 2026-02-17T05:48:31.259 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 431 k 2026-02-17T05:48:31.259 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:31.311 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-17T05:48:31.312 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:31.312 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:31.312 INFO:teuthology.orchestra.run.trial164.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-17T05:48:31.312 INFO:teuthology.orchestra.run.trial164.stdout: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.312 INFO:teuthology.orchestra.run.trial164.stdout: ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.312 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.312 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.312 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.312 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-17T05:48:31.313 INFO:teuthology.orchestra.run.trial164.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-17T05:48:31.314 INFO:teuthology.orchestra.run.trial164.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-17T05:48:31.314 INFO:teuthology.orchestra.run.trial164.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-17T05:48:31.314 INFO:teuthology.orchestra.run.trial164.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-17T05:48:31.314 INFO:teuthology.orchestra.run.trial164.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-17T05:48:31.314 INFO:teuthology.orchestra.run.trial164.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-17T05:48:31.314 INFO:teuthology.orchestra.run.trial164.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-17T05:48:31.314 INFO:teuthology.orchestra.run.trial164.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-17T05:48:31.314 INFO:teuthology.orchestra.run.trial164.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:48:31.314 INFO:teuthology.orchestra.run.trial164.stdout: lua-5.4.4-4.el9.x86_64 2026-02-17T05:48:31.314 INFO:teuthology.orchestra.run.trial164.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-17T05:48:31.315 INFO:teuthology.orchestra.run.trial164.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-17T05:48:31.315 INFO:teuthology.orchestra.run.trial164.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-17T05:48:31.315 INFO:teuthology.orchestra.run.trial164.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-17T05:48:31.315 INFO:teuthology.orchestra.run.trial164.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-17T05:48:31.315 INFO:teuthology.orchestra.run.trial164.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-17T05:48:31.315 INFO:teuthology.orchestra.run.trial164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-17T05:48:31.315 INFO:teuthology.orchestra.run.trial164.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-17T05:48:31.315 INFO:teuthology.orchestra.run.trial164.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-17T05:48:31.315 INFO:teuthology.orchestra.run.trial164.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-17T05:48:31.315 INFO:teuthology.orchestra.run.trial164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-17T05:48:31.316 INFO:teuthology.orchestra.run.trial164.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:48:31.317 INFO:teuthology.orchestra.run.trial164.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-17T05:48:31.318 INFO:teuthology.orchestra.run.trial164.stdout: python3-ply-3.11-14.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-17T05:48:31.319 INFO:teuthology.orchestra.run.trial164.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-17T05:48:31.320 INFO:teuthology.orchestra.run.trial164.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-17T05:48:31.321 INFO:teuthology.orchestra.run.trial164.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-17T05:48:31.321 INFO:teuthology.orchestra.run.trial164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-17T05:48:31.321 INFO:teuthology.orchestra.run.trial164.stdout: unzip-6.0-59.el9.x86_64 2026-02-17T05:48:31.321 INFO:teuthology.orchestra.run.trial164.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-17T05:48:31.321 INFO:teuthology.orchestra.run.trial164.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-17T05:48:31.321 INFO:teuthology.orchestra.run.trial164.stdout: zip-3.0-35.el9.x86_64 2026-02-17T05:48:31.321 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:31.321 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:31.339 INFO:teuthology.orchestra.run.trial128.stdout:ceph-immutable-object-cache-19.2.3-705.g04cf6dd 1.7 MB/s | 145 kB 00:00 2026-02-17T05:48:31.339 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:31.339 INFO:teuthology.orchestra.run.trial128.stdout:Total 1.6 MB/s | 145 kB 00:00 2026-02-17T05:48:31.339 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:31.342 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:31.342 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:31.367 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-base 2026-02-17T05:48:31.373 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:31.373 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:31.401 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:31.413 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:48:31.413 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/99 2026-02-17T05:48:31.414 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/99 2026-02-17T05:48:31.414 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/99 2026-02-17T05:48:31.414 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/99 2026-02-17T05:48:31.414 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/99 2026-02-17T05:48:31.414 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 6/99 2026-02-17T05:48:31.414 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 7/99 2026-02-17T05:48:31.414 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 8/99 2026-02-17T05:48:31.414 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 9/99 2026-02-17T05:48:31.414 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 10/99 2026-02-17T05:48:31.415 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 11/99 2026-02-17T05:48:31.415 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 12/99 2026-02-17T05:48:31.415 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 13/99 2026-02-17T05:48:31.415 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 14/99 2026-02-17T05:48:31.415 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 15/99 2026-02-17T05:48:31.415 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 16/99 2026-02-17T05:48:31.415 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-17T05:48:31.415 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-17T05:48:31.415 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 39/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-17T05:48:31.416 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-17T05:48:31.417 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-17T05:48:31.418 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-17T05:48:31.428 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:31.429 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:31.451 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:31.451 INFO:teuthology.orchestra.run.trial128.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:31.451 INFO:teuthology.orchestra.run.trial128.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-17T05:48:31.451 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:31.464 INFO:teuthology.orchestra.run.trial016.stdout:Package ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:31.476 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-17T05:48:31.476 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:31.476 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:31.476 INFO:teuthology.orchestra.run.trial148.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-17T05:48:31.476 INFO:teuthology.orchestra.run.trial148.stdout: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.476 INFO:teuthology.orchestra.run.trial148.stdout: ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.477 INFO:teuthology.orchestra.run.trial148.stdout: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:48:31.478 INFO:teuthology.orchestra.run.trial148.stdout: lua-5.4.4-4.el9.x86_64 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-17T05:48:31.479 INFO:teuthology.orchestra.run.trial148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-17T05:48:31.480 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-17T05:48:31.481 INFO:teuthology.orchestra.run.trial148.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-ply-3.11-14.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-17T05:48:31.482 INFO:teuthology.orchestra.run.trial148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-17T05:48:31.483 INFO:teuthology.orchestra.run.trial148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-17T05:48:31.484 INFO:teuthology.orchestra.run.trial148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-17T05:48:31.484 INFO:teuthology.orchestra.run.trial148.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-17T05:48:31.484 INFO:teuthology.orchestra.run.trial148.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-17T05:48:31.484 INFO:teuthology.orchestra.run.trial148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-17T05:48:31.484 INFO:teuthology.orchestra.run.trial148.stdout: unzip-6.0-59.el9.x86_64 2026-02-17T05:48:31.484 INFO:teuthology.orchestra.run.trial148.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-17T05:48:31.484 INFO:teuthology.orchestra.run.trial148.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-17T05:48:31.484 INFO:teuthology.orchestra.run.trial148.stdout: zip-3.0-35.el9.x86_64 2026-02-17T05:48:31.484 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:31.484 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:31.485 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:31.485 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:31.485 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:31.503 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install librados-devel 2026-02-17T05:48:31.530 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-base 2026-02-17T05:48:31.685 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:29 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:31.749 INFO:teuthology.orchestra.run.trial164.stdout:Package ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:31.758 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:31.758 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:31.758 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:31.758 INFO:teuthology.orchestra.run.trial128.stdout: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:31.758 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:31.758 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:31.766 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:31.767 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:31.767 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:31.784 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-mgr 2026-02-17T05:48:31.788 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install cephadm 2026-02-17T05:48:31.811 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:31.843 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout: librados-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 126 k 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:48:31.892 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:31.893 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 126 k 2026-02-17T05:48:31.893 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 456 k 2026-02-17T05:48:31.893 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:31.908 INFO:teuthology.orchestra.run.trial148.stdout:Package ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:31.926 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:31.926 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:31.926 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:31.948 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install cephadm 2026-02-17T05:48:31.977 INFO:teuthology.orchestra.run.trial016.stdout:librados-devel-19.2.3-705.g04cf6dd4.el9.x86_64. 1.5 MB/s | 126 kB 00:00 2026-02-17T05:48:31.978 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:31.978 INFO:teuthology.orchestra.run.trial016.stdout:Total 1.5 MB/s | 126 kB 00:00 2026-02-17T05:48:31.978 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:31.980 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:31.980 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:32.011 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:32.011 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:32.065 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:32.082 INFO:teuthology.orchestra.run.trial016.stdout: Installing : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:32.092 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:32 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:32.095 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:32.156 INFO:teuthology.orchestra.run.trial128.stdout:Package ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:32.159 INFO:teuthology.orchestra.run.trial164.stdout:Package cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:32.162 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:32.173 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:32.174 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:32.174 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:32.176 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:32.176 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:32.176 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:32.196 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T05:48:32.197 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T05:48:32.220 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:32.221 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:32.221 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:32.221 INFO:teuthology.orchestra.run.trial016.stdout: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:32.221 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:32.221 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:32.247 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install libcephfs2 2026-02-17T05:48:32.256 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:32.321 INFO:teuthology.orchestra.run.trial148.stdout:Package cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:32.338 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:32.339 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:32.339 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:32.360 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T05:48:32.504 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:32 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:32.508 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:32.557 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:32.570 INFO:teuthology.orchestra.run.trial128.stdout:Package ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:32.588 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:32.588 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:32.589 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:32.590 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:32.591 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:32.591 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:48:32.591 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:32.591 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:32.591 INFO:teuthology.orchestra.run.trial164.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 145 k 2026-02-17T05:48:32.592 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:32.592 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:32.592 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:32.592 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:48:32.592 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:32.592 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 145 k 2026-02-17T05:48:32.593 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 431 k 2026-02-17T05:48:32.593 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:32.610 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T05:48:32.622 INFO:teuthology.orchestra.run.trial016.stdout:Package libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:32.639 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:32.640 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:32.640 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:32.662 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install libcephfs-devel 2026-02-17T05:48:32.666 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:32.676 INFO:teuthology.orchestra.run.trial164.stdout:ceph-immutable-object-cache-19.2.3-705.g04cf6dd 1.7 MB/s | 145 kB 00:00 2026-02-17T05:48:32.676 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:32.676 INFO:teuthology.orchestra.run.trial164.stdout:Total 1.7 MB/s | 145 kB 00:00 2026-02-17T05:48:32.676 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:32.679 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:32.680 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:32.711 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:32.711 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:32.749 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:32.749 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:32.750 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:48:32.750 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:32.750 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:32.750 INFO:teuthology.orchestra.run.trial148.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 145 k 2026-02-17T05:48:32.750 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:32.750 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:32.750 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:32.750 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:48:32.750 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:32.751 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 145 k 2026-02-17T05:48:32.751 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 431 k 2026-02-17T05:48:32.751 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:32.768 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:32.769 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:32.792 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:32.792 INFO:teuthology.orchestra.run.trial164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:32.792 INFO:teuthology.orchestra.run.trial164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-17T05:48:32.792 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:32.835 INFO:teuthology.orchestra.run.trial148.stdout:ceph-immutable-object-cache-19.2.3-705.g04cf6dd 1.7 MB/s | 145 kB 00:00 2026-02-17T05:48:32.835 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:32.835 INFO:teuthology.orchestra.run.trial148.stdout:Total 1.7 MB/s | 145 kB 00:00 2026-02-17T05:48:32.835 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:32.838 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:32.838 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:32.869 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:32.869 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:32.914 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:32 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:32.923 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:32.924 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:32.947 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:32.947 INFO:teuthology.orchestra.run.trial148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:32.947 INFO:teuthology.orchestra.run.trial148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-17T05:48:32.947 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:32.972 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:32.977 INFO:teuthology.orchestra.run.trial128.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:32.994 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:32.995 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:32.995 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:33.016 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-mgr-rook 2026-02-17T05:48:33.055 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:33.056 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:33.056 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:48:33.056 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:33.056 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:33.056 INFO:teuthology.orchestra.run.trial016.stdout: libcephfs-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 34 k 2026-02-17T05:48:33.056 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:33.056 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:33.056 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:33.056 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:48:33.057 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:33.057 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 34 k 2026-02-17T05:48:33.057 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 153 k 2026-02-17T05:48:33.057 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:33.073 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:33.073 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:33.073 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:33.073 INFO:teuthology.orchestra.run.trial164.stdout: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:33.073 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:33.073 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:33.099 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-mgr 2026-02-17T05:48:33.119 INFO:teuthology.orchestra.run.trial016.stdout:libcephfs-devel-19.2.3-705.g04cf6dd4.el9.x86_64 532 kB/s | 34 kB 00:00 2026-02-17T05:48:33.119 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:33.120 INFO:teuthology.orchestra.run.trial016.stdout:Total 526 kB/s | 34 kB 00:00 2026-02-17T05:48:33.120 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:33.121 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:33.121 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:33.123 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:33.123 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:33.144 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:33.160 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:33.214 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:33.226 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:48:33.226 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:33.227 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:33.227 INFO:teuthology.orchestra.run.trial148.stdout: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:33.227 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:33.227 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:33.254 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-mgr 2026-02-17T05:48:33.271 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:33.271 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:33.271 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:33.271 INFO:teuthology.orchestra.run.trial016.stdout: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:33.271 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:33.271 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:33.297 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install librados2 2026-02-17T05:48:33.320 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:33.383 INFO:teuthology.orchestra.run.trial128.stdout:Package ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:33.401 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:33.401 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:33.401 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:33.414 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:31 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:33.422 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T05:48:33.477 INFO:teuthology.orchestra.run.trial164.stdout:Package ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:33.494 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:33.495 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:33.495 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:33.516 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T05:48:33.566 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:33.609 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:33.631 INFO:teuthology.orchestra.run.trial148.stdout:Package ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:33.649 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:33.649 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:33.649 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:33.671 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T05:48:33.673 INFO:teuthology.orchestra.run.trial016.stdout:Package librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:33.690 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:33.691 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:33.691 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:33.713 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install librbd1 2026-02-17T05:48:33.733 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:33.797 INFO:teuthology.orchestra.run.trial128.stdout:Package ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:33.814 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:33.815 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:33.815 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:33.825 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:31 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:33.836 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-fuse 2026-02-17T05:48:33.889 INFO:teuthology.orchestra.run.trial164.stdout:Package ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:33.907 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:33.907 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:33.907 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:33.928 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T05:48:33.982 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:34.021 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:34.047 INFO:teuthology.orchestra.run.trial148.stdout:Package ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:34.065 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:34.066 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:34.066 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:34.086 INFO:teuthology.orchestra.run.trial016.stdout:Package librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:34.088 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T05:48:34.103 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:34.104 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:34.104 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:34.125 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install python3-rados 2026-02-17T05:48:34.149 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout: ceph-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.2 M 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 1.2 M 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 3.6 M 2026-02-17T05:48:34.231 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:34.236 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:32 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:34.300 INFO:teuthology.orchestra.run.trial164.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:34.319 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:34.320 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:34.320 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:34.341 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-mgr-rook 2026-02-17T05:48:34.352 INFO:teuthology.orchestra.run.trial128.stdout:ceph-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 9.6 MB/s | 1.2 MB 00:00 2026-02-17T05:48:34.353 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:34.353 INFO:teuthology.orchestra.run.trial128.stdout:Total 9.5 MB/s | 1.2 MB 00:00 2026-02-17T05:48:34.353 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:34.357 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:34.357 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:34.391 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:34.391 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:34.397 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:34.432 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:34.463 INFO:teuthology.orchestra.run.trial148.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:34.480 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:34.481 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:34.481 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:34.491 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:34.495 INFO:teuthology.orchestra.run.trial016.stdout:Package python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:34.503 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-mgr-rook 2026-02-17T05:48:34.508 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:34.513 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:34.513 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:34.514 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:34.534 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install python3-rgw 2026-02-17T05:48:34.648 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:32 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:34.711 INFO:teuthology.orchestra.run.trial164.stdout:Package ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:34.728 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:34.729 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:34.729 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:34.745 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:34.750 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T05:48:34.801 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:34.801 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:34.801 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:34.801 INFO:teuthology.orchestra.run.trial128.stdout: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:34.801 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:34.801 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:34.814 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:34.828 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-volume 2026-02-17T05:48:34.844 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:34.878 INFO:teuthology.orchestra.run.trial148.stdout:Package ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:34.896 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:34.897 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:34.897 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:34.909 INFO:teuthology.orchestra.run.trial016.stdout:Package python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:34.918 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T05:48:34.926 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:34.927 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:34.927 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:34.948 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install python3-cephfs 2026-02-17T05:48:35.061 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:35.125 INFO:teuthology.orchestra.run.trial164.stdout:Package ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:35.141 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:35.143 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:35.143 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:35.143 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:35.165 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-fuse 2026-02-17T05:48:35.205 INFO:teuthology.orchestra.run.trial128.stdout:Package ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:35.221 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:35.223 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:35.223 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:35.223 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:35.244 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install librados-devel 2026-02-17T05:48:35.256 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:35.287 INFO:teuthology.orchestra.run.trial148.stdout:Package ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:35.305 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:35.305 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:35.306 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:35.320 INFO:teuthology.orchestra.run.trial016.stdout:Package python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:35.326 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-fuse 2026-02-17T05:48:35.338 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:35.338 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:35.338 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:35.359 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install python3-rbd 2026-02-17T05:48:35.469 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:35.550 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:35.551 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:35.551 INFO:teuthology.orchestra.run.trial164.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:35.551 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:35.551 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:35.551 INFO:teuthology.orchestra.run.trial164.stdout: ceph-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.2 M 2026-02-17T05:48:35.551 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:35.551 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:35.551 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:35.552 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:48:35.552 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:35.552 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 1.2 M 2026-02-17T05:48:35.552 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 3.6 M 2026-02-17T05:48:35.552 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:35.559 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:35.634 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout: librados-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 126 k 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 126 k 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 456 k 2026-02-17T05:48:35.640 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:35.663 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:35.671 INFO:teuthology.orchestra.run.trial164.stdout:ceph-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 9.6 MB/s | 1.2 MB 00:00 2026-02-17T05:48:35.672 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:35.672 INFO:teuthology.orchestra.run.trial164.stdout:Total 9.5 MB/s | 1.2 MB 00:00 2026-02-17T05:48:35.672 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:35.676 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:35.676 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:35.710 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:35.710 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:35.717 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:35.719 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:35.719 INFO:teuthology.orchestra.run.trial148.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:35.719 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:35.719 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:35.720 INFO:teuthology.orchestra.run.trial148.stdout: ceph-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.2 M 2026-02-17T05:48:35.720 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:35.720 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:35.720 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:35.720 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:48:35.721 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:35.721 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 1.2 M 2026-02-17T05:48:35.721 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 3.6 M 2026-02-17T05:48:35.721 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:35.726 INFO:teuthology.orchestra.run.trial016.stdout:Package python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:35.728 INFO:teuthology.orchestra.run.trial128.stdout:librados-devel-19.2.3-705.g04cf6dd4.el9.x86_64. 1.4 MB/s | 126 kB 00:00 2026-02-17T05:48:35.728 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:35.728 INFO:teuthology.orchestra.run.trial128.stdout:Total 1.4 MB/s | 126 kB 00:00 2026-02-17T05:48:35.728 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:35.731 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:35.731 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:35.744 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:35.745 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:48:35.745 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:35.762 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:35.762 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:35.766 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install rbd-fuse 2026-02-17T05:48:35.810 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:35.816 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:35.827 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:35.833 INFO:teuthology.orchestra.run.trial128.stdout: Installing : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:35.838 INFO:teuthology.orchestra.run.trial148.stdout:ceph-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2026-02-17T05:48:35.838 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:35.839 INFO:teuthology.orchestra.run.trial148.stdout:Total 9.6 MB/s | 1.2 MB 00:00 2026-02-17T05:48:35.839 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:35.843 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:35.843 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:35.877 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:35.877 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:35.913 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:35.970 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:35.970 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:35.970 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:35.971 INFO:teuthology.orchestra.run.trial128.stdout: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:35.971 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:35.971 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:35.978 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:35.995 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:35.998 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install libcephfs2 2026-02-17T05:48:36.036 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:36.070 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:36.092 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:36.092 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:36.093 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:36.093 INFO:teuthology.orchestra.run.trial164.stdout: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:36.093 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:36.093 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:36.119 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-volume 2026-02-17T05:48:36.151 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:36.152 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:36.152 INFO:teuthology.orchestra.run.trial016.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:36.152 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:36.152 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:36.153 INFO:teuthology.orchestra.run.trial016.stdout: rbd-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 85 k 2026-02-17T05:48:36.153 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:36.153 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:36.153 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:36.153 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:48:36.153 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:36.153 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 85 k 2026-02-17T05:48:36.153 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 227 k 2026-02-17T05:48:36.153 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:36.212 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:36.226 INFO:teuthology.orchestra.run.trial016.stdout:rbd-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-17T05:48:36.226 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:36.226 INFO:teuthology.orchestra.run.trial016.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-17T05:48:36.226 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:36.230 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:36.230 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:36.261 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:36.261 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:36.268 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:36.268 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:36.268 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:36.268 INFO:teuthology.orchestra.run.trial148.stdout: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:36.268 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:36.268 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:36.297 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-volume 2026-02-17T05:48:36.312 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:36.315 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:36.329 INFO:teuthology.orchestra.run.trial016.stdout: Installing : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:36.378 INFO:teuthology.orchestra.run.trial128.stdout:Package libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:36.395 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:36.396 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:36.396 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:36.404 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:36.417 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install libcephfs-devel 2026-02-17T05:48:36.431 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:36.462 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:36.462 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:36.462 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:36.462 INFO:teuthology.orchestra.run.trial016.stdout: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:36.462 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:36.462 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:36.489 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install rbd-mirror 2026-02-17T05:48:36.494 INFO:teuthology.orchestra.run.trial164.stdout:Package ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:36.512 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:36.512 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:36.512 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:36.533 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install librados-devel 2026-02-17T05:48:36.615 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:36.679 INFO:teuthology.orchestra.run.trial148.stdout:Package ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:48:36.697 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:36.698 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:36.698 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:36.720 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install librados-devel 2026-02-17T05:48:36.731 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:36.802 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:36.812 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:36.813 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:36.813 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:48:36.814 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:36.814 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:36.814 INFO:teuthology.orchestra.run.trial128.stdout: libcephfs-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 34 k 2026-02-17T05:48:36.814 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:36.814 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:36.814 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:36.814 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:48:36.814 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:36.814 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 34 k 2026-02-17T05:48:36.814 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 153 k 2026-02-17T05:48:36.815 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:36.840 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:36.877 INFO:teuthology.orchestra.run.trial128.stdout:libcephfs-devel-19.2.3-705.g04cf6dd4.el9.x86_64 527 kB/s | 34 kB 00:00 2026-02-17T05:48:36.878 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:36.878 INFO:teuthology.orchestra.run.trial128.stdout:Total 518 kB/s | 34 kB 00:00 2026-02-17T05:48:36.879 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:36.880 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:36.880 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:36.882 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:36.882 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:36.884 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:36.885 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:36.885 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:48:36.885 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:36.885 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:36.885 INFO:teuthology.orchestra.run.trial016.stdout: rbd-mirror x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.1 M 2026-02-17T05:48:36.885 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:36.886 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:36.886 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:36.886 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:48:36.886 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:36.886 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 3.1 M 2026-02-17T05:48:36.886 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 13 M 2026-02-17T05:48:36.886 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:36.904 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:36.921 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:36.921 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:36.921 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:36.921 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:48:36.921 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:36.921 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:36.921 INFO:teuthology.orchestra.run.trial164.stdout: librados-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 126 k 2026-02-17T05:48:36.922 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:36.922 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:36.922 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:36.922 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:48:36.922 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:36.922 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 126 k 2026-02-17T05:48:36.922 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 456 k 2026-02-17T05:48:36.922 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:36.977 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.009 INFO:teuthology.orchestra.run.trial164.stdout:librados-devel-19.2.3-705.g04cf6dd4.el9.x86_64. 1.4 MB/s | 126 kB 00:00 2026-02-17T05:48:37.009 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:37.010 INFO:teuthology.orchestra.run.trial164.stdout:Total 1.4 MB/s | 126 kB 00:00 2026-02-17T05:48:37.010 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:37.012 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:37.012 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:37.028 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:37.033 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.034 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:37.034 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:37.034 INFO:teuthology.orchestra.run.trial128.stdout: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:37.034 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:37.034 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:37.035 INFO:teuthology.orchestra.run.trial016.stdout:rbd-mirror-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 21 MB/s | 3.1 MB 00:00 2026-02-17T05:48:37.036 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:37.036 INFO:teuthology.orchestra.run.trial016.stdout:Total 21 MB/s | 3.1 MB 00:00 2026-02-17T05:48:37.036 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:37.040 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:37.040 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:37.043 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:37.044 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:37.062 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install librados2 2026-02-17T05:48:37.082 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:37.083 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:37.098 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:37.113 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:37.114 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:37.114 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:48:37.114 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:37.114 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:37.114 INFO:teuthology.orchestra.run.trial148.stdout: librados-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 126 k 2026-02-17T05:48:37.114 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:37.114 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:37.114 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:37.115 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:48:37.115 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:37.115 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 126 k 2026-02-17T05:48:37.115 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 456 k 2026-02-17T05:48:37.115 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:37.115 INFO:teuthology.orchestra.run.trial164.stdout: Installing : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.200 INFO:teuthology.orchestra.run.trial148.stdout:librados-devel-19.2.3-705.g04cf6dd4.el9.x86_64. 1.5 MB/s | 126 kB 00:00 2026-02-17T05:48:37.200 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:37.200 INFO:teuthology.orchestra.run.trial148.stdout:Total 1.4 MB/s | 126 kB 00:00 2026-02-17T05:48:37.200 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:37.201 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.202 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:37.202 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:37.233 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:37.234 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:37.257 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.257 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:37.258 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:37.258 INFO:teuthology.orchestra.run.trial164.stdout: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:37.258 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:37.258 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:37.285 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install libcephfs2 2026-02-17T05:48:37.287 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:37.289 INFO:teuthology.orchestra.run.trial016.stdout: Installing : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.290 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:37.307 INFO:teuthology.orchestra.run.trial148.stdout: Installing : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.311 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.311 INFO:teuthology.orchestra.run.trial016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:37.311 INFO:teuthology.orchestra.run.trial016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-17T05:48:37.311 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:48:37.311 INFO:teuthology.orchestra.run.trial016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:48:37.311 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:37.374 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:37.388 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.439 INFO:teuthology.orchestra.run.trial128.stdout:Package librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:37.445 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.445 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:37.445 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:37.445 INFO:teuthology.orchestra.run.trial148.stdout: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:37.445 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:37.445 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:37.456 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:37.456 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:37.456 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:37.473 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install libcephfs2 2026-02-17T05:48:37.477 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install librbd1 2026-02-17T05:48:37.601 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:37.603 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:37.603 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:37.603 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:37.603 INFO:teuthology.orchestra.run.trial016.stdout: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:37.603 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:37.603 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:37.629 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install rbd-nbd 2026-02-17T05:48:37.665 INFO:teuthology.orchestra.run.trial164.stdout:Package libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:37.682 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:37.682 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:37.683 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:37.703 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install libcephfs-devel 2026-02-17T05:48:37.781 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:37.791 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:37.846 INFO:teuthology.orchestra.run.trial148.stdout:Package libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:37.857 INFO:teuthology.orchestra.run.trial128.stdout:Package librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:37.863 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:37.863 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:37.864 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:37.875 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:37.875 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:37.875 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:37.886 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install libcephfs-devel 2026-02-17T05:48:37.897 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install python3-rados 2026-02-17T05:48:37.939 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:38.010 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:38.020 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:38.021 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:38.021 INFO:teuthology.orchestra.run.trial016.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:38.021 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:38.022 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:38.022 INFO:teuthology.orchestra.run.trial016.stdout: rbd-nbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 171 k 2026-02-17T05:48:38.022 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:38.022 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:38.022 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:38.022 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:48:38.022 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:38.022 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 171 k 2026-02-17T05:48:38.022 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 490 k 2026-02-17T05:48:38.022 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:38.093 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:38.094 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:38.094 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:48:38.094 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:38.094 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:38.094 INFO:teuthology.orchestra.run.trial164.stdout: libcephfs-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 34 k 2026-02-17T05:48:38.094 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:38.094 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:38.094 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:38.094 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:48:38.094 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:38.095 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 34 k 2026-02-17T05:48:38.095 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 153 k 2026-02-17T05:48:38.095 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:38.108 INFO:teuthology.orchestra.run.trial016.stdout:rbd-nbd-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 1.9 MB/s | 171 kB 00:00 2026-02-17T05:48:38.109 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:38.109 INFO:teuthology.orchestra.run.trial016.stdout:Total 1.9 MB/s | 171 kB 00:00 2026-02-17T05:48:38.109 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:38.112 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:38.113 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:38.144 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:38.144 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:38.156 INFO:teuthology.orchestra.run.trial164.stdout:libcephfs-devel-19.2.3-705.g04cf6dd4.el9.x86_64 534 kB/s | 34 kB 00:00 2026-02-17T05:48:38.157 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:38.157 INFO:teuthology.orchestra.run.trial164.stdout:Total 524 kB/s | 34 kB 00:00 2026-02-17T05:48:38.158 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:38.159 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:38.160 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:38.162 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:38.162 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:38.183 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:38.196 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:41 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:38.199 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:38.199 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:38.208 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:38.215 INFO:teuthology.orchestra.run.trial016.stdout: Installing : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:38.253 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:38.272 INFO:teuthology.orchestra.run.trial128.stdout:Package python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:38.277 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:38.278 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:38.278 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:48:38.278 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:38.279 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:38.279 INFO:teuthology.orchestra.run.trial148.stdout: libcephfs-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 34 k 2026-02-17T05:48:38.279 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:38.279 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:38.279 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:38.279 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:48:38.279 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:38.279 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 34 k 2026-02-17T05:48:38.279 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 153 k 2026-02-17T05:48:38.279 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:38.289 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:38.290 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:38.290 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:38.298 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:38.310 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:38.310 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:38.310 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:38.311 INFO:teuthology.orchestra.run.trial164.stdout: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:38.311 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:38.311 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:38.312 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install python3-rgw 2026-02-17T05:48:38.336 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install librados2 2026-02-17T05:48:38.341 INFO:teuthology.orchestra.run.trial148.stdout:libcephfs-devel-19.2.3-705.g04cf6dd4.el9.x86_64 535 kB/s | 34 kB 00:00 2026-02-17T05:48:38.342 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:38.342 INFO:teuthology.orchestra.run.trial148.stdout:Total 529 kB/s | 34 kB 00:00 2026-02-17T05:48:38.342 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:38.343 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:38.344 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:38.345 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:38.345 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:38.355 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:38.356 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:38.356 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:38.356 INFO:teuthology.orchestra.run.trial016.stdout: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:38.356 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:38.356 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:38.367 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:38.383 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-base-debuginfo 2026-02-17T05:48:38.384 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:38.438 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:38.494 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:38.495 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:38.495 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:38.495 INFO:teuthology.orchestra.run.trial148.stdout: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:38.495 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:38.495 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:38.522 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install librados2 2026-02-17T05:48:38.626 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:38.645 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:38.691 INFO:teuthology.orchestra.run.trial128.stdout:Package python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:38.698 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:38.708 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:38.709 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:38.709 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:38.709 INFO:teuthology.orchestra.run.trial164.stdout:Package librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:38.726 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:38.727 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:38.727 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:38.730 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install python3-cephfs 2026-02-17T05:48:38.748 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install librbd1 2026-02-17T05:48:38.780 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:38.782 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:38.782 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:48:38.782 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:38.782 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:38.782 INFO:teuthology.orchestra.run.trial016.stdout: ceph-base-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 72 M 2026-02-17T05:48:38.782 INFO:teuthology.orchestra.run.trial016.stdout:Installing dependencies: 2026-02-17T05:48:38.782 INFO:teuthology.orchestra.run.trial016.stdout: ceph-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 24 M 2026-02-17T05:48:38.783 INFO:teuthology.orchestra.run.trial016.stdout:Installing weak dependencies: 2026-02-17T05:48:38.783 INFO:teuthology.orchestra.run.trial016.stdout: ceph-debugsource x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 11 M 2026-02-17T05:48:38.783 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:38.783 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:38.783 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:38.783 INFO:teuthology.orchestra.run.trial016.stdout:Install 3 Packages 2026-02-17T05:48:38.783 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:38.783 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 107 M 2026-02-17T05:48:38.783 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 654 M 2026-02-17T05:48:38.783 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:38.840 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:41 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:38.904 INFO:teuthology.orchestra.run.trial148.stdout:Package librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:38.922 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:38.922 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:38.922 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:38.944 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install librbd1 2026-02-17T05:48:39.042 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:39.057 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:39.106 INFO:teuthology.orchestra.run.trial128.stdout:Package python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:39.121 INFO:teuthology.orchestra.run.trial164.stdout:Package librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:39.124 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:39.124 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:39.124 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:39.138 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:39.139 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:39.140 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:39.147 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install python3-rbd 2026-02-17T05:48:39.160 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install python3-rados 2026-02-17T05:48:39.252 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:42 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:39.317 INFO:teuthology.orchestra.run.trial148.stdout:Package librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:39.335 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:39.335 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:39.336 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:39.357 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install python3-rados 2026-02-17T05:48:39.455 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:39.476 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:39.518 INFO:teuthology.orchestra.run.trial128.stdout:Package python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:39.535 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:39.536 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:48:39.536 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:39.539 INFO:teuthology.orchestra.run.trial164.stdout:Package python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:39.557 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install rbd-fuse 2026-02-17T05:48:39.558 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:39.558 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:39.558 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:39.579 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install python3-rgw 2026-02-17T05:48:39.602 INFO:teuthology.orchestra.run.trial016.stdout:(1/3): ceph-debugsource-19.2.3-705.g04cf6dd4.el 13 MB/s | 11 MB 00:00 2026-02-17T05:48:39.666 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:42 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:39.730 INFO:teuthology.orchestra.run.trial148.stdout:Package python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:39.748 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:39.749 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:39.749 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:39.769 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install python3-rgw 2026-02-17T05:48:39.870 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:39.888 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:39.953 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:39.953 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:39.953 INFO:teuthology.orchestra.run.trial128.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:39.953 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:39.953 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:39.953 INFO:teuthology.orchestra.run.trial128.stdout: rbd-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 85 k 2026-02-17T05:48:39.953 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:39.953 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:39.954 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:39.954 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:48:39.954 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:39.954 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 85 k 2026-02-17T05:48:39.954 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 227 k 2026-02-17T05:48:39.954 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:39.954 INFO:teuthology.orchestra.run.trial164.stdout:Package python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:39.969 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:39.970 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:39.970 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:39.991 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install python3-cephfs 2026-02-17T05:48:40.026 INFO:teuthology.orchestra.run.trial128.stdout:rbd-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-17T05:48:40.026 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:40.026 INFO:teuthology.orchestra.run.trial128.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-17T05:48:40.027 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:40.030 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:40.030 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:40.061 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:40.061 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:40.077 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:43 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:40.112 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:40.129 INFO:teuthology.orchestra.run.trial128.stdout: Installing : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:40.142 INFO:teuthology.orchestra.run.trial148.stdout:Package python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:40.159 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:40.160 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:40.160 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:40.182 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install python3-cephfs 2026-02-17T05:48:40.209 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:40.267 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:40.268 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:40.268 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:40.268 INFO:teuthology.orchestra.run.trial128.stdout: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:40.268 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:40.268 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:40.297 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install rbd-mirror 2026-02-17T05:48:40.301 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:40.365 INFO:teuthology.orchestra.run.trial164.stdout:Package python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:40.383 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:40.383 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:40.384 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:40.404 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install python3-rbd 2026-02-17T05:48:40.492 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:43 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:40.556 INFO:teuthology.orchestra.run.trial148.stdout:Package python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:40.573 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:40.574 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:40.574 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:40.596 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install python3-rbd 2026-02-17T05:48:40.611 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:40.693 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:40.693 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:40.694 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:48:40.694 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:40.694 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:40.694 INFO:teuthology.orchestra.run.trial128.stdout: rbd-mirror x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.1 M 2026-02-17T05:48:40.694 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:40.694 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:40.694 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:40.694 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:48:40.694 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:40.694 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 3.1 M 2026-02-17T05:48:40.695 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 13 M 2026-02-17T05:48:40.695 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:40.712 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:40.775 INFO:teuthology.orchestra.run.trial164.stdout:Package python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:40.792 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:40.793 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:48:40.793 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:40.813 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install rbd-fuse 2026-02-17T05:48:40.852 INFO:teuthology.orchestra.run.trial016.stdout:(2/3): ceph-base-debuginfo-19.2.3-705.g04cf6dd4 35 MB/s | 72 MB 00:02 2026-02-17T05:48:40.905 INFO:teuthology.orchestra.run.trial128.stdout:rbd-mirror-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 15 MB/s | 3.1 MB 00:00 2026-02-17T05:48:40.905 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:40.905 INFO:teuthology.orchestra.run.trial128.stdout:Total 15 MB/s | 3.1 MB 00:00 2026-02-17T05:48:40.905 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:40.906 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:43 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:40.909 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:40.910 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:40.952 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:40.952 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:40.970 INFO:teuthology.orchestra.run.trial148.stdout:Package python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:48:40.988 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:40.988 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:48:40.989 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:41.009 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install rbd-fuse 2026-02-17T05:48:41.061 INFO:teuthology.orchestra.run.trial016.stdout:(3/3): ceph-debuginfo-19.2.3-705.g04cf6dd4.el9. 11 MB/s | 24 MB 00:02 2026-02-17T05:48:41.063 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:41.063 INFO:teuthology.orchestra.run.trial016.stdout:Total 47 MB/s | 107 MB 00:02 2026-02-17T05:48:41.063 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:48:41.068 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:48:41.068 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:48:41.117 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:41.147 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:41.148 INFO:teuthology.orchestra.run.trial128.stdout: Installing : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:41.170 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:41.170 INFO:teuthology.orchestra.run.trial128.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:41.170 INFO:teuthology.orchestra.run.trial128.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-17T05:48:41.170 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:48:41.170 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:48:41.170 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:41.198 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:41.198 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:41.198 INFO:teuthology.orchestra.run.trial164.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:41.198 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:41.198 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:41.199 INFO:teuthology.orchestra.run.trial164.stdout: rbd-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 85 k 2026-02-17T05:48:41.199 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:41.199 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:41.199 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:41.199 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:48:41.199 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:41.200 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 85 k 2026-02-17T05:48:41.200 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 227 k 2026-02-17T05:48:41.200 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:41.273 INFO:teuthology.orchestra.run.trial164.stdout:rbd-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-17T05:48:41.273 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:41.273 INFO:teuthology.orchestra.run.trial164.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-17T05:48:41.273 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:41.276 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:41.277 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:41.308 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:41.308 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:41.323 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:44 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:41.359 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:41.375 INFO:teuthology.orchestra.run.trial164.stdout: Installing : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:41.407 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:41.408 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:41.408 INFO:teuthology.orchestra.run.trial148.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:41.408 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:41.408 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:41.408 INFO:teuthology.orchestra.run.trial148.stdout: rbd-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 85 k 2026-02-17T05:48:41.408 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:41.408 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:41.409 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:41.409 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:48:41.409 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:41.409 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 85 k 2026-02-17T05:48:41.409 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 227 k 2026-02-17T05:48:41.409 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:41.450 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:48:41.450 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:48:41.451 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:41.466 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:41.466 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:41.466 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:41.466 INFO:teuthology.orchestra.run.trial128.stdout: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:41.466 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:41.466 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:41.481 INFO:teuthology.orchestra.run.trial148.stdout:rbd-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-17T05:48:41.482 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:41.482 INFO:teuthology.orchestra.run.trial148.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-17T05:48:41.482 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:41.485 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:41.485 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:41.495 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install rbd-nbd 2026-02-17T05:48:41.510 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:41.510 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:41.511 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:41.511 INFO:teuthology.orchestra.run.trial164.stdout: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:41.511 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:41.511 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:41.516 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:41.516 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:41.539 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install rbd-mirror 2026-02-17T05:48:41.566 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:41.583 INFO:teuthology.orchestra.run.trial148.stdout: Installing : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:41.662 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:41.719 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:41.719 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:41.719 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:41.719 INFO:teuthology.orchestra.run.trial148.stdout: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:41.719 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:41.719 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:41.748 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install rbd-mirror 2026-02-17T05:48:41.812 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:41 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:41.853 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:41.893 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:41.894 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:41.894 INFO:teuthology.orchestra.run.trial128.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:41.894 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:41.894 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:41.894 INFO:teuthology.orchestra.run.trial128.stdout: rbd-nbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 171 k 2026-02-17T05:48:41.894 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:41.894 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:41.894 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:41.895 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:48:41.895 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:41.895 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 171 k 2026-02-17T05:48:41.895 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 490 k 2026-02-17T05:48:41.895 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:41.935 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:41.935 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:41.936 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:48:41.936 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:41.936 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:41.936 INFO:teuthology.orchestra.run.trial164.stdout: rbd-mirror x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.1 M 2026-02-17T05:48:41.936 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:41.936 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:41.936 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:41.936 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:48:41.937 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:41.937 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 3.1 M 2026-02-17T05:48:41.937 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 13 M 2026-02-17T05:48:41.937 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:41.981 INFO:teuthology.orchestra.run.trial128.stdout:rbd-nbd-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 1.9 MB/s | 171 kB 00:00 2026-02-17T05:48:41.981 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:41.981 INFO:teuthology.orchestra.run.trial128.stdout:Total 1.9 MB/s | 171 kB 00:00 2026-02-17T05:48:41.981 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:41.985 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:41.985 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:42.017 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:42.017 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:42.060 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:45 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:42.074 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:42.086 INFO:teuthology.orchestra.run.trial164.stdout:rbd-mirror-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 21 MB/s | 3.1 MB 00:00 2026-02-17T05:48:42.087 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:42.087 INFO:teuthology.orchestra.run.trial164.stdout:Total 21 MB/s | 3.1 MB 00:00 2026-02-17T05:48:42.087 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:42.091 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:42.091 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:42.092 INFO:teuthology.orchestra.run.trial128.stdout: Installing : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:42.132 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:42.132 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:42.142 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:42.143 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:42.143 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:48:42.143 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:42.143 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:42.143 INFO:teuthology.orchestra.run.trial148.stdout: rbd-mirror x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.1 M 2026-02-17T05:48:42.144 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:42.144 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:42.144 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:42.144 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:48:42.144 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:42.144 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 3.1 M 2026-02-17T05:48:42.144 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 13 M 2026-02-17T05:48:42.144 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:42.172 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:42.230 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:42.230 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:42.231 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:42.231 INFO:teuthology.orchestra.run.trial128.stdout: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:42.231 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:42.231 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:42.258 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-base-debuginfo 2026-02-17T05:48:42.294 INFO:teuthology.orchestra.run.trial148.stdout:rbd-mirror-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 21 MB/s | 3.1 MB 00:00 2026-02-17T05:48:42.294 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:42.295 INFO:teuthology.orchestra.run.trial148.stdout:Total 21 MB/s | 3.1 MB 00:00 2026-02-17T05:48:42.295 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:42.299 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:42.299 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:42.330 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:42.331 INFO:teuthology.orchestra.run.trial164.stdout: Installing : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:42.340 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:42.340 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:42.353 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:42.353 INFO:teuthology.orchestra.run.trial164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:42.354 INFO:teuthology.orchestra.run.trial164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-17T05:48:42.354 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:48:42.354 INFO:teuthology.orchestra.run.trial164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:48:42.354 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:42.534 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:42.535 INFO:teuthology.orchestra.run.trial148.stdout: Installing : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:42.559 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:42.559 INFO:teuthology.orchestra.run.trial148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:48:42.560 INFO:teuthology.orchestra.run.trial148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-17T05:48:42.560 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:48:42.560 INFO:teuthology.orchestra.run.trial148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:48:42.560 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:42.573 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:42 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:42.654 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:42.655 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:42.655 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:42.655 INFO:teuthology.orchestra.run.trial164.stdout: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:42.655 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:42.655 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:42.656 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:42.656 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout: ceph-base-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 72 M 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout:Installing dependencies: 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout: ceph-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 24 M 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout:Installing weak dependencies: 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout: ceph-debugsource x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 11 M 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout:Install 3 Packages 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:42.657 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 107 M 2026-02-17T05:48:42.658 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 654 M 2026-02-17T05:48:42.658 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:42.681 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install rbd-nbd 2026-02-17T05:48:42.859 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:42.859 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:42.860 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:42.860 INFO:teuthology.orchestra.run.trial148.stdout: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:42.860 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:42.860 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:42.890 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install rbd-nbd 2026-02-17T05:48:42.989 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:43.071 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:43.071 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:43.071 INFO:teuthology.orchestra.run.trial164.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:43.072 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:43.072 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:43.072 INFO:teuthology.orchestra.run.trial164.stdout: rbd-nbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 171 k 2026-02-17T05:48:43.072 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:43.072 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:43.072 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:43.072 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:48:43.073 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:43.073 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 171 k 2026-02-17T05:48:43.073 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 490 k 2026-02-17T05:48:43.073 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:43.197 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:46 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:43.280 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:43.280 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:43.280 INFO:teuthology.orchestra.run.trial148.stdout: Package Architecture Version Repository Size 2026-02-17T05:48:43.280 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:43.280 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:43.281 INFO:teuthology.orchestra.run.trial148.stdout: rbd-nbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 171 k 2026-02-17T05:48:43.281 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:43.281 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:43.281 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:43.281 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:48:43.281 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:43.281 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 171 k 2026-02-17T05:48:43.281 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 490 k 2026-02-17T05:48:43.281 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:43.409 INFO:teuthology.orchestra.run.trial128.stdout:(1/3): ceph-debugsource-19.2.3-705.g04cf6dd4.el 14 MB/s | 11 MB 00:00 2026-02-17T05:48:43.468 INFO:teuthology.orchestra.run.trial164.stdout:rbd-nbd-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 431 kB/s | 171 kB 00:00 2026-02-17T05:48:43.468 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:43.469 INFO:teuthology.orchestra.run.trial164.stdout:Total 430 kB/s | 171 kB 00:00 2026-02-17T05:48:43.469 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:43.472 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:43.473 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:43.504 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:43.504 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:43.559 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:43.576 INFO:teuthology.orchestra.run.trial164.stdout: Installing : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:43.583 INFO:teuthology.orchestra.run.trial148.stdout:rbd-nbd-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 565 kB/s | 171 kB 00:00 2026-02-17T05:48:43.585 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:43.585 INFO:teuthology.orchestra.run.trial148.stdout:Total 562 kB/s | 171 kB 00:00 2026-02-17T05:48:43.585 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:43.589 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:43.589 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:43.620 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:43.620 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:43.656 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:43.675 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:43.692 INFO:teuthology.orchestra.run.trial148.stdout: Installing : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:43.713 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:43.713 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:43.713 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:43.714 INFO:teuthology.orchestra.run.trial164.stdout: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:43.714 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:43.714 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:43.742 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-base-debuginfo 2026-02-17T05:48:43.774 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:43.831 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:48:43.831 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:43.832 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:43.832 INFO:teuthology.orchestra.run.trial148.stdout: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:43.832 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:43.832 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:43.861 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-base-debuginfo 2026-02-17T05:48:43.910 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:48:43.965 INFO:teuthology.orchestra.run.trial128.stdout:(2/3): ceph-debuginfo-19.2.3-705.g04cf6dd4.el9. 19 MB/s | 24 MB 00:01 2026-02-17T05:48:44.054 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:42 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:44.137 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:44.138 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:44.139 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:48:44.139 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:44.139 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:44.139 INFO:teuthology.orchestra.run.trial164.stdout: ceph-base-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 72 M 2026-02-17T05:48:44.139 INFO:teuthology.orchestra.run.trial164.stdout:Installing dependencies: 2026-02-17T05:48:44.139 INFO:teuthology.orchestra.run.trial164.stdout: ceph-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 24 M 2026-02-17T05:48:44.139 INFO:teuthology.orchestra.run.trial164.stdout:Installing weak dependencies: 2026-02-17T05:48:44.139 INFO:teuthology.orchestra.run.trial164.stdout: ceph-debugsource x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 11 M 2026-02-17T05:48:44.139 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:44.140 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:44.140 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:44.140 INFO:teuthology.orchestra.run.trial164.stdout:Install 3 Packages 2026-02-17T05:48:44.140 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:44.140 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 107 M 2026-02-17T05:48:44.140 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 654 M 2026-02-17T05:48:44.140 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:48:44.172 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:47 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:44.258 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:44.258 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:44.258 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:48:44.258 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:44.258 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:44.258 INFO:teuthology.orchestra.run.trial148.stdout: ceph-base-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 72 M 2026-02-17T05:48:44.258 INFO:teuthology.orchestra.run.trial148.stdout:Installing dependencies: 2026-02-17T05:48:44.259 INFO:teuthology.orchestra.run.trial148.stdout: ceph-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 24 M 2026-02-17T05:48:44.259 INFO:teuthology.orchestra.run.trial148.stdout:Installing weak dependencies: 2026-02-17T05:48:44.259 INFO:teuthology.orchestra.run.trial148.stdout: ceph-debugsource x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 11 M 2026-02-17T05:48:44.259 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:44.259 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:44.259 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:44.259 INFO:teuthology.orchestra.run.trial148.stdout:Install 3 Packages 2026-02-17T05:48:44.259 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:44.259 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 107 M 2026-02-17T05:48:44.259 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 654 M 2026-02-17T05:48:44.260 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:45.203 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/3 2026-02-17T05:48:45.385 INFO:teuthology.orchestra.run.trial128.stdout:(3/3): ceph-base-debuginfo-19.2.3-705.g04cf6dd4 26 MB/s | 72 MB 00:02 2026-02-17T05:48:45.388 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:45.388 INFO:teuthology.orchestra.run.trial128.stdout:Total 39 MB/s | 107 MB 00:02 2026-02-17T05:48:45.388 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:48:45.394 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:48:45.394 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:48:45.624 INFO:teuthology.orchestra.run.trial164.stdout:(1/3): ceph-debugsource-19.2.3-705.g04cf6dd4.el 7.3 MB/s | 11 MB 00:01 2026-02-17T05:48:45.739 INFO:teuthology.orchestra.run.trial148.stdout:(1/3): ceph-debugsource-19.2.3-705.g04cf6dd4.el 7.3 MB/s | 11 MB 00:01 2026-02-17T05:48:45.774 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:48:45.775 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:48:46.714 INFO:teuthology.orchestra.run.trial148.stdout:(2/3): ceph-debuginfo-19.2.3-705.g04cf6dd4.el9. 10 MB/s | 24 MB 00:02 2026-02-17T05:48:47.005 INFO:teuthology.orchestra.run.trial164.stdout:(2/3): ceph-debuginfo-19.2.3-705.g04cf6dd4.el9. 8.5 MB/s | 24 MB 00:02 2026-02-17T05:48:47.729 INFO:teuthology.orchestra.run.trial148.stdout:(3/3): ceph-base-debuginfo-19.2.3-705.g04cf6dd4 21 MB/s | 72 MB 00:03 2026-02-17T05:48:47.731 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:47.731 INFO:teuthology.orchestra.run.trial148.stdout:Total 31 MB/s | 107 MB 00:03 2026-02-17T05:48:47.731 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:48:47.736 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:48:47.736 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:48:48.116 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:48:48.117 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:48:48.221 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:48:48.969 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/3 2026-02-17T05:48:48.989 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 3/3 2026-02-17T05:48:49.004 INFO:teuthology.orchestra.run.trial164.stdout:(3/3): ceph-base-debuginfo-19.2.3-705.g04cf6dd4 15 MB/s | 72 MB 00:04 2026-02-17T05:48:49.006 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:48:49.006 INFO:teuthology.orchestra.run.trial164.stdout:Total 22 MB/s | 107 MB 00:04 2026-02-17T05:48:49.006 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:48:49.012 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:48:49.012 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:48:49.390 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:48:49.390 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:48:49.488 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/3 2026-02-17T05:48:49.543 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 3/3 2026-02-17T05:48:49.544 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/3 2026-02-17T05:48:49.544 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/3 2026-02-17T05:48:49.602 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/3 2026-02-17T05:48:49.602 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:49.602 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:48:49.602 INFO:teuthology.orchestra.run.trial016.stdout: ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:49.603 INFO:teuthology.orchestra.run.trial016.stdout: ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:49.603 INFO:teuthology.orchestra.run.trial016.stdout: ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:49.603 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:49.603 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:48:49.644 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-common-debuginfo 2026-02-17T05:48:49.957 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:00:51 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:48:50.038 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:48:50.039 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:50.039 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:48:50.039 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:50.039 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:48:50.039 INFO:teuthology.orchestra.run.trial016.stdout: ceph-common-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 474 M 2026-02-17T05:48:50.039 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:50.039 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:48:50.039 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:48:50.039 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:48:50.040 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:48:50.040 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 474 M 2026-02-17T05:48:50.040 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 2.0 G 2026-02-17T05:48:50.040 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:48:51.818 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:48:51.837 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/3 2026-02-17T05:48:53.083 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/3 2026-02-17T05:48:53.177 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/3 2026-02-17T05:48:53.196 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 3/3 2026-02-17T05:48:53.742 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 3/3 2026-02-17T05:48:53.743 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/3 2026-02-17T05:48:53.743 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/3 2026-02-17T05:48:53.802 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/3 2026-02-17T05:48:53.802 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:53.803 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:48:53.803 INFO:teuthology.orchestra.run.trial128.stdout: ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:53.803 INFO:teuthology.orchestra.run.trial128.stdout: ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:53.803 INFO:teuthology.orchestra.run.trial128.stdout: ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:53.803 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:53.803 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:48:53.844 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-common-debuginfo 2026-02-17T05:48:54.160 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:00:54 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:48:54.242 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:48:54.242 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:54.243 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:48:54.243 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:54.243 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:48:54.243 INFO:teuthology.orchestra.run.trial128.stdout: ceph-common-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 474 M 2026-02-17T05:48:54.243 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:54.243 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:48:54.243 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:48:54.243 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:48:54.243 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:48:54.243 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 474 M 2026-02-17T05:48:54.244 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 2.0 G 2026-02-17T05:48:54.244 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:48:55.515 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/3 2026-02-17T05:48:55.535 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 3/3 2026-02-17T05:48:56.076 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 3/3 2026-02-17T05:48:56.076 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/3 2026-02-17T05:48:56.076 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/3 2026-02-17T05:48:56.134 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/3 2026-02-17T05:48:56.134 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:56.134 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:48:56.134 INFO:teuthology.orchestra.run.trial148.stdout: ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:56.134 INFO:teuthology.orchestra.run.trial148.stdout: ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:56.134 INFO:teuthology.orchestra.run.trial148.stdout: ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:56.134 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:56.135 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:48:56.177 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-common-debuginfo 2026-02-17T05:48:56.495 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:48:56.578 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:48:56.579 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:56.579 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:48:56.579 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:56.579 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:48:56.579 INFO:teuthology.orchestra.run.trial148.stdout: ceph-common-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 474 M 2026-02-17T05:48:56.579 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:56.579 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:48:56.579 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:48:56.580 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:48:56.580 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:48:56.580 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 474 M 2026-02-17T05:48:56.580 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 2.0 G 2026-02-17T05:48:56.580 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:48:56.759 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/3 2026-02-17T05:48:56.778 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 3/3 2026-02-17T05:48:57.275 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 3/3 2026-02-17T05:48:57.275 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/3 2026-02-17T05:48:57.275 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/3 2026-02-17T05:48:57.333 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/3 2026-02-17T05:48:57.333 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:57.334 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:48:57.334 INFO:teuthology.orchestra.run.trial164.stdout: ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:57.334 INFO:teuthology.orchestra.run.trial164.stdout: ceph-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:57.334 INFO:teuthology.orchestra.run.trial164.stdout: ceph-debugsource-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:48:57.334 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:57.334 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:48:57.374 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-common-debuginfo 2026-02-17T05:48:57.683 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:48:57.765 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:48:57.765 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:57.766 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:48:57.766 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:57.766 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:48:57.766 INFO:teuthology.orchestra.run.trial164.stdout: ceph-common-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 474 M 2026-02-17T05:48:57.766 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:57.766 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:48:57.766 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:48:57.766 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:48:57.766 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:48:57.766 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 474 M 2026-02-17T05:48:57.767 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 2.0 G 2026-02-17T05:48:57.767 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:49:00.181 INFO:teuthology.orchestra.run.trial016.stdout:ceph-common-debuginfo-19.2.3-705.g04cf6dd4.el9. 47 MB/s | 474 MB 00:10 2026-02-17T05:49:00.182 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:49:00.182 INFO:teuthology.orchestra.run.trial016.stdout:Total 47 MB/s | 474 MB 00:10 2026-02-17T05:49:00.182 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:49:00.184 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:49:00.184 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:49:01.591 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:49:01.592 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:49:19.383 INFO:teuthology.orchestra.run.trial128.stdout:ceph-common-debuginfo-19.2.3-705.g04cf6dd4.el9. 19 MB/s | 474 MB 00:25 2026-02-17T05:49:19.384 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:49:19.384 INFO:teuthology.orchestra.run.trial128.stdout:Total 19 MB/s | 474 MB 00:25 2026-02-17T05:49:19.384 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:49:19.386 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:49:19.386 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:49:20.797 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:49:20.797 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:49:21.311 INFO:teuthology.orchestra.run.trial164.stdout:ceph-common-debuginfo-19.2.3-705.g04cf6dd4.el9. 20 MB/s | 474 MB 00:23 2026-02-17T05:49:21.312 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:49:21.313 INFO:teuthology.orchestra.run.trial164.stdout:Total 20 MB/s | 474 MB 00:23 2026-02-17T05:49:21.313 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:49:21.314 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:49:21.315 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:49:22.721 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:49:22.722 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:49:23.337 INFO:teuthology.orchestra.run.trial148.stdout:ceph-common-debuginfo-19.2.3-705.g04cf6dd4.el9. 18 MB/s | 474 MB 00:26 2026-02-17T05:49:23.338 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:49:23.339 INFO:teuthology.orchestra.run.trial148.stdout:Total 18 MB/s | 474 MB 00:26 2026-02-17T05:49:23.339 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:49:23.340 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:49:23.340 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:49:24.749 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:49:24.750 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:49:25.233 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:49:25.251 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:26.803 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:26.860 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:26.860 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:26.860 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:49:26.860 INFO:teuthology.orchestra.run.trial016.stdout: ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:49:26.860 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:26.860 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:49:26.933 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-17T05:49:27.245 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:01:29 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:49:27.328 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:49:27.328 INFO:teuthology.orchestra.run.trial016.stdout:======================================================================================= 2026-02-17T05:49:27.328 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:49:27.328 INFO:teuthology.orchestra.run.trial016.stdout:======================================================================================= 2026-02-17T05:49:27.328 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:49:27.328 INFO:teuthology.orchestra.run.trial016.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.5 M 2026-02-17T05:49:27.328 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:27.328 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:49:27.328 INFO:teuthology.orchestra.run.trial016.stdout:======================================================================================= 2026-02-17T05:49:27.328 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:49:27.329 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:27.329 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 1.5 M 2026-02-17T05:49:27.329 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 7.6 M 2026-02-17T05:49:27.329 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:49:27.454 INFO:teuthology.orchestra.run.trial016.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-70 12 MB/s | 1.5 MB 00:00 2026-02-17T05:49:27.455 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:49:27.455 INFO:teuthology.orchestra.run.trial016.stdout:Total 12 MB/s | 1.5 MB 00:00 2026-02-17T05:49:27.455 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:49:27.456 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:49:27.456 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:49:27.479 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:49:27.480 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:49:27.595 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:49:27.612 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:27.671 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:27.731 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:27.731 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:27.731 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:49:27.731 INFO:teuthology.orchestra.run.trial016.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:49:27.731 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:27.731 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:49:27.759 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-17T05:49:28.064 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:01:30 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:49:28.146 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:49:28.147 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:49:28.147 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:49:28.147 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:49:28.147 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:49:28.147 INFO:teuthology.orchestra.run.trial016.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 200 M 2026-02-17T05:49:28.147 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:28.147 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:49:28.147 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:49:28.147 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:49:28.147 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:28.148 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 200 M 2026-02-17T05:49:28.148 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 893 M 2026-02-17T05:49:28.148 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:49:31.947 INFO:teuthology.orchestra.run.trial016.stdout:ceph-radosgw-debuginfo-19.2.3-705.g04cf6dd4.el9 53 MB/s | 200 MB 00:03 2026-02-17T05:49:31.947 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:49:31.948 INFO:teuthology.orchestra.run.trial016.stdout:Total 53 MB/s | 200 MB 00:03 2026-02-17T05:49:31.948 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:49:31.950 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:49:31.950 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:49:32.552 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:49:32.553 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:49:42.805 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:49:42.822 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:49:43.534 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:49:43.593 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:49:43.593 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:43.594 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:49:43.594 INFO:teuthology.orchestra.run.trial016.stdout: ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:49:43.594 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:43.594 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:49:43.635 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-test-debuginfo 2026-02-17T05:49:43.944 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:49:44.026 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:49:44.026 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:49:44.026 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:49:44.026 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:49:44.027 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:49:44.027 INFO:teuthology.orchestra.run.trial016.stdout: ceph-test-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 729 M 2026-02-17T05:49:44.027 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:44.027 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:49:44.027 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:49:44.027 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:49:44.027 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:49:44.027 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 729 M 2026-02-17T05:49:44.027 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 3.6 G 2026-02-17T05:49:44.027 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:49:44.326 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:49:44.344 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:45.843 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:45.900 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:45.900 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:49:45.900 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:49:45.900 INFO:teuthology.orchestra.run.trial128.stdout: ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:49:45.900 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:49:45.900 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:49:45.973 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-17T05:49:46.278 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:49:46.287 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:49:46.297 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:46.369 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:49:46.369 INFO:teuthology.orchestra.run.trial128.stdout:======================================================================================= 2026-02-17T05:49:46.370 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:49:46.370 INFO:teuthology.orchestra.run.trial128.stdout:======================================================================================= 2026-02-17T05:49:46.370 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:49:46.370 INFO:teuthology.orchestra.run.trial128.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.5 M 2026-02-17T05:49:46.370 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:49:46.370 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:49:46.370 INFO:teuthology.orchestra.run.trial128.stdout:======================================================================================= 2026-02-17T05:49:46.371 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:49:46.371 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:49:46.371 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 1.5 M 2026-02-17T05:49:46.371 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 7.6 M 2026-02-17T05:49:46.371 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:49:46.929 INFO:teuthology.orchestra.run.trial128.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-70 2.7 MB/s | 1.5 MB 00:00 2026-02-17T05:49:46.930 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:49:46.930 INFO:teuthology.orchestra.run.trial128.stdout:Total 2.7 MB/s | 1.5 MB 00:00 2026-02-17T05:49:46.930 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:49:46.931 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:49:46.931 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:49:46.953 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:49:46.954 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:49:47.067 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:49:47.084 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:47.150 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:47.208 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:47.208 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:49:47.209 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:49:47.209 INFO:teuthology.orchestra.run.trial128.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:49:47.209 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:49:47.209 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:49:47.237 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-17T05:49:47.554 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:49:47.636 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 200 M 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 200 M 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 893 M 2026-02-17T05:49:47.637 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:49:47.837 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:47.895 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:47.896 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:49:47.896 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:49:47.896 INFO:teuthology.orchestra.run.trial164.stdout: ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:49:47.896 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:49:47.896 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:49:47.969 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-17T05:49:48.216 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:49:48.234 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:48.280 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:49:48.361 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:49:48.362 INFO:teuthology.orchestra.run.trial164.stdout:======================================================================================= 2026-02-17T05:49:48.362 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:49:48.362 INFO:teuthology.orchestra.run.trial164.stdout:======================================================================================= 2026-02-17T05:49:48.362 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:49:48.362 INFO:teuthology.orchestra.run.trial164.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.5 M 2026-02-17T05:49:48.362 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:49:48.362 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:49:48.362 INFO:teuthology.orchestra.run.trial164.stdout:======================================================================================= 2026-02-17T05:49:48.362 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:49:48.363 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:49:48.363 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 1.5 M 2026-02-17T05:49:48.363 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 7.6 M 2026-02-17T05:49:48.363 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:49:49.095 INFO:teuthology.orchestra.run.trial164.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-70 2.1 MB/s | 1.5 MB 00:00 2026-02-17T05:49:49.095 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:49:49.096 INFO:teuthology.orchestra.run.trial164.stdout:Total 2.1 MB/s | 1.5 MB 00:00 2026-02-17T05:49:49.096 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:49:49.097 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:49:49.097 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:49:49.119 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:49:49.119 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:49:49.232 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:49:49.250 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:49.309 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:49.367 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:49.368 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:49:49.368 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:49:49.368 INFO:teuthology.orchestra.run.trial164.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:49:49.368 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:49:49.368 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:49:49.394 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-17T05:49:49.703 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:49:49.759 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:49.785 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:49:49.786 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:49:49.786 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:49:49.787 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:49:49.787 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:49:49.787 INFO:teuthology.orchestra.run.trial164.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 200 M 2026-02-17T05:49:49.787 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:49:49.787 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:49:49.788 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:49:49.788 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:49:49.788 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:49:49.788 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 200 M 2026-02-17T05:49:49.788 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 893 M 2026-02-17T05:49:49.788 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:49:49.816 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:49:49.816 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:49:49.817 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:49:49.817 INFO:teuthology.orchestra.run.trial148.stdout: ceph-common-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:49:49.817 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:49:49.817 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:49:49.884 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-17T05:49:50.189 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:01:53 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:49:50.272 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:49:50.272 INFO:teuthology.orchestra.run.trial148.stdout:======================================================================================= 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout:======================================================================================= 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.5 M 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout:======================================================================================= 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 1.5 M 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 7.6 M 2026-02-17T05:49:50.273 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:49:50.849 INFO:teuthology.orchestra.run.trial148.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-70 2.7 MB/s | 1.5 MB 00:00 2026-02-17T05:49:50.850 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:49:50.850 INFO:teuthology.orchestra.run.trial148.stdout:Total 2.7 MB/s | 1.5 MB 00:00 2026-02-17T05:49:50.851 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:49:50.852 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:49:50.852 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:49:50.876 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:49:50.876 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:49:50.989 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:49:51.007 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:51.069 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:51.129 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g 1/1 2026-02-17T05:49:51.129 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:49:51.129 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:49:51.129 INFO:teuthology.orchestra.run.trial148.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:49:51.129 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:49:51.129 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:49:51.155 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-17T05:49:51.461 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:01:54 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:49:51.545 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:49:51.545 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:49:51.546 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:49:51.546 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:49:51.546 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:49:51.546 INFO:teuthology.orchestra.run.trial148.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 200 M 2026-02-17T05:49:51.546 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:49:51.546 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:49:51.546 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:49:51.546 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:49:51.546 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:49:51.546 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 200 M 2026-02-17T05:49:51.547 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 893 M 2026-02-17T05:49:51.547 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:49:59.275 INFO:teuthology.orchestra.run.trial128.stdout:ceph-radosgw-debuginfo-19.2.3-705.g04cf6dd4.el9 17 MB/s | 200 MB 00:11 2026-02-17T05:49:59.276 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:49:59.276 INFO:teuthology.orchestra.run.trial128.stdout:Total 17 MB/s | 200 MB 00:11 2026-02-17T05:49:59.276 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:49:59.278 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:49:59.278 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:49:59.882 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:49:59.883 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:50:04.081 INFO:teuthology.orchestra.run.trial016.stdout:ceph-test-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 36 MB/s | 729 MB 00:20 2026-02-17T05:50:04.082 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:50:04.083 INFO:teuthology.orchestra.run.trial016.stdout:Total 36 MB/s | 729 MB 00:20 2026-02-17T05:50:04.083 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:50:04.085 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:50:04.085 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:50:05.450 INFO:teuthology.orchestra.run.trial148.stdout:ceph-radosgw-debuginfo-19.2.3-705.g04cf6dd4.el9 14 MB/s | 200 MB 00:13 2026-02-17T05:50:05.451 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:50:05.451 INFO:teuthology.orchestra.run.trial148.stdout:Total 14 MB/s | 200 MB 00:13 2026-02-17T05:50:05.452 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:50:05.453 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:50:05.454 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:50:06.059 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:50:06.059 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:50:06.245 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:50:06.246 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:50:07.482 INFO:teuthology.orchestra.run.trial164.stdout:ceph-radosgw-debuginfo-19.2.3-705.g04cf6dd4.el9 11 MB/s | 200 MB 00:17 2026-02-17T05:50:07.483 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:50:07.483 INFO:teuthology.orchestra.run.trial164.stdout:Total 11 MB/s | 200 MB 00:17 2026-02-17T05:50:07.483 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:50:07.485 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:50:07.485 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:50:08.094 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:50:08.094 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:50:09.927 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:50:09.945 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:50:10.647 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:50:10.704 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:50:10.705 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:50:10.705 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:50:10.705 INFO:teuthology.orchestra.run.trial128.stdout: ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:50:10.705 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:50:10.705 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:50:10.750 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-test-debuginfo 2026-02-17T05:50:11.061 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:02:11 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:50:11.143 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout: ceph-test-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 729 M 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 729 M 2026-02-17T05:50:11.144 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 3.6 G 2026-02-17T05:50:11.145 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:50:16.270 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:50:16.288 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:50:16.973 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:50:17.031 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:50:17.032 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:50:17.032 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:50:17.032 INFO:teuthology.orchestra.run.trial148.stdout: ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:50:17.032 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:50:17.032 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:50:17.076 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-test-debuginfo 2026-02-17T05:50:17.385 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:02:20 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout: ceph-test-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 729 M 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 729 M 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 3.6 G 2026-02-17T05:50:17.468 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:50:18.491 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:50:18.509 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:50:19.206 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:50:19.265 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x8 1/1 2026-02-17T05:50:19.265 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:50:19.265 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:50:19.265 INFO:teuthology.orchestra.run.trial164.stdout: ceph-radosgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:50:19.265 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:50:19.265 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:50:19.308 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-test-debuginfo 2026-02-17T05:50:19.617 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:02:17 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:50:19.699 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout: ceph-test-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 729 M 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 729 M 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 3.6 G 2026-02-17T05:50:19.700 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:50:27.563 INFO:teuthology.orchestra.run.trial128.stdout:ceph-test-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 44 MB/s | 729 MB 00:16 2026-02-17T05:50:27.564 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:50:27.564 INFO:teuthology.orchestra.run.trial128.stdout:Total 44 MB/s | 729 MB 00:16 2026-02-17T05:50:27.564 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:50:27.567 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:50:27.567 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:50:29.721 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:50:29.721 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:50:45.852 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:50:45.872 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:50:46.568 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:50:46.626 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:50:46.626 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:46.626 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:50:46.627 INFO:teuthology.orchestra.run.trial016.stdout: ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:50:46.627 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:46.627 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:50:46.725 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-base-debuginfo 2026-02-17T05:50:47.035 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:02:49 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:50:47.098 INFO:teuthology.orchestra.run.trial016.stdout:Package ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:50:47.116 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:50:47.117 INFO:teuthology.orchestra.run.trial016.stdout:Nothing to do. 2026-02-17T05:50:47.117 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:50:47.139 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-mgr-debuginfo 2026-02-17T05:50:47.449 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:02:49 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:50:47.531 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:50:47.532 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:50:47.532 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:50:47.532 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:50:47.532 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:50:47.532 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 18 M 2026-02-17T05:50:47.532 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:47.532 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:50:47.532 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:50:47.532 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:50:47.532 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:47.533 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 18 M 2026-02-17T05:50:47.533 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 110 M 2026-02-17T05:50:47.533 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:50:49.386 INFO:teuthology.orchestra.run.trial016.stdout:ceph-mgr-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 9.7 MB/s | 18 MB 00:01 2026-02-17T05:50:49.387 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:50:49.387 INFO:teuthology.orchestra.run.trial016.stdout:Total 9.7 MB/s | 18 MB 00:01 2026-02-17T05:50:49.387 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:50:49.389 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:50:49.389 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:50:49.460 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:50:49.460 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:50:50.497 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:50:50.515 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:50:50.657 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:50:50.715 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:50:50.715 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:50.715 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:50:50.715 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:50:50.715 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:50.715 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:50:50.743 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-mds-debuginfo 2026-02-17T05:50:51.052 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:02:53 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:50:51.133 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 55 M 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 55 M 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 211 M 2026-02-17T05:50:51.134 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:50:53.395 INFO:teuthology.orchestra.run.trial164.stdout:ceph-test-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 22 MB/s | 729 MB 00:33 2026-02-17T05:50:53.396 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:50:53.396 INFO:teuthology.orchestra.run.trial164.stdout:Total 22 MB/s | 729 MB 00:33 2026-02-17T05:50:53.396 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:50:53.399 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:50:53.399 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:50:53.463 INFO:teuthology.orchestra.run.trial016.stdout:ceph-mds-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 24 MB/s | 55 MB 00:02 2026-02-17T05:50:53.464 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:50:53.464 INFO:teuthology.orchestra.run.trial016.stdout:Total 24 MB/s | 55 MB 00:02 2026-02-17T05:50:53.464 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:50:53.466 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:50:53.466 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:50:53.645 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:50:53.646 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:50:55.570 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:50:55.570 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:50:55.621 INFO:teuthology.orchestra.run.trial148.stdout:ceph-test-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 19 MB/s | 729 MB 00:38 2026-02-17T05:50:55.622 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:50:55.622 INFO:teuthology.orchestra.run.trial148.stdout:Total 19 MB/s | 729 MB 00:38 2026-02-17T05:50:55.622 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:50:55.625 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:50:55.625 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:50:56.486 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:50:56.504 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:50:56.693 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:50:56.750 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:50:56.750 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:56.750 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:50:56.750 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:50:56.750 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:56.750 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:50:56.783 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-mon-debuginfo 2026-02-17T05:50:57.093 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:02:59 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:50:57.175 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:50:57.175 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:50:57.175 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 102 M 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 102 M 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 497 M 2026-02-17T05:50:57.176 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:50:57.777 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:50:57.777 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:50:59.163 INFO:teuthology.orchestra.run.trial016.stdout:ceph-mon-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 51 MB/s | 102 MB 00:01 2026-02-17T05:50:59.164 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:50:59.165 INFO:teuthology.orchestra.run.trial016.stdout:Total 51 MB/s | 102 MB 00:01 2026-02-17T05:50:59.165 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:50:59.166 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:50:59.166 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:50:59.485 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:50:59.486 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:04.701 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:04.719 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:05.106 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:05.163 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:05.164 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:05.164 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:05.164 INFO:teuthology.orchestra.run.trial016.stdout: ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:05.164 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:05.164 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:05.199 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-osd-debuginfo 2026-02-17T05:51:05.503 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:07 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:05.586 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:05.586 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:05.586 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:05.586 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:05.586 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:05.587 INFO:teuthology.orchestra.run.trial016.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 298 M 2026-02-17T05:51:05.587 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:05.587 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:05.587 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:05.587 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:05.587 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:05.587 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 298 M 2026-02-17T05:51:05.587 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 1.4 G 2026-02-17T05:51:05.587 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:07.386 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:51:07.406 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:08.076 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:08.134 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:08.134 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:08.134 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:51:08.134 INFO:teuthology.orchestra.run.trial128.stdout: ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:08.134 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:08.135 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:51:08.230 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-base-debuginfo 2026-02-17T05:51:08.545 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:03:08 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:51:08.609 INFO:teuthology.orchestra.run.trial128.stdout:Package ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:51:08.627 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:51:08.628 INFO:teuthology.orchestra.run.trial128.stdout:Nothing to do. 2026-02-17T05:51:08.628 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:51:08.650 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-mgr-debuginfo 2026-02-17T05:51:08.959 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:03:08 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:51:09.042 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:51:09.042 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:09.042 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:51:09.042 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:09.042 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:51:09.042 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 18 M 2026-02-17T05:51:09.042 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:09.042 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:51:09.043 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:09.043 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:51:09.043 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:09.043 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 18 M 2026-02-17T05:51:09.043 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 110 M 2026-02-17T05:51:09.043 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:51:10.384 INFO:teuthology.orchestra.run.trial128.stdout:ceph-mgr-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 13 MB/s | 18 MB 00:01 2026-02-17T05:51:10.385 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:10.386 INFO:teuthology.orchestra.run.trial128.stdout:Total 13 MB/s | 18 MB 00:01 2026-02-17T05:51:10.386 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:51:10.387 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:51:10.387 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:51:10.458 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:51:10.458 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:51:11.491 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:51:11.508 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:11.620 INFO:teuthology.orchestra.run.trial016.stdout:ceph-osd-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 49 MB/s | 298 MB 00:06 2026-02-17T05:51:11.620 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:11.621 INFO:teuthology.orchestra.run.trial016.stdout:Total 49 MB/s | 298 MB 00:06 2026-02-17T05:51:11.621 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:11.622 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:11.623 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:11.638 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:11.695 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:11.696 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:11.696 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:51:11.696 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:11.696 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:11.696 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:51:11.724 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-mds-debuginfo 2026-02-17T05:51:12.033 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:03:12 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 55 M 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:12.114 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 55 M 2026-02-17T05:51:12.115 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 211 M 2026-02-17T05:51:12.115 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:51:12.513 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:12.513 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:13.225 INFO:teuthology.orchestra.run.trial128.stdout:ceph-mds-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 50 MB/s | 55 MB 00:01 2026-02-17T05:51:13.226 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:13.226 INFO:teuthology.orchestra.run.trial128.stdout:Total 50 MB/s | 55 MB 00:01 2026-02-17T05:51:13.226 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:51:13.227 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:51:13.228 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:51:13.406 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:51:13.407 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:51:16.084 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:51:16.103 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:16.291 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:16.349 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:16.349 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:16.349 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:51:16.349 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:16.350 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:16.350 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:51:16.382 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-mon-debuginfo 2026-02-17T05:51:16.693 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:03:16 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:51:16.775 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:51:16.775 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:16.776 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:51:16.776 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:16.776 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:51:16.776 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 102 M 2026-02-17T05:51:16.776 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:16.776 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:51:16.776 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:16.776 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:51:16.776 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:16.776 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 102 M 2026-02-17T05:51:16.777 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 497 M 2026-02-17T05:51:16.777 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:51:18.757 INFO:teuthology.orchestra.run.trial128.stdout:ceph-mon-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 51 MB/s | 102 MB 00:01 2026-02-17T05:51:18.758 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:18.758 INFO:teuthology.orchestra.run.trial128.stdout:Total 51 MB/s | 102 MB 00:01 2026-02-17T05:51:18.758 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:51:18.760 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:51:18.760 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:51:19.079 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:51:19.079 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:51:24.510 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:51:24.527 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:24.921 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:24.978 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:24.978 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:24.978 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:51:24.978 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:24.978 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:24.978 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:51:25.014 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-osd-debuginfo 2026-02-17T05:51:25.326 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:03:25 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:51:25.407 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 298 M 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 298 M 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 1.4 G 2026-02-17T05:51:25.408 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:51:27.503 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:27.521 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:28.576 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:28.635 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:28.635 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:28.635 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:28.635 INFO:teuthology.orchestra.run.trial016.stdout: ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:28.635 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:28.635 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:28.692 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install ceph-fuse-debuginfo 2026-02-17T05:51:29.000 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:31 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.8 M 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 6.8 M 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 36 M 2026-02-17T05:51:29.082 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:30.813 INFO:teuthology.orchestra.run.trial016.stdout:ceph-fuse-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 4.0 MB/s | 6.8 MB 00:01 2026-02-17T05:51:30.813 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:30.813 INFO:teuthology.orchestra.run.trial016.stdout:Total 4.0 MB/s | 6.8 MB 00:01 2026-02-17T05:51:30.814 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:30.815 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:30.815 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:30.853 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:30.853 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:31.112 INFO:teuthology.orchestra.run.trial128.stdout:ceph-osd-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 52 MB/s | 298 MB 00:05 2026-02-17T05:51:31.112 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:31.112 INFO:teuthology.orchestra.run.trial128.stdout:Total 52 MB/s | 298 MB 00:05 2026-02-17T05:51:31.113 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:51:31.115 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:51:31.115 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:51:31.253 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:31.271 INFO:teuthology.orchestra.run.trial016.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:31.354 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:31.412 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:31.412 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:31.412 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:31.412 INFO:teuthology.orchestra.run.trial016.stdout: ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:31.412 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:31.412 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:31.436 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install librados-devel-debuginfo 2026-02-17T05:51:31.746 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:33 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:31.827 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:31.827 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout: librados-devel-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 112 k 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 112 k 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 493 k 2026-02-17T05:51:31.828 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:31.925 INFO:teuthology.orchestra.run.trial016.stdout:librados-devel-debuginfo-19.2.3-705.g04cf6dd4.e 1.1 MB/s | 112 kB 00:00 2026-02-17T05:51:31.926 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:31.926 INFO:teuthology.orchestra.run.trial016.stdout:Total 1.1 MB/s | 112 kB 00:00 2026-02-17T05:51:31.926 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:31.928 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:31.928 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:31.946 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:31.946 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:31.986 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:32.003 INFO:teuthology.orchestra.run.trial016.stdout: Installing : librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:51:32.007 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:51:32.008 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:51:32.058 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:51:32.115 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:51:32.116 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:32.116 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:32.116 INFO:teuthology.orchestra.run.trial016.stdout: librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:32.116 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:32.116 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:32.142 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install libcephfs2-debuginfo 2026-02-17T05:51:32.450 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:34 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:32.532 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:32.533 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:32.533 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:32.533 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:32.533 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:32.533 INFO:teuthology.orchestra.run.trial016.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.0 M 2026-02-17T05:51:32.533 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:32.533 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:32.534 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:32.534 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:32.534 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:32.534 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 6.0 M 2026-02-17T05:51:32.534 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 31 M 2026-02-17T05:51:32.534 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:32.736 INFO:teuthology.orchestra.run.trial016.stdout:libcephfs2-debuginfo-19.2.3-705.g04cf6dd4.el9.x 29 MB/s | 6.0 MB 00:00 2026-02-17T05:51:32.737 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:32.737 INFO:teuthology.orchestra.run.trial016.stdout:Total 29 MB/s | 6.0 MB 00:00 2026-02-17T05:51:32.737 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:32.738 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:32.738 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:32.774 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:32.775 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:33.126 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:33.144 INFO:teuthology.orchestra.run.trial016.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:51:33.226 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:51:33.286 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:51:33.286 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:33.286 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:33.286 INFO:teuthology.orchestra.run.trial016.stdout: libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:33.286 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:33.287 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:33.291 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:51:33.312 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:33.315 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install librados2-debuginfo 2026-02-17T05:51:33.623 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:35 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:33.706 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:33.706 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:33.706 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout: librados2-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 47 M 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 47 M 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 302 M 2026-02-17T05:51:33.707 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:34.675 INFO:teuthology.orchestra.run.trial016.stdout:librados2-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 49 MB/s | 47 MB 00:00 2026-02-17T05:51:34.675 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:34.675 INFO:teuthology.orchestra.run.trial016.stdout:Total 49 MB/s | 47 MB 00:00 2026-02-17T05:51:34.676 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:34.678 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:34.678 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:34.836 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:34.837 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:35.402 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:51:35.533 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:35.688 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:35.746 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:35.746 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:35.746 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:51:35.746 INFO:teuthology.orchestra.run.trial164.stdout: ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:35.746 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:35.746 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:51:35.841 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-base-debuginfo 2026-02-17T05:51:36.158 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:03:34 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:51:36.221 INFO:teuthology.orchestra.run.trial164.stdout:Package ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:51:36.239 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:51:36.240 INFO:teuthology.orchestra.run.trial164.stdout:Nothing to do. 2026-02-17T05:51:36.240 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:51:36.262 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-mgr-debuginfo 2026-02-17T05:51:36.577 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:03:34 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:51:36.659 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:51:36.659 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:36.659 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:51:36.659 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:36.659 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:51:36.659 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 18 M 2026-02-17T05:51:36.659 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:36.660 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:51:36.660 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:36.660 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:51:36.660 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:36.660 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 18 M 2026-02-17T05:51:36.660 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 110 M 2026-02-17T05:51:36.660 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:51:36.960 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:37.017 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:37.017 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:37.017 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:51:37.017 INFO:teuthology.orchestra.run.trial148.stdout: ceph-test-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:37.017 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:37.017 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:51:37.090 INFO:teuthology.orchestra.run.trial164.stdout:ceph-mgr-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 42 MB/s | 18 MB 00:00 2026-02-17T05:51:37.091 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:37.091 INFO:teuthology.orchestra.run.trial164.stdout:Total 42 MB/s | 18 MB 00:00 2026-02-17T05:51:37.091 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:51:37.093 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:51:37.093 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:51:37.112 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-base-debuginfo 2026-02-17T05:51:37.165 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:51:37.165 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:51:37.420 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:03:40 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:51:37.481 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:37.485 INFO:teuthology.orchestra.run.trial148.stdout:Package ceph-base-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:51:37.498 INFO:teuthology.orchestra.run.trial016.stdout: Installing : librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:37.502 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:51:37.503 INFO:teuthology.orchestra.run.trial148.stdout:Nothing to do. 2026-02-17T05:51:37.503 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:51:37.525 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-mgr-debuginfo 2026-02-17T05:51:37.769 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:37.827 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:37.828 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:37.828 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:37.828 INFO:teuthology.orchestra.run.trial016.stdout: librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:37.828 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:37.828 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:37.830 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:03:40 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:51:37.860 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install librbd1-debuginfo 2026-02-17T05:51:37.913 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:51:37.913 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:37.913 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 18 M 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 18 M 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 110 M 2026-02-17T05:51:37.914 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:51:38.171 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:40 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:38.179 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:51:38.197 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:38.253 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:38.253 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:38.254 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:38.254 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:38.254 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:38.254 INFO:teuthology.orchestra.run.trial016.stdout: librbd1-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 54 M 2026-02-17T05:51:38.254 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:38.254 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:38.254 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:38.254 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:38.254 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:38.254 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 54 M 2026-02-17T05:51:38.255 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 374 M 2026-02-17T05:51:38.255 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:38.332 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:38.349 INFO:teuthology.orchestra.run.trial148.stdout:ceph-mgr-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 41 MB/s | 18 MB 00:00 2026-02-17T05:51:38.350 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:38.350 INFO:teuthology.orchestra.run.trial148.stdout:Total 41 MB/s | 18 MB 00:00 2026-02-17T05:51:38.351 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:51:38.352 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:51:38.352 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:51:38.391 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:38.392 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:38.392 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:51:38.392 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:38.392 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:38.392 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:51:38.421 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-mds-debuginfo 2026-02-17T05:51:38.423 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:51:38.424 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:51:38.737 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:03:36 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:51:38.819 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:51:38.819 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:38.819 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:51:38.819 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:38.820 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:51:38.820 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 55 M 2026-02-17T05:51:38.820 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:38.820 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:51:38.820 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:38.820 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:51:38.820 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:38.821 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 55 M 2026-02-17T05:51:38.821 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 211 M 2026-02-17T05:51:38.821 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:51:39.418 INFO:teuthology.orchestra.run.trial016.stdout:librbd1-debuginfo-19.2.3-705.g04cf6dd4.el9.x86_ 47 MB/s | 54 MB 00:01 2026-02-17T05:51:39.419 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:39.419 INFO:teuthology.orchestra.run.trial016.stdout:Total 47 MB/s | 54 MB 00:01 2026-02-17T05:51:39.419 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:39.421 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:39.421 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:39.481 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:51:39.499 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:39.600 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:39.600 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:39.639 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:39.696 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:39.696 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:39.696 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:51:39.697 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mgr-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:39.697 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:39.697 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:51:39.725 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-mds-debuginfo 2026-02-17T05:51:40.036 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:03:43 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:51:40.118 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 55 M 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 55 M 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 211 M 2026-02-17T05:51:40.119 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:51:40.502 INFO:teuthology.orchestra.run.trial164.stdout:ceph-mds-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 33 MB/s | 55 MB 00:01 2026-02-17T05:51:40.503 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:40.503 INFO:teuthology.orchestra.run.trial164.stdout:Total 33 MB/s | 55 MB 00:01 2026-02-17T05:51:40.503 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:51:40.506 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:51:40.506 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:51:40.687 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:51:40.687 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:51:41.776 INFO:teuthology.orchestra.run.trial148.stdout:ceph-mds-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 33 MB/s | 55 MB 00:01 2026-02-17T05:51:41.777 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:41.777 INFO:teuthology.orchestra.run.trial148.stdout:Total 33 MB/s | 55 MB 00:01 2026-02-17T05:51:41.777 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:51:41.778 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:51:41.779 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:51:41.959 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:51:41.959 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:51:42.794 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:42.812 INFO:teuthology.orchestra.run.trial016.stdout: Installing : librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:43.141 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:43.199 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:43.199 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:43.199 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:43.199 INFO:teuthology.orchestra.run.trial016.stdout: librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:43.200 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:43.200 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:43.230 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install python3-cephfs-debuginfo 2026-02-17T05:51:43.363 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:51:43.380 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:43.540 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:45 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:43.588 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:43.622 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:43.622 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:43.622 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:43.622 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:43.622 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:43.622 INFO:teuthology.orchestra.run.trial016.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 629 k 2026-02-17T05:51:43.622 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:43.622 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:43.623 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:43.623 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:43.623 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:43.623 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 629 k 2026-02-17T05:51:43.623 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 2.1 M 2026-02-17T05:51:43.623 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:43.647 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:43.647 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:43.647 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:51:43.647 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:43.647 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:43.647 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:51:43.678 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-mon-debuginfo 2026-02-17T05:51:43.772 INFO:teuthology.orchestra.run.trial016.stdout:python3-cephfs-debuginfo-19.2.3-705.g04cf6dd4.e 4.1 MB/s | 629 kB 00:00 2026-02-17T05:51:43.773 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:43.773 INFO:teuthology.orchestra.run.trial016.stdout:Total 4.1 MB/s | 629 kB 00:00 2026-02-17T05:51:43.773 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:43.774 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:43.774 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:43.794 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:43.794 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:43.858 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:43.875 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:51:43.991 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:51:43.991 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:43.991 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:43.991 INFO:teuthology.orchestra.run.trial016.stdout: python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:43.991 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:43.991 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:43.992 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:03:41 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:51:44.018 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install python3-rados-debuginfo 2026-02-17T05:51:44.073 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:51:44.073 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:44.073 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:51:44.073 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:44.073 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:51:44.073 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 102 M 2026-02-17T05:51:44.073 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:44.073 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:51:44.074 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:44.074 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:51:44.074 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:44.074 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 102 M 2026-02-17T05:51:44.074 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 497 M 2026-02-17T05:51:44.074 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:51:44.326 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:46 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:44.409 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:44.409 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:44.410 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:44.410 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:44.410 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:44.410 INFO:teuthology.orchestra.run.trial016.stdout: python3-rados-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 778 k 2026-02-17T05:51:44.410 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:44.411 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:44.411 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:44.411 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:44.411 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:44.411 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 778 k 2026-02-17T05:51:44.411 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 2.8 M 2026-02-17T05:51:44.412 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:44.687 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:51:44.704 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:44.873 INFO:teuthology.orchestra.run.trial016.stdout:python3-rados-debuginfo-19.2.3-705.g04cf6dd4.el 1.6 MB/s | 778 kB 00:00 2026-02-17T05:51:44.874 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:44.875 INFO:teuthology.orchestra.run.trial016.stdout:Total 1.6 MB/s | 778 kB 00:00 2026-02-17T05:51:44.875 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:44.876 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:44.876 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:44.895 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:44.896 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:44.918 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:44.966 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:44.975 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:44.975 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:44.975 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:51:44.976 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mds-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:44.976 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:44.976 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:51:44.983 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:51:45.008 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-mon-debuginfo 2026-02-17T05:51:45.038 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:51:45.096 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:51:45.096 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:45.096 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:45.096 INFO:teuthology.orchestra.run.trial016.stdout: python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:45.096 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:45.096 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:45.122 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install python3-rbd-debuginfo 2026-02-17T05:51:45.320 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:03:48 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 102 M 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 102 M 2026-02-17T05:51:45.403 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 497 M 2026-02-17T05:51:45.404 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:51:45.432 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:47 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:45.515 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:45.516 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:45.516 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:45.516 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:45.516 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:45.516 INFO:teuthology.orchestra.run.trial016.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.1 M 2026-02-17T05:51:45.516 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:45.516 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:45.516 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:45.516 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:45.517 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:45.517 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 1.1 M 2026-02-17T05:51:45.517 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 4.3 M 2026-02-17T05:51:45.517 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:46.069 INFO:teuthology.orchestra.run.trial016.stdout:python3-rbd-debuginfo-19.2.3-705.g04cf6dd4.el9. 2.1 MB/s | 1.1 MB 00:00 2026-02-17T05:51:46.070 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:46.070 INFO:teuthology.orchestra.run.trial016.stdout:Total 2.1 MB/s | 1.1 MB 00:00 2026-02-17T05:51:46.070 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:46.072 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:46.072 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:46.090 INFO:teuthology.orchestra.run.trial164.stdout:ceph-mon-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 50 MB/s | 102 MB 00:02 2026-02-17T05:51:46.091 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:46.091 INFO:teuthology.orchestra.run.trial164.stdout:Total 50 MB/s | 102 MB 00:02 2026-02-17T05:51:46.091 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:51:46.093 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:51:46.093 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:51:46.094 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:46.094 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:46.184 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:46.201 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:51:46.258 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:51:46.316 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:51:46.316 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:46.316 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:46.316 INFO:teuthology.orchestra.run.trial016.stdout: python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:46.317 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:46.317 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:46.344 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install python3-rgw-debuginfo 2026-02-17T05:51:46.411 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:51:46.411 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:51:46.654 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:48 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:46.736 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 339 k 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 339 k 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 1.0 M 2026-02-17T05:51:46.737 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:47.033 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:51:47.051 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:47.144 INFO:teuthology.orchestra.run.trial016.stdout:python3-rgw-debuginfo-19.2.3-705.g04cf6dd4.el9. 834 kB/s | 339 kB 00:00 2026-02-17T05:51:47.145 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:47.145 INFO:teuthology.orchestra.run.trial016.stdout:Total 829 kB/s | 339 kB 00:00 2026-02-17T05:51:47.146 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:47.148 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:47.148 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:47.167 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:47.168 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:47.217 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:47.234 INFO:teuthology.orchestra.run.trial016.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:51:47.289 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:51:47.347 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:51:47.347 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:47.347 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:47.347 INFO:teuthology.orchestra.run.trial016.stdout: python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:47.347 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:47.347 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:47.374 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install rbd-fuse-debuginfo 2026-02-17T05:51:47.683 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:49 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:47.764 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:47.765 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:47.765 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:47.765 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:47.765 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:47.765 INFO:teuthology.orchestra.run.trial016.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 546 k 2026-02-17T05:51:47.765 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:47.765 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:47.765 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:47.765 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:47.765 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:47.766 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 546 k 2026-02-17T05:51:47.766 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 3.0 M 2026-02-17T05:51:47.766 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:47.922 INFO:teuthology.orchestra.run.trial016.stdout:rbd-fuse-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 3.4 MB/s | 546 kB 00:00 2026-02-17T05:51:47.922 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:47.922 INFO:teuthology.orchestra.run.trial016.stdout:Total 3.4 MB/s | 546 kB 00:00 2026-02-17T05:51:47.922 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:47.924 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:47.924 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:47.944 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:47.944 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:47.969 INFO:teuthology.orchestra.run.trial148.stdout:ceph-mon-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 40 MB/s | 102 MB 00:02 2026-02-17T05:51:47.970 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:47.970 INFO:teuthology.orchestra.run.trial148.stdout:Total 40 MB/s | 102 MB 00:02 2026-02-17T05:51:47.971 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:51:47.972 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:51:47.973 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:51:48.009 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:48.026 INFO:teuthology.orchestra.run.trial016.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:48.059 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:48.083 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:48.121 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:48.121 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:48.121 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:51:48.122 INFO:teuthology.orchestra.run.trial128.stdout: ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:48.122 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:48.122 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:51:48.141 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:48.142 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:48.142 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:48.142 INFO:teuthology.orchestra.run.trial016.stdout: rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:48.142 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:48.142 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:48.169 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install rbd-mirror-debuginfo 2026-02-17T05:51:48.175 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install ceph-fuse-debuginfo 2026-02-17T05:51:48.288 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:51:48.288 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:51:48.475 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:50 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:48.484 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:03:48 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 54 M 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:48.558 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 54 M 2026-02-17T05:51:48.559 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 353 M 2026-02-17T05:51:48.559 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:48.566 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:51:48.566 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:48.566 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:51:48.566 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:48.566 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:51:48.567 INFO:teuthology.orchestra.run.trial128.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.8 M 2026-02-17T05:51:48.567 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:48.567 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:51:48.567 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:48.567 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:51:48.567 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:48.567 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 6.8 M 2026-02-17T05:51:48.567 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 36 M 2026-02-17T05:51:48.567 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:51:48.894 INFO:teuthology.orchestra.run.trial128.stdout:ceph-fuse-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 21 MB/s | 6.8 MB 00:00 2026-02-17T05:51:48.895 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:48.895 INFO:teuthology.orchestra.run.trial128.stdout:Total 21 MB/s | 6.8 MB 00:00 2026-02-17T05:51:48.895 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:51:48.897 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:51:48.897 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:51:48.935 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:51:48.935 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:51:49.333 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:51:49.351 INFO:teuthology.orchestra.run.trial128.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:49.432 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:49.490 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:49.491 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:49.491 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:51:49.491 INFO:teuthology.orchestra.run.trial128.stdout: ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:49.491 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:49.491 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:51:49.517 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install librados-devel-debuginfo 2026-02-17T05:51:49.752 INFO:teuthology.orchestra.run.trial016.stdout:rbd-mirror-debuginfo-19.2.3-705.g04cf6dd4.el9.x 45 MB/s | 54 MB 00:01 2026-02-17T05:51:49.753 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:49.754 INFO:teuthology.orchestra.run.trial016.stdout:Total 45 MB/s | 54 MB 00:01 2026-02-17T05:51:49.754 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:49.756 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:49.756 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:49.830 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:03:49 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:51:49.912 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:51:49.913 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:49.913 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:51:49.913 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:49.913 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:51:49.913 INFO:teuthology.orchestra.run.trial128.stdout: librados-devel-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 112 k 2026-02-17T05:51:49.913 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:49.913 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:51:49.913 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:49.913 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:51:49.914 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:49.914 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 112 k 2026-02-17T05:51:49.914 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 493 k 2026-02-17T05:51:49.914 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:51:49.933 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:49.933 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:49.998 INFO:teuthology.orchestra.run.trial128.stdout:librados-devel-debuginfo-19.2.3-705.g04cf6dd4.e 1.3 MB/s | 112 kB 00:00 2026-02-17T05:51:49.999 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:49.999 INFO:teuthology.orchestra.run.trial128.stdout:Total 1.3 MB/s | 112 kB 00:00 2026-02-17T05:51:49.999 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:51:50.001 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:51:50.001 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:51:50.020 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:51:50.020 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:51:50.060 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:51:50.077 INFO:teuthology.orchestra.run.trial128.stdout: Installing : librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:51:50.131 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:51:50.190 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:51:50.190 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:50.190 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:51:50.190 INFO:teuthology.orchestra.run.trial128.stdout: librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:50.190 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:50.190 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:51:50.216 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install libcephfs2-debuginfo 2026-02-17T05:51:50.530 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:03:50 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:51:50.612 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:51:50.612 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:50.612 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:51:50.612 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:50.613 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:51:50.613 INFO:teuthology.orchestra.run.trial128.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.0 M 2026-02-17T05:51:50.613 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:50.613 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:51:50.613 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:50.613 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:51:50.613 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:50.613 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 6.0 M 2026-02-17T05:51:50.613 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 31 M 2026-02-17T05:51:50.613 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:51:50.814 INFO:teuthology.orchestra.run.trial128.stdout:libcephfs2-debuginfo-19.2.3-705.g04cf6dd4.el9.x 30 MB/s | 6.0 MB 00:00 2026-02-17T05:51:50.815 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:50.815 INFO:teuthology.orchestra.run.trial128.stdout:Total 29 MB/s | 6.0 MB 00:00 2026-02-17T05:51:50.815 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:51:50.817 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:51:50.817 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:51:50.853 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:51:50.853 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:51:51.199 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:51:51.217 INFO:teuthology.orchestra.run.trial128.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:51:51.296 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:51:51.353 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:51:51.354 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:51.354 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:51:51.354 INFO:teuthology.orchestra.run.trial128.stdout: libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:51.354 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:51.354 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:51:51.383 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install librados2-debuginfo 2026-02-17T05:51:51.617 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:51:51.634 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:51.691 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:03:51 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout: librados2-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 47 M 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 47 M 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 302 M 2026-02-17T05:51:51.775 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:51:52.058 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:52.116 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:52.116 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:52.116 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:51:52.117 INFO:teuthology.orchestra.run.trial164.stdout: ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:52.117 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:52.117 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:51:52.155 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-osd-debuginfo 2026-02-17T05:51:52.462 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:03:50 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:51:52.545 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:51:52.545 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:52.545 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:51:52.545 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:52.545 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:51:52.545 INFO:teuthology.orchestra.run.trial164.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 298 M 2026-02-17T05:51:52.546 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:52.546 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:51:52.546 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:51:52.546 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:51:52.546 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:51:52.546 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 298 M 2026-02-17T05:51:52.546 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 1.4 G 2026-02-17T05:51:52.546 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:51:52.743 INFO:teuthology.orchestra.run.trial128.stdout:librados2-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 49 MB/s | 47 MB 00:00 2026-02-17T05:51:52.744 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:52.744 INFO:teuthology.orchestra.run.trial128.stdout:Total 49 MB/s | 47 MB 00:00 2026-02-17T05:51:52.744 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:51:52.746 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:51:52.746 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:51:52.903 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:51:52.904 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:51:53.032 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:53.049 INFO:teuthology.orchestra.run.trial016.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:51:53.362 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:51:53.421 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:51:53.421 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:53.421 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:53.421 INFO:teuthology.orchestra.run.trial016.stdout: rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:53.421 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:53.422 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:53.452 DEBUG:teuthology.orchestra.run.trial016:> sudo yum -y install rbd-nbd-debuginfo 2026-02-17T05:51:53.503 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:51:53.521 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:53.767 INFO:teuthology.orchestra.run.trial016.stdout:Last metadata expiration check: 0:03:55 ago on Tue 17 Feb 2026 05:47:58 AM UTC. 2026-02-17T05:51:53.849 INFO:teuthology.orchestra.run.trial016.stdout:Dependencies resolved. 2026-02-17T05:51:53.849 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout: Package Arch Version Repo Size 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout:Installing: 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.4 M 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout:Transaction Summary 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout:================================================================================ 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout:Install 1 Package 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout:Total download size: 1.4 M 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout:Installed size: 6.6 M 2026-02-17T05:51:53.850 INFO:teuthology.orchestra.run.trial016.stdout:Downloading Packages: 2026-02-17T05:51:53.895 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:53.953 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:53.953 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:53.953 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:51:53.954 INFO:teuthology.orchestra.run.trial148.stdout: ceph-mon-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:53.954 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:53.954 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:51:53.990 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-osd-debuginfo 2026-02-17T05:51:54.304 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:03:57 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:51:54.386 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 298 M 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 298 M 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 1.4 G 2026-02-17T05:51:54.387 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:51:54.448 INFO:teuthology.orchestra.run.trial016.stdout:rbd-nbd-debuginfo-19.2.3-705.g04cf6dd4.el9.x86_ 2.3 MB/s | 1.4 MB 00:00 2026-02-17T05:51:54.449 INFO:teuthology.orchestra.run.trial016.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:51:54.449 INFO:teuthology.orchestra.run.trial016.stdout:Total 2.3 MB/s | 1.4 MB 00:00 2026-02-17T05:51:54.450 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction check 2026-02-17T05:51:54.452 INFO:teuthology.orchestra.run.trial016.stdout:Transaction check succeeded. 2026-02-17T05:51:54.452 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction test 2026-02-17T05:51:54.474 INFO:teuthology.orchestra.run.trial016.stdout:Transaction test succeeded. 2026-02-17T05:51:54.474 INFO:teuthology.orchestra.run.trial016.stdout:Running transaction 2026-02-17T05:51:54.578 INFO:teuthology.orchestra.run.trial016.stdout: Preparing : 1/1 2026-02-17T05:51:54.595 INFO:teuthology.orchestra.run.trial016.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:54.654 INFO:teuthology.orchestra.run.trial016.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:54.712 INFO:teuthology.orchestra.run.trial016.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:51:54.713 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:54.713 INFO:teuthology.orchestra.run.trial016.stdout:Installed: 2026-02-17T05:51:54.713 INFO:teuthology.orchestra.run.trial016.stdout: rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:54.713 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:51:54.713 INFO:teuthology.orchestra.run.trial016.stdout:Complete! 2026-02-17T05:51:54.743 DEBUG:teuthology.parallel:result is None 2026-02-17T05:51:55.596 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:51:55.613 INFO:teuthology.orchestra.run.trial128.stdout: Installing : librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:55.888 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:55.946 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:51:55.946 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:55.946 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:51:55.946 INFO:teuthology.orchestra.run.trial128.stdout: librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:51:55.946 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:55.946 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:51:55.977 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install librbd1-debuginfo 2026-02-17T05:51:56.285 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:03:56 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:51:56.368 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:51:56.368 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:56.368 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:51:56.368 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:56.369 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:51:56.369 INFO:teuthology.orchestra.run.trial128.stdout: librbd1-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 54 M 2026-02-17T05:51:56.369 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:56.369 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:51:56.369 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:51:56.369 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:51:56.369 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:51:56.369 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 54 M 2026-02-17T05:51:56.369 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 374 M 2026-02-17T05:51:56.370 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:52:00.306 INFO:teuthology.orchestra.run.trial164.stdout:ceph-osd-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 38 MB/s | 298 MB 00:07 2026-02-17T05:52:00.307 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:00.307 INFO:teuthology.orchestra.run.trial164.stdout:Total 38 MB/s | 298 MB 00:07 2026-02-17T05:52:00.307 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:00.309 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:00.309 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:01.201 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:01.201 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:01.560 INFO:teuthology.orchestra.run.trial128.stdout:librbd1-debuginfo-19.2.3-705.g04cf6dd4.el9.x86_ 10 MB/s | 54 MB 00:05 2026-02-17T05:52:01.560 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:01.560 INFO:teuthology.orchestra.run.trial128.stdout:Total 10 MB/s | 54 MB 00:05 2026-02-17T05:52:01.560 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:52:01.562 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:52:01.562 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:52:01.742 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:52:01.743 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:52:04.466 INFO:teuthology.orchestra.run.trial148.stdout:ceph-osd-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 30 MB/s | 298 MB 00:10 2026-02-17T05:52:04.467 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:04.467 INFO:teuthology.orchestra.run.trial148.stdout:Total 30 MB/s | 298 MB 00:10 2026-02-17T05:52:04.467 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:04.470 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:04.470 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:04.927 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:52:04.946 INFO:teuthology.orchestra.run.trial128.stdout: Installing : librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:05.262 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:05.320 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:05.320 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:05.320 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:52:05.321 INFO:teuthology.orchestra.run.trial128.stdout: librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:05.321 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:05.321 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:52:05.353 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install python3-cephfs-debuginfo 2026-02-17T05:52:05.361 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:05.361 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:05.664 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:04:05 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:52:05.746 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:52:05.747 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:05.747 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:52:05.747 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:05.747 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:52:05.747 INFO:teuthology.orchestra.run.trial128.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 629 k 2026-02-17T05:52:05.747 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:05.747 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:52:05.747 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:05.747 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:52:05.748 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:05.748 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 629 k 2026-02-17T05:52:05.748 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 2.1 M 2026-02-17T05:52:05.748 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:52:05.867 INFO:teuthology.orchestra.run.trial128.stdout:python3-cephfs-debuginfo-19.2.3-705.g04cf6dd4.e 5.1 MB/s | 629 kB 00:00 2026-02-17T05:52:05.868 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:05.868 INFO:teuthology.orchestra.run.trial128.stdout:Total 5.0 MB/s | 629 kB 00:00 2026-02-17T05:52:05.868 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:52:05.870 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:52:05.870 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:52:05.890 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:52:05.890 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:52:05.954 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:52:05.971 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:06.027 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:06.084 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:06.085 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:06.085 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:52:06.085 INFO:teuthology.orchestra.run.trial128.stdout: python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:06.085 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:06.085 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:52:06.111 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install python3-rados-debuginfo 2026-02-17T05:52:06.421 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:04:06 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:52:06.502 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout: python3-rados-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 778 k 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 778 k 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 2.8 M 2026-02-17T05:52:06.503 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:52:06.610 INFO:teuthology.orchestra.run.trial128.stdout:python3-rados-debuginfo-19.2.3-705.g04cf6dd4.el 7.1 MB/s | 778 kB 00:00 2026-02-17T05:52:06.611 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:06.611 INFO:teuthology.orchestra.run.trial128.stdout:Total 7.0 MB/s | 778 kB 00:00 2026-02-17T05:52:06.611 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:52:06.613 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:52:06.613 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:52:06.633 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:52:06.633 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:52:06.705 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:52:06.722 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:52:06.781 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:52:06.839 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:52:06.839 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:06.839 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:52:06.839 INFO:teuthology.orchestra.run.trial128.stdout: python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:06.839 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:06.839 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:52:06.867 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install python3-rbd-debuginfo 2026-02-17T05:52:07.173 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:04:07 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.1 M 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:52:07.255 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:07.256 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 1.1 M 2026-02-17T05:52:07.256 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 4.3 M 2026-02-17T05:52:07.256 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:52:07.385 INFO:teuthology.orchestra.run.trial128.stdout:python3-rbd-debuginfo-19.2.3-705.g04cf6dd4.el9. 8.8 MB/s | 1.1 MB 00:00 2026-02-17T05:52:07.385 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:07.385 INFO:teuthology.orchestra.run.trial128.stdout:Total 8.8 MB/s | 1.1 MB 00:00 2026-02-17T05:52:07.385 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:52:07.387 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:52:07.387 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:52:07.408 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:52:07.408 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:52:07.498 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:52:07.515 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:07.571 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:07.628 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:07.629 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:07.629 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:52:07.629 INFO:teuthology.orchestra.run.trial128.stdout: python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:07.629 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:07.629 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:52:07.654 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install python3-rgw-debuginfo 2026-02-17T05:52:07.959 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:04:07 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 339 k 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:52:08.041 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:08.042 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 339 k 2026-02-17T05:52:08.042 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 1.0 M 2026-02-17T05:52:08.042 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:52:08.149 INFO:teuthology.orchestra.run.trial128.stdout:python3-rgw-debuginfo-19.2.3-705.g04cf6dd4.el9. 3.1 MB/s | 339 kB 00:00 2026-02-17T05:52:08.149 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:08.149 INFO:teuthology.orchestra.run.trial128.stdout:Total 3.1 MB/s | 339 kB 00:00 2026-02-17T05:52:08.150 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:52:08.151 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:52:08.152 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:52:08.171 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:52:08.172 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:52:08.222 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:52:08.239 INFO:teuthology.orchestra.run.trial128.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:08.295 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:08.352 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:08.352 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:08.352 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:52:08.352 INFO:teuthology.orchestra.run.trial128.stdout: python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:08.353 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:08.353 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:52:08.379 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install rbd-fuse-debuginfo 2026-02-17T05:52:08.687 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:04:08 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:52:08.770 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:52:08.770 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:08.770 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:52:08.771 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:08.771 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:52:08.771 INFO:teuthology.orchestra.run.trial128.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 546 k 2026-02-17T05:52:08.771 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:08.771 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:52:08.771 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:08.771 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:52:08.771 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:08.771 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 546 k 2026-02-17T05:52:08.771 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 3.0 M 2026-02-17T05:52:08.772 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:52:08.878 INFO:teuthology.orchestra.run.trial128.stdout:rbd-fuse-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 4.9 MB/s | 546 kB 00:00 2026-02-17T05:52:08.879 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:08.879 INFO:teuthology.orchestra.run.trial128.stdout:Total 4.9 MB/s | 546 kB 00:00 2026-02-17T05:52:08.879 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:52:08.881 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:52:08.881 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:52:08.902 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:52:08.902 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:52:08.965 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:52:08.982 INFO:teuthology.orchestra.run.trial128.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:09.041 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:09.099 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:09.099 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:09.100 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:52:09.100 INFO:teuthology.orchestra.run.trial128.stdout: rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:09.100 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:09.100 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:52:09.124 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install rbd-mirror-debuginfo 2026-02-17T05:52:09.438 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:04:09 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:52:09.519 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:52:09.520 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:09.521 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:52:09.521 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:09.521 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:52:09.521 INFO:teuthology.orchestra.run.trial128.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 54 M 2026-02-17T05:52:09.521 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:09.521 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:52:09.521 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:09.521 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:52:09.521 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:09.522 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 54 M 2026-02-17T05:52:09.522 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 353 M 2026-02-17T05:52:09.522 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:52:10.611 INFO:teuthology.orchestra.run.trial128.stdout:rbd-mirror-debuginfo-19.2.3-705.g04cf6dd4.el9.x 49 MB/s | 54 MB 00:01 2026-02-17T05:52:10.612 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:10.612 INFO:teuthology.orchestra.run.trial128.stdout:Total 49 MB/s | 54 MB 00:01 2026-02-17T05:52:10.612 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:52:10.615 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:52:10.615 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:52:10.791 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:52:10.791 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:52:13.841 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:52:13.859 INFO:teuthology.orchestra.run.trial128.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:14.165 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:14.223 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:14.223 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:14.223 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:52:14.223 INFO:teuthology.orchestra.run.trial128.stdout: rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:14.223 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:14.224 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:52:14.255 DEBUG:teuthology.orchestra.run.trial128:> sudo yum -y install rbd-nbd-debuginfo 2026-02-17T05:52:14.563 INFO:teuthology.orchestra.run.trial128.stdout:Last metadata expiration check: 0:04:14 ago on Tue 17 Feb 2026 05:48:00 AM UTC. 2026-02-17T05:52:14.646 INFO:teuthology.orchestra.run.trial128.stdout:Dependencies resolved. 2026-02-17T05:52:14.647 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:14.647 INFO:teuthology.orchestra.run.trial128.stdout: Package Arch Version Repo Size 2026-02-17T05:52:14.647 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:14.647 INFO:teuthology.orchestra.run.trial128.stdout:Installing: 2026-02-17T05:52:14.647 INFO:teuthology.orchestra.run.trial128.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.4 M 2026-02-17T05:52:14.647 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:14.647 INFO:teuthology.orchestra.run.trial128.stdout:Transaction Summary 2026-02-17T05:52:14.648 INFO:teuthology.orchestra.run.trial128.stdout:================================================================================ 2026-02-17T05:52:14.648 INFO:teuthology.orchestra.run.trial128.stdout:Install 1 Package 2026-02-17T05:52:14.648 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:14.648 INFO:teuthology.orchestra.run.trial128.stdout:Total download size: 1.4 M 2026-02-17T05:52:14.648 INFO:teuthology.orchestra.run.trial128.stdout:Installed size: 6.6 M 2026-02-17T05:52:14.648 INFO:teuthology.orchestra.run.trial128.stdout:Downloading Packages: 2026-02-17T05:52:14.782 INFO:teuthology.orchestra.run.trial128.stdout:rbd-nbd-debuginfo-19.2.3-705.g04cf6dd4.el9.x86_ 10 MB/s | 1.4 MB 00:00 2026-02-17T05:52:14.782 INFO:teuthology.orchestra.run.trial128.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:14.783 INFO:teuthology.orchestra.run.trial128.stdout:Total 10 MB/s | 1.4 MB 00:00 2026-02-17T05:52:14.783 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction check 2026-02-17T05:52:14.784 INFO:teuthology.orchestra.run.trial128.stdout:Transaction check succeeded. 2026-02-17T05:52:14.784 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction test 2026-02-17T05:52:14.805 INFO:teuthology.orchestra.run.trial128.stdout:Transaction test succeeded. 2026-02-17T05:52:14.805 INFO:teuthology.orchestra.run.trial128.stdout:Running transaction 2026-02-17T05:52:14.910 INFO:teuthology.orchestra.run.trial128.stdout: Preparing : 1/1 2026-02-17T05:52:14.928 INFO:teuthology.orchestra.run.trial128.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:14.995 INFO:teuthology.orchestra.run.trial128.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:15.054 INFO:teuthology.orchestra.run.trial128.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:15.055 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:15.055 INFO:teuthology.orchestra.run.trial128.stdout:Installed: 2026-02-17T05:52:15.055 INFO:teuthology.orchestra.run.trial128.stdout: rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:15.055 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:15.055 INFO:teuthology.orchestra.run.trial128.stdout:Complete! 2026-02-17T05:52:15.083 DEBUG:teuthology.parallel:result is None 2026-02-17T05:52:16.218 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:16.236 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:17.304 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:17.362 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:17.362 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:17.362 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:17.362 INFO:teuthology.orchestra.run.trial164.stdout: ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:17.362 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:17.362 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:17.418 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install ceph-fuse-debuginfo 2026-02-17T05:52:17.729 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:15 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:17.810 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.8 M 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 6.8 M 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 36 M 2026-02-17T05:52:17.811 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:18.085 INFO:teuthology.orchestra.run.trial164.stdout:ceph-fuse-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 25 MB/s | 6.8 MB 00:00 2026-02-17T05:52:18.085 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:18.085 INFO:teuthology.orchestra.run.trial164.stdout:Total 25 MB/s | 6.8 MB 00:00 2026-02-17T05:52:18.085 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:18.088 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:18.088 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:18.126 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:18.127 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:18.519 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:18.537 INFO:teuthology.orchestra.run.trial164.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:18.621 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:18.680 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:18.680 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:18.680 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:18.680 INFO:teuthology.orchestra.run.trial164.stdout: ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:18.681 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:18.681 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:18.707 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install librados-devel-debuginfo 2026-02-17T05:52:19.016 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:17 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:19.099 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:19.099 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:19.100 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:19.100 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:19.100 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:19.100 INFO:teuthology.orchestra.run.trial164.stdout: librados-devel-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 112 k 2026-02-17T05:52:19.100 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:19.100 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:19.100 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:19.100 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:19.100 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:19.100 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 112 k 2026-02-17T05:52:19.101 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 493 k 2026-02-17T05:52:19.101 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:19.185 INFO:teuthology.orchestra.run.trial164.stdout:librados-devel-debuginfo-19.2.3-705.g04cf6dd4.e 1.3 MB/s | 112 kB 00:00 2026-02-17T05:52:19.186 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:19.186 INFO:teuthology.orchestra.run.trial164.stdout:Total 1.3 MB/s | 112 kB 00:00 2026-02-17T05:52:19.186 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:19.187 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:19.187 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:19.206 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:19.206 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:19.246 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:19.263 INFO:teuthology.orchestra.run.trial164.stdout: Installing : librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:19.318 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:19.376 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:19.377 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:19.377 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:19.377 INFO:teuthology.orchestra.run.trial164.stdout: librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:19.377 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:19.377 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:19.403 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install libcephfs2-debuginfo 2026-02-17T05:52:19.707 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:17 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.0 M 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 6.0 M 2026-02-17T05:52:19.788 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 31 M 2026-02-17T05:52:19.789 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:19.992 INFO:teuthology.orchestra.run.trial164.stdout:libcephfs2-debuginfo-19.2.3-705.g04cf6dd4.el9.x 29 MB/s | 6.0 MB 00:00 2026-02-17T05:52:19.992 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:19.992 INFO:teuthology.orchestra.run.trial164.stdout:Total 29 MB/s | 6.0 MB 00:00 2026-02-17T05:52:19.992 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:19.994 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:19.994 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:20.030 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:20.030 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:20.352 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:20.370 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:20.381 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:20.398 INFO:teuthology.orchestra.run.trial164.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:20.477 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:20.535 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:20.535 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:20.535 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:20.535 INFO:teuthology.orchestra.run.trial164.stdout: libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:20.535 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:20.535 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:20.560 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install librados2-debuginfo 2026-02-17T05:52:20.870 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:18 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:20.952 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:20.952 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:20.952 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:20.952 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:20.953 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:20.953 INFO:teuthology.orchestra.run.trial164.stdout: librados2-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 47 M 2026-02-17T05:52:20.953 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:20.953 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:20.953 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:20.953 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:20.953 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:20.953 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 47 M 2026-02-17T05:52:20.953 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 302 M 2026-02-17T05:52:20.953 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:21.444 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:21.502 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:21.503 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:21.503 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:21.503 INFO:teuthology.orchestra.run.trial148.stdout: ceph-osd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:21.503 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:21.503 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:21.556 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install ceph-fuse-debuginfo 2026-02-17T05:52:21.868 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:24 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.8 M 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 6.8 M 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 36 M 2026-02-17T05:52:21.951 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:23.009 INFO:teuthology.orchestra.run.trial164.stdout:librados2-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 23 MB/s | 47 MB 00:02 2026-02-17T05:52:23.010 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:23.010 INFO:teuthology.orchestra.run.trial164.stdout:Total 23 MB/s | 47 MB 00:02 2026-02-17T05:52:23.010 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:23.012 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:23.012 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:23.170 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:23.171 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:24.092 INFO:teuthology.orchestra.run.trial148.stdout:ceph-fuse-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 3.2 MB/s | 6.8 MB 00:02 2026-02-17T05:52:24.093 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:24.093 INFO:teuthology.orchestra.run.trial148.stdout:Total 3.2 MB/s | 6.8 MB 00:02 2026-02-17T05:52:24.093 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:24.094 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:24.095 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:24.132 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:24.133 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:24.532 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:24.549 INFO:teuthology.orchestra.run.trial148.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:24.628 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:24.686 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:24.686 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:24.686 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:24.687 INFO:teuthology.orchestra.run.trial148.stdout: ceph-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:24.687 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:24.687 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:24.714 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install librados-devel-debuginfo 2026-02-17T05:52:25.022 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:28 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:25.106 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:25.106 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:25.106 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:25.106 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:25.106 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:25.106 INFO:teuthology.orchestra.run.trial148.stdout: librados-devel-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 112 k 2026-02-17T05:52:25.106 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:25.107 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:25.107 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:25.107 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:25.107 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:25.107 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 112 k 2026-02-17T05:52:25.107 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 493 k 2026-02-17T05:52:25.107 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:25.198 INFO:teuthology.orchestra.run.trial148.stdout:librados-devel-debuginfo-19.2.3-705.g04cf6dd4.e 1.2 MB/s | 112 kB 00:00 2026-02-17T05:52:25.199 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:25.199 INFO:teuthology.orchestra.run.trial148.stdout:Total 1.2 MB/s | 112 kB 00:00 2026-02-17T05:52:25.199 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:25.201 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:25.201 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:25.219 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:25.219 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:25.258 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:25.275 INFO:teuthology.orchestra.run.trial148.stdout: Installing : librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:25.330 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:25.387 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:25.388 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:25.388 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:25.388 INFO:teuthology.orchestra.run.trial148.stdout: librados-devel-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:25.388 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:25.388 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:25.414 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install libcephfs2-debuginfo 2026-02-17T05:52:25.727 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:28 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:25.811 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:25.811 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:25.811 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.0 M 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 6.0 M 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 31 M 2026-02-17T05:52:25.812 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:25.858 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:25.876 INFO:teuthology.orchestra.run.trial164.stdout: Installing : librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:26.013 INFO:teuthology.orchestra.run.trial148.stdout:libcephfs2-debuginfo-19.2.3-705.g04cf6dd4.el9.x 30 MB/s | 6.0 MB 00:00 2026-02-17T05:52:26.013 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:26.013 INFO:teuthology.orchestra.run.trial148.stdout:Total 30 MB/s | 6.0 MB 00:00 2026-02-17T05:52:26.013 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:26.015 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:26.015 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:26.051 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:26.051 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:26.146 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:26.205 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:26.205 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:26.205 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:26.206 INFO:teuthology.orchestra.run.trial164.stdout: librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:26.206 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:26.206 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:26.235 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install librbd1-debuginfo 2026-02-17T05:52:26.398 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:26.416 INFO:teuthology.orchestra.run.trial148.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:26.493 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:26.542 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:24 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:26.550 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:26.550 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:26.550 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:26.551 INFO:teuthology.orchestra.run.trial148.stdout: libcephfs2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:26.551 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:26.551 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:26.577 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install librados2-debuginfo 2026-02-17T05:52:26.623 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:26.624 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout: librbd1-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 54 M 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 54 M 2026-02-17T05:52:26.625 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 374 M 2026-02-17T05:52:26.626 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:26.889 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:29 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:26.973 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:26.973 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:26.973 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:26.974 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:26.974 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:26.974 INFO:teuthology.orchestra.run.trial148.stdout: librados2-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 47 M 2026-02-17T05:52:26.974 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:26.974 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:26.974 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:26.974 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:26.974 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:26.974 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 47 M 2026-02-17T05:52:26.974 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 302 M 2026-02-17T05:52:26.975 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:27.940 INFO:teuthology.orchestra.run.trial164.stdout:librbd1-debuginfo-19.2.3-705.g04cf6dd4.el9.x86_ 41 MB/s | 54 MB 00:01 2026-02-17T05:52:27.941 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:27.941 INFO:teuthology.orchestra.run.trial164.stdout:Total 41 MB/s | 54 MB 00:01 2026-02-17T05:52:27.941 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:27.943 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:27.943 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:28.122 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:28.122 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:28.718 INFO:teuthology.orchestra.run.trial148.stdout:librados2-debuginfo-19.2.3-705.g04cf6dd4.el9.x8 27 MB/s | 47 MB 00:01 2026-02-17T05:52:28.718 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:28.719 INFO:teuthology.orchestra.run.trial148.stdout:Total 27 MB/s | 47 MB 00:01 2026-02-17T05:52:28.719 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:28.721 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:28.721 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:28.878 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:28.878 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:31.246 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:31.264 INFO:teuthology.orchestra.run.trial164.stdout: Installing : librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:31.568 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:31.586 INFO:teuthology.orchestra.run.trial148.stdout: Installing : librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:31.589 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:31.647 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:31.647 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:31.647 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:31.647 INFO:teuthology.orchestra.run.trial164.stdout: librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:31.647 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:31.647 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:31.678 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install python3-cephfs-debuginfo 2026-02-17T05:52:31.851 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:31.909 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_6 1/1 2026-02-17T05:52:31.909 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:31.909 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:31.910 INFO:teuthology.orchestra.run.trial148.stdout: librados2-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:31.910 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:31.910 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:31.940 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install librbd1-debuginfo 2026-02-17T05:52:31.985 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:29 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:32.067 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:32.068 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:32.068 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:32.068 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:32.068 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:32.068 INFO:teuthology.orchestra.run.trial164.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 629 k 2026-02-17T05:52:32.068 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:32.068 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:32.068 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:32.068 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:32.068 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:32.069 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 629 k 2026-02-17T05:52:32.069 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 2.1 M 2026-02-17T05:52:32.069 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:32.174 INFO:teuthology.orchestra.run.trial164.stdout:python3-cephfs-debuginfo-19.2.3-705.g04cf6dd4.e 5.8 MB/s | 629 kB 00:00 2026-02-17T05:52:32.174 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:32.174 INFO:teuthology.orchestra.run.trial164.stdout:Total 5.7 MB/s | 629 kB 00:00 2026-02-17T05:52:32.175 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:32.176 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:32.176 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:32.196 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:32.197 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:32.249 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:35 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:32.261 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:32.279 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:32.333 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:32.333 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:32.333 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:32.334 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:32.334 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:32.334 INFO:teuthology.orchestra.run.trial148.stdout: librbd1-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 54 M 2026-02-17T05:52:32.334 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:32.334 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:32.334 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:32.334 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:32.334 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:32.334 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 54 M 2026-02-17T05:52:32.334 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 374 M 2026-02-17T05:52:32.335 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:32.339 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:32.397 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:32.397 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:32.397 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:32.397 INFO:teuthology.orchestra.run.trial164.stdout: python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:32.397 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:32.398 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:32.426 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install python3-rados-debuginfo 2026-02-17T05:52:32.730 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:30 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout: python3-rados-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 778 k 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:32.812 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 778 k 2026-02-17T05:52:32.813 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 2.8 M 2026-02-17T05:52:32.813 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:33.536 INFO:teuthology.orchestra.run.trial164.stdout:python3-rados-debuginfo-19.2.3-705.g04cf6dd4.el 1.1 MB/s | 778 kB 00:00 2026-02-17T05:52:33.536 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:33.537 INFO:teuthology.orchestra.run.trial164.stdout:Total 1.1 MB/s | 778 kB 00:00 2026-02-17T05:52:33.537 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:33.538 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:33.538 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:33.559 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:33.559 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:33.601 INFO:teuthology.orchestra.run.trial148.stdout:librbd1-debuginfo-19.2.3-705.g04cf6dd4.el9.x86_ 43 MB/s | 54 MB 00:01 2026-02-17T05:52:33.602 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:33.602 INFO:teuthology.orchestra.run.trial148.stdout:Total 43 MB/s | 54 MB 00:01 2026-02-17T05:52:33.602 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:33.604 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:33.604 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:33.630 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:33.647 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:52:33.710 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:52:33.769 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:52:33.770 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:33.770 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:33.770 INFO:teuthology.orchestra.run.trial164.stdout: python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:33.770 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:33.770 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:33.783 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:33.783 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:33.795 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install python3-rbd-debuginfo 2026-02-17T05:52:34.103 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:32 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:34.184 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:34.185 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:34.185 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:34.185 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:34.185 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:34.185 INFO:teuthology.orchestra.run.trial164.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.1 M 2026-02-17T05:52:34.185 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:34.186 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:34.186 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:34.186 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:34.186 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:34.186 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 1.1 M 2026-02-17T05:52:34.186 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 4.3 M 2026-02-17T05:52:34.186 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:34.862 INFO:teuthology.orchestra.run.trial164.stdout:python3-rbd-debuginfo-19.2.3-705.g04cf6dd4.el9. 1.7 MB/s | 1.1 MB 00:00 2026-02-17T05:52:34.863 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:34.863 INFO:teuthology.orchestra.run.trial164.stdout:Total 1.7 MB/s | 1.1 MB 00:00 2026-02-17T05:52:34.863 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:34.865 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:34.865 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:34.887 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:34.887 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:34.978 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:34.995 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:35.054 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:35.112 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:35.112 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:35.112 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:35.112 INFO:teuthology.orchestra.run.trial164.stdout: python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:35.112 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:35.112 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:35.138 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install python3-rgw-debuginfo 2026-02-17T05:52:35.450 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:33 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:35.532 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:35.532 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:35.532 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:35.532 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:35.532 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:35.532 INFO:teuthology.orchestra.run.trial164.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 339 k 2026-02-17T05:52:35.532 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:35.532 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:35.532 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:35.533 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:35.533 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:35.533 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 339 k 2026-02-17T05:52:35.533 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 1.0 M 2026-02-17T05:52:35.533 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:36.104 INFO:teuthology.orchestra.run.trial164.stdout:python3-rgw-debuginfo-19.2.3-705.g04cf6dd4.el9. 593 kB/s | 339 kB 00:00 2026-02-17T05:52:36.104 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:36.104 INFO:teuthology.orchestra.run.trial164.stdout:Total 592 kB/s | 339 kB 00:00 2026-02-17T05:52:36.104 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:36.106 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:36.106 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:36.125 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:36.126 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:36.175 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:36.193 INFO:teuthology.orchestra.run.trial164.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:36.248 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:36.306 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:36.306 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:36.306 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:36.307 INFO:teuthology.orchestra.run.trial164.stdout: python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:36.307 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:36.307 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:36.333 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install rbd-fuse-debuginfo 2026-02-17T05:52:36.646 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:34 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:36.729 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:36.729 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:36.729 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:36.729 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:36.730 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:36.730 INFO:teuthology.orchestra.run.trial164.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 546 k 2026-02-17T05:52:36.730 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:36.730 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:36.730 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:36.730 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:36.730 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:36.730 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 546 k 2026-02-17T05:52:36.730 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 3.0 M 2026-02-17T05:52:36.730 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:36.920 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:36.939 INFO:teuthology.orchestra.run.trial148.stdout: Installing : librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:37.110 INFO:teuthology.orchestra.run.trial164.stdout:rbd-fuse-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 1.4 MB/s | 546 kB 00:00 2026-02-17T05:52:37.111 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:37.111 INFO:teuthology.orchestra.run.trial164.stdout:Total 1.4 MB/s | 546 kB 00:00 2026-02-17T05:52:37.111 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:37.113 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:37.113 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:37.132 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:37.133 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:37.196 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:37.213 INFO:teuthology.orchestra.run.trial164.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:37.269 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:37.279 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:37.327 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:37.327 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:37.327 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:37.327 INFO:teuthology.orchestra.run.trial164.stdout: rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:37.327 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:37.328 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:37.337 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:37.338 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:37.338 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:37.338 INFO:teuthology.orchestra.run.trial148.stdout: librbd1-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:37.338 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:37.338 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:37.354 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install rbd-mirror-debuginfo 2026-02-17T05:52:37.370 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install python3-cephfs-debuginfo 2026-02-17T05:52:37.666 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:35 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:37.679 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:40 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:37.749 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:37.749 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:37.749 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:37.749 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:37.749 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:37.749 INFO:teuthology.orchestra.run.trial164.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 54 M 2026-02-17T05:52:37.749 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:37.749 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:37.750 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:37.750 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:37.750 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:37.750 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 54 M 2026-02-17T05:52:37.750 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 353 M 2026-02-17T05:52:37.750 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:37.762 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:37.762 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:37.762 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:37.763 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:37.763 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:37.763 INFO:teuthology.orchestra.run.trial148.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 629 k 2026-02-17T05:52:37.763 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:37.763 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:37.763 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:37.763 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:37.763 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:37.763 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 629 k 2026-02-17T05:52:37.763 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 2.1 M 2026-02-17T05:52:37.764 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:37.869 INFO:teuthology.orchestra.run.trial148.stdout:python3-cephfs-debuginfo-19.2.3-705.g04cf6dd4.e 5.8 MB/s | 629 kB 00:00 2026-02-17T05:52:37.870 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:37.870 INFO:teuthology.orchestra.run.trial148.stdout:Total 5.7 MB/s | 629 kB 00:00 2026-02-17T05:52:37.870 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:37.872 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:37.872 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:37.891 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:37.891 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:37.954 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:37.971 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:38.028 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:38.087 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9. 1/1 2026-02-17T05:52:38.088 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:38.088 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:38.088 INFO:teuthology.orchestra.run.trial148.stdout: python3-cephfs-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:38.088 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:38.088 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:38.116 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install python3-rados-debuginfo 2026-02-17T05:52:38.424 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:41 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:38.508 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:38.508 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:38.508 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:38.508 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:38.508 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:38.509 INFO:teuthology.orchestra.run.trial148.stdout: python3-rados-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 778 k 2026-02-17T05:52:38.509 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:38.509 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:38.509 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:38.509 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:38.509 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:38.509 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 778 k 2026-02-17T05:52:38.509 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 2.8 M 2026-02-17T05:52:38.509 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:38.956 INFO:teuthology.orchestra.run.trial164.stdout:rbd-mirror-debuginfo-19.2.3-705.g04cf6dd4.el9.x 44 MB/s | 54 MB 00:01 2026-02-17T05:52:38.956 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:38.957 INFO:teuthology.orchestra.run.trial164.stdout:Total 44 MB/s | 54 MB 00:01 2026-02-17T05:52:38.957 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:38.959 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:38.959 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:38.966 INFO:teuthology.orchestra.run.trial148.stdout:python3-rados-debuginfo-19.2.3-705.g04cf6dd4.el 1.7 MB/s | 778 kB 00:00 2026-02-17T05:52:38.967 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:38.967 INFO:teuthology.orchestra.run.trial148.stdout:Total 1.7 MB/s | 778 kB 00:00 2026-02-17T05:52:38.967 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:38.969 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:38.969 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:38.989 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:38.989 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:39.060 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:39.077 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:52:39.132 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:52:39.135 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:39.136 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:39.190 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x 1/1 2026-02-17T05:52:39.190 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:39.190 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:39.191 INFO:teuthology.orchestra.run.trial148.stdout: python3-rados-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:39.191 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:39.191 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:39.217 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install python3-rbd-debuginfo 2026-02-17T05:52:39.527 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:42 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:39.611 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:39.612 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:39.612 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:39.612 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:39.612 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:39.612 INFO:teuthology.orchestra.run.trial148.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.1 M 2026-02-17T05:52:39.612 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:39.612 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:39.613 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:39.613 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:39.613 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:39.613 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 1.1 M 2026-02-17T05:52:39.613 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 4.3 M 2026-02-17T05:52:39.613 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:39.899 INFO:teuthology.orchestra.run.trial148.stdout:python3-rbd-debuginfo-19.2.3-705.g04cf6dd4.el9. 4.0 MB/s | 1.1 MB 00:00 2026-02-17T05:52:39.900 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:39.901 INFO:teuthology.orchestra.run.trial148.stdout:Total 3.9 MB/s | 1.1 MB 00:00 2026-02-17T05:52:39.901 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:39.903 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:39.903 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:39.924 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:39.924 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:40.018 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:40.035 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:40.090 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:40.149 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:40.149 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:40.149 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:40.149 INFO:teuthology.orchestra.run.trial148.stdout: python3-rbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:40.150 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:40.150 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:40.177 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install python3-rgw-debuginfo 2026-02-17T05:52:40.490 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:43 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:40.572 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:40.573 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:40.573 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:40.573 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:40.573 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:40.573 INFO:teuthology.orchestra.run.trial148.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 339 k 2026-02-17T05:52:40.573 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:40.573 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:40.573 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:40.573 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:40.573 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:40.574 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 339 k 2026-02-17T05:52:40.574 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 1.0 M 2026-02-17T05:52:40.574 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:40.671 INFO:teuthology.orchestra.run.trial148.stdout:python3-rgw-debuginfo-19.2.3-705.g04cf6dd4.el9. 3.4 MB/s | 339 kB 00:00 2026-02-17T05:52:40.671 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:40.671 INFO:teuthology.orchestra.run.trial148.stdout:Total 3.4 MB/s | 339 kB 00:00 2026-02-17T05:52:40.672 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:40.673 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:40.673 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:40.691 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:40.691 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:40.740 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:40.757 INFO:teuthology.orchestra.run.trial148.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:40.818 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:40.877 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86 1/1 2026-02-17T05:52:40.878 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:40.878 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:40.878 INFO:teuthology.orchestra.run.trial148.stdout: python3-rgw-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:40.878 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:40.878 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:40.908 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install rbd-fuse-debuginfo 2026-02-17T05:52:41.218 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:44 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:41.302 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:41.303 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:41.303 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:41.303 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:41.303 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:41.303 INFO:teuthology.orchestra.run.trial148.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 546 k 2026-02-17T05:52:41.303 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:41.304 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:41.304 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:41.304 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:41.304 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:41.304 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 546 k 2026-02-17T05:52:41.304 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 3.0 M 2026-02-17T05:52:41.304 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:41.446 INFO:teuthology.orchestra.run.trial148.stdout:rbd-fuse-debuginfo-19.2.3-705.g04cf6dd4.el9.x86 3.7 MB/s | 546 kB 00:00 2026-02-17T05:52:41.447 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:41.447 INFO:teuthology.orchestra.run.trial148.stdout:Total 3.7 MB/s | 546 kB 00:00 2026-02-17T05:52:41.447 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:41.449 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:41.449 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:41.469 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:41.470 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:41.535 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:41.552 INFO:teuthology.orchestra.run.trial148.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:41.609 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:41.667 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:41.667 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:41.667 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:41.668 INFO:teuthology.orchestra.run.trial148.stdout: rbd-fuse-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:41.668 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:41.668 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:41.694 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install rbd-mirror-debuginfo 2026-02-17T05:52:42.007 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:45 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:42.091 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:42.091 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:42.091 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:42.092 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:42.092 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:42.092 INFO:teuthology.orchestra.run.trial148.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 54 M 2026-02-17T05:52:42.092 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:42.092 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:42.092 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:42.092 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:42.092 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:42.092 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 54 M 2026-02-17T05:52:42.092 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 353 M 2026-02-17T05:52:42.093 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:42.180 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:42.198 INFO:teuthology.orchestra.run.trial164.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:42.513 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:42.572 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:42.572 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:42.572 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:42.572 INFO:teuthology.orchestra.run.trial164.stdout: rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:42.573 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:42.573 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:42.606 DEBUG:teuthology.orchestra.run.trial164:> sudo yum -y install rbd-nbd-debuginfo 2026-02-17T05:52:42.925 INFO:teuthology.orchestra.run.trial164.stdout:Last metadata expiration check: 0:04:40 ago on Tue 17 Feb 2026 05:48:02 AM UTC. 2026-02-17T05:52:43.008 INFO:teuthology.orchestra.run.trial164.stdout:Dependencies resolved. 2026-02-17T05:52:43.008 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:43.008 INFO:teuthology.orchestra.run.trial164.stdout: Package Arch Version Repo Size 2026-02-17T05:52:43.008 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:43.008 INFO:teuthology.orchestra.run.trial164.stdout:Installing: 2026-02-17T05:52:43.008 INFO:teuthology.orchestra.run.trial164.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.4 M 2026-02-17T05:52:43.008 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:43.008 INFO:teuthology.orchestra.run.trial164.stdout:Transaction Summary 2026-02-17T05:52:43.009 INFO:teuthology.orchestra.run.trial164.stdout:================================================================================ 2026-02-17T05:52:43.009 INFO:teuthology.orchestra.run.trial164.stdout:Install 1 Package 2026-02-17T05:52:43.009 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:43.009 INFO:teuthology.orchestra.run.trial164.stdout:Total download size: 1.4 M 2026-02-17T05:52:43.009 INFO:teuthology.orchestra.run.trial164.stdout:Installed size: 6.6 M 2026-02-17T05:52:43.009 INFO:teuthology.orchestra.run.trial164.stdout:Downloading Packages: 2026-02-17T05:52:43.609 INFO:teuthology.orchestra.run.trial164.stdout:rbd-nbd-debuginfo-19.2.3-705.g04cf6dd4.el9.x86_ 2.3 MB/s | 1.4 MB 00:00 2026-02-17T05:52:43.610 INFO:teuthology.orchestra.run.trial164.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:43.610 INFO:teuthology.orchestra.run.trial164.stdout:Total 2.2 MB/s | 1.4 MB 00:00 2026-02-17T05:52:43.610 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction check 2026-02-17T05:52:43.611 INFO:teuthology.orchestra.run.trial164.stdout:Transaction check succeeded. 2026-02-17T05:52:43.611 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction test 2026-02-17T05:52:43.633 INFO:teuthology.orchestra.run.trial164.stdout:Transaction test succeeded. 2026-02-17T05:52:43.633 INFO:teuthology.orchestra.run.trial164.stdout:Running transaction 2026-02-17T05:52:43.739 INFO:teuthology.orchestra.run.trial164.stdout: Preparing : 1/1 2026-02-17T05:52:43.756 INFO:teuthology.orchestra.run.trial164.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:43.817 INFO:teuthology.orchestra.run.trial164.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:43.875 INFO:teuthology.orchestra.run.trial164.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:43.875 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:43.875 INFO:teuthology.orchestra.run.trial164.stdout:Installed: 2026-02-17T05:52:43.875 INFO:teuthology.orchestra.run.trial164.stdout: rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:43.875 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:43.876 INFO:teuthology.orchestra.run.trial164.stdout:Complete! 2026-02-17T05:52:43.906 DEBUG:teuthology.parallel:result is None 2026-02-17T05:52:49.585 INFO:teuthology.orchestra.run.trial148.stdout:rbd-mirror-debuginfo-19.2.3-705.g04cf6dd4.el9.x 7.1 MB/s | 54 MB 00:07 2026-02-17T05:52:49.585 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:49.586 INFO:teuthology.orchestra.run.trial148.stdout:Total 7.1 MB/s | 54 MB 00:07 2026-02-17T05:52:49.586 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:49.587 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:49.587 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:49.764 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:49.764 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:52.817 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:52.835 INFO:teuthology.orchestra.run.trial148.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:53.130 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:53.189 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_ 1/1 2026-02-17T05:52:53.189 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:53.189 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:53.189 INFO:teuthology.orchestra.run.trial148.stdout: rbd-mirror-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:53.189 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:53.190 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:53.220 DEBUG:teuthology.orchestra.run.trial148:> sudo yum -y install rbd-nbd-debuginfo 2026-02-17T05:52:53.526 INFO:teuthology.orchestra.run.trial148.stdout:Last metadata expiration check: 0:04:56 ago on Tue 17 Feb 2026 05:47:57 AM UTC. 2026-02-17T05:52:53.610 INFO:teuthology.orchestra.run.trial148.stdout:Dependencies resolved. 2026-02-17T05:52:53.610 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:53.610 INFO:teuthology.orchestra.run.trial148.stdout: Package Arch Version Repo Size 2026-02-17T05:52:53.610 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:53.610 INFO:teuthology.orchestra.run.trial148.stdout:Installing: 2026-02-17T05:52:53.611 INFO:teuthology.orchestra.run.trial148.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.4 M 2026-02-17T05:52:53.611 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:53.611 INFO:teuthology.orchestra.run.trial148.stdout:Transaction Summary 2026-02-17T05:52:53.611 INFO:teuthology.orchestra.run.trial148.stdout:================================================================================ 2026-02-17T05:52:53.611 INFO:teuthology.orchestra.run.trial148.stdout:Install 1 Package 2026-02-17T05:52:53.611 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:53.611 INFO:teuthology.orchestra.run.trial148.stdout:Total download size: 1.4 M 2026-02-17T05:52:53.611 INFO:teuthology.orchestra.run.trial148.stdout:Installed size: 6.6 M 2026-02-17T05:52:53.611 INFO:teuthology.orchestra.run.trial148.stdout:Downloading Packages: 2026-02-17T05:52:53.741 INFO:teuthology.orchestra.run.trial148.stdout:rbd-nbd-debuginfo-19.2.3-705.g04cf6dd4.el9.x86_ 10 MB/s | 1.4 MB 00:00 2026-02-17T05:52:53.741 INFO:teuthology.orchestra.run.trial148.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:52:53.741 INFO:teuthology.orchestra.run.trial148.stdout:Total 10 MB/s | 1.4 MB 00:00 2026-02-17T05:52:53.741 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction check 2026-02-17T05:52:53.743 INFO:teuthology.orchestra.run.trial148.stdout:Transaction check succeeded. 2026-02-17T05:52:53.743 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction test 2026-02-17T05:52:53.764 INFO:teuthology.orchestra.run.trial148.stdout:Transaction test succeeded. 2026-02-17T05:52:53.764 INFO:teuthology.orchestra.run.trial148.stdout:Running transaction 2026-02-17T05:52:53.867 INFO:teuthology.orchestra.run.trial148.stdout: Preparing : 1/1 2026-02-17T05:52:53.884 INFO:teuthology.orchestra.run.trial148.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:53.943 INFO:teuthology.orchestra.run.trial148.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:54.001 INFO:teuthology.orchestra.run.trial148.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:52:54.001 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:54.001 INFO:teuthology.orchestra.run.trial148.stdout:Installed: 2026-02-17T05:52:54.001 INFO:teuthology.orchestra.run.trial148.stdout: rbd-nbd-debuginfo-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:52:54.001 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:54.001 INFO:teuthology.orchestra.run.trial148.stdout:Complete! 2026-02-17T05:52:54.028 DEBUG:teuthology.parallel:result is None 2026-02-17T05:52:54.029 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:52:54.157 DEBUG:teuthology.orchestra.run.trial016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T05:52:54.178 INFO:teuthology.orchestra.run.trial016.stdout:19.2.3-705.g04cf6dd4.el9 2026-02-17T05:52:54.179 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-705.g04cf6dd4.el9 2026-02-17T05:52:54.179 INFO:teuthology.task.install:The correct ceph version 19.2.3-705.g04cf6dd4 is installed. 2026-02-17T05:52:54.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:52:54.302 DEBUG:teuthology.orchestra.run.trial128:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T05:52:54.323 INFO:teuthology.orchestra.run.trial128.stdout:19.2.3-705.g04cf6dd4.el9 2026-02-17T05:52:54.324 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-705.g04cf6dd4.el9 2026-02-17T05:52:54.324 INFO:teuthology.task.install:The correct ceph version 19.2.3-705.g04cf6dd4 is installed. 2026-02-17T05:52:54.326 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:52:54.469 DEBUG:teuthology.orchestra.run.trial148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T05:52:54.490 INFO:teuthology.orchestra.run.trial148.stdout:19.2.3-705.g04cf6dd4.el9 2026-02-17T05:52:54.490 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-705.g04cf6dd4.el9 2026-02-17T05:52:54.490 INFO:teuthology.task.install:The correct ceph version 19.2.3-705.g04cf6dd4 is installed. 2026-02-17T05:52:54.492 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:52:54.617 DEBUG:teuthology.orchestra.run.trial164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T05:52:54.637 INFO:teuthology.orchestra.run.trial164.stdout:19.2.3-705.g04cf6dd4.el9 2026-02-17T05:52:54.637 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-705.g04cf6dd4.el9 2026-02-17T05:52:54.637 INFO:teuthology.task.install:The correct ceph version 19.2.3-705.g04cf6dd4 is installed. 2026-02-17T05:52:54.639 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-17T05:52:54.639 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:52:54.639 DEBUG:teuthology.orchestra.run.trial016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T05:52:54.665 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:52:54.665 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T05:52:54.692 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:52:54.692 DEBUG:teuthology.orchestra.run.trial148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T05:52:54.718 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:52:54.719 DEBUG:teuthology.orchestra.run.trial164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T05:52:54.744 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-17T05:52:54.745 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:52:54.745 DEBUG:teuthology.orchestra.run.trial016:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T05:52:54.770 DEBUG:teuthology.orchestra.run.trial016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T05:52:54.834 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:52:54.834 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T05:52:54.860 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T05:52:54.923 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:52:54.923 DEBUG:teuthology.orchestra.run.trial148:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T05:52:54.949 DEBUG:teuthology.orchestra.run.trial148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T05:52:55.013 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:52:55.014 DEBUG:teuthology.orchestra.run.trial164:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T05:52:55.040 DEBUG:teuthology.orchestra.run.trial164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T05:52:55.104 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-17T05:52:55.104 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:52:55.104 DEBUG:teuthology.orchestra.run.trial016:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T05:52:55.129 DEBUG:teuthology.orchestra.run.trial016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T05:52:55.191 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:52:55.191 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T05:52:55.216 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T05:52:55.279 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:52:55.279 DEBUG:teuthology.orchestra.run.trial148:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T05:52:55.304 DEBUG:teuthology.orchestra.run.trial148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T05:52:55.368 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:52:55.368 DEBUG:teuthology.orchestra.run.trial164:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T05:52:55.393 DEBUG:teuthology.orchestra.run.trial164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T05:52:55.458 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-17T05:52:55.458 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:52:55.458 DEBUG:teuthology.orchestra.run.trial016:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T05:52:55.484 DEBUG:teuthology.orchestra.run.trial016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T05:52:55.548 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:52:55.548 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T05:52:55.574 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T05:52:55.637 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:52:55.637 DEBUG:teuthology.orchestra.run.trial148:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T05:52:55.663 DEBUG:teuthology.orchestra.run.trial148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T05:52:55.727 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:52:55.727 DEBUG:teuthology.orchestra.run.trial164:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T05:52:55.752 DEBUG:teuthology.orchestra.run.trial164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T05:52:55.818 INFO:teuthology.run_tasks:Running task ceph... 2026-02-17T05:52:55.930 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-17T05:52:55.930 DEBUG:teuthology.orchestra.run.trial016:> sudo chmod 777 /var/log/ceph 2026-02-17T05:52:55.932 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod 777 /var/log/ceph 2026-02-17T05:52:55.934 DEBUG:teuthology.orchestra.run.trial148:> sudo chmod 777 /var/log/ceph 2026-02-17T05:52:55.936 DEBUG:teuthology.orchestra.run.trial164:> sudo chmod 777 /var/log/ceph 2026-02-17T05:52:55.960 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-17T05:52:55.960 DEBUG:teuthology.orchestra.run.trial016:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-17T05:52:55.998 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-17T05:52:56.000 DEBUG:teuthology.orchestra.run.trial148:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-17T05:52:56.002 DEBUG:teuthology.orchestra.run.trial164:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-17T05:52:56.026 INFO:tasks.ceph:Creating extra log directories... 2026-02-17T05:52:56.026 DEBUG:teuthology.orchestra.run.trial016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-17T05:52:56.063 DEBUG:teuthology.orchestra.run.trial128:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-17T05:52:56.064 DEBUG:teuthology.orchestra.run.trial148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-17T05:52:56.066 DEBUG:teuthology.orchestra.run.trial164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-17T05:52:56.098 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-17T05:52:56.098 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request', 'OSD bench result'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-17T05:52:56.099 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065', 'branch': 'wip-bharath19-testing-2026-02-13-0903-squid', 'description': 'rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'flavor': 'default', 'job_id': '51065', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath19-testing-2026-02-13-0903-squid'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request', 'OSD bench result'], 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'stop-daemons-timeout': 1200, 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath19-testing-2026-02-13-0903-squid', 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'}}, 'owner': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 3603, 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath19-testing-2026-02-13-0903-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'targets': {'trial016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCOjwa/S7SIgHS0J6Rc+z8xdczVwmSiw674yoGf3MnN2u2eE72QcKYWAMwnIBoXYZbuW+iU9UR4UPtxbDHHms8=', 'trial128.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAv6HpIRX32j0pnlKdV17Nlu4rkKitJRYQCcVQiuw7qymgmi010/TejjZJfvSg3e31GxB3swfyVvGG0hUjWXckA=', 'trial148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1B6qsZT1vfivcc0xwAdKN6dsBIR3T4QrW36iADtgSj9tO05NUIGNn4g71dyyjoHBO9o5NhHeUtWlHJJ1TstW4=', 'trial164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB985FXmImc249j7rRFpaYT3Ole+BKLL89gUND7dwdAO3d5hgUn5IviwpEAZldK4XwzgGHL3KtnmH0aakzKaauQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'workunit': {'clients': {'client.0': ['rados/test.sh']}, 'env': {'ALLOW_TIMEOUTS': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["-- Don't run this test on tiny clusters\nif py_len(yaml.roles) < 4 then reject() end\n"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-17_04:03:08', 'tube': 'trial', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682'} 2026-02-17T05:52:56.099 DEBUG:teuthology.orchestra.run.trial016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-17T05:52:56.126 DEBUG:teuthology.orchestra.run.trial128:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-17T05:52:56.128 DEBUG:teuthology.orchestra.run.trial148:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-17T05:52:56.129 DEBUG:teuthology.orchestra.run.trial164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-17T05:52:56.145 DEBUG:teuthology.orchestra.run.trial016:> sudo install -d -m0777 -- /var/run/ceph 2026-02-17T05:52:56.183 DEBUG:teuthology.orchestra.run.trial128:> sudo install -d -m0777 -- /var/run/ceph 2026-02-17T05:52:56.186 DEBUG:teuthology.orchestra.run.trial148:> sudo install -d -m0777 -- /var/run/ceph 2026-02-17T05:52:56.188 DEBUG:teuthology.orchestra.run.trial164:> sudo install -d -m0777 -- /var/run/ceph 2026-02-17T05:52:56.212 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:52:56.213 DEBUG:teuthology.orchestra.run.trial016:> dd if=/scratch_devs of=/dev/stdout 2026-02-17T05:52:56.261 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-17T05:52:56.262 DEBUG:teuthology.orchestra.run.trial016:> stat /dev/vg_nvme/lv_1 2026-02-17T05:52:56.318 INFO:teuthology.orchestra.run.trial016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-17T05:52:56.318 INFO:teuthology.orchestra.run.trial016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:56.319 INFO:teuthology.orchestra.run.trial016.stdout:Device: 6h/6d Inode: 1315 Links: 1 2026-02-17T05:52:56.319 INFO:teuthology.orchestra.run.trial016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:56.319 INFO:teuthology.orchestra.run.trial016.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:56.319 INFO:teuthology.orchestra.run.trial016.stdout:Access: 2026-02-17 05:48:37.455660212 +0000 2026-02-17T05:52:56.319 INFO:teuthology.orchestra.run.trial016.stdout:Modify: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.319 INFO:teuthology.orchestra.run.trial016.stdout:Change: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.319 INFO:teuthology.orchestra.run.trial016.stdout: Birth: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.319 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-17T05:52:56.380 INFO:teuthology.orchestra.run.trial016.stderr:1+0 records in 2026-02-17T05:52:56.380 INFO:teuthology.orchestra.run.trial016.stderr:1+0 records out 2026-02-17T05:52:56.380 INFO:teuthology.orchestra.run.trial016.stderr:512 bytes copied, 0.000149197 s, 3.4 MB/s 2026-02-17T05:52:56.381 DEBUG:teuthology.orchestra.run.trial016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-17T05:52:56.437 DEBUG:teuthology.orchestra.run.trial016:> stat /dev/vg_nvme/lv_2 2026-02-17T05:52:56.494 INFO:teuthology.orchestra.run.trial016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-17T05:52:56.495 INFO:teuthology.orchestra.run.trial016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:56.495 INFO:teuthology.orchestra.run.trial016.stdout:Device: 6h/6d Inode: 1304 Links: 1 2026-02-17T05:52:56.495 INFO:teuthology.orchestra.run.trial016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:56.495 INFO:teuthology.orchestra.run.trial016.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:56.495 INFO:teuthology.orchestra.run.trial016.stdout:Access: 2026-02-17 05:48:37.455660212 +0000 2026-02-17T05:52:56.495 INFO:teuthology.orchestra.run.trial016.stdout:Modify: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.495 INFO:teuthology.orchestra.run.trial016.stdout:Change: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.495 INFO:teuthology.orchestra.run.trial016.stdout: Birth: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.496 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-17T05:52:56.559 INFO:teuthology.orchestra.run.trial016.stderr:1+0 records in 2026-02-17T05:52:56.559 INFO:teuthology.orchestra.run.trial016.stderr:1+0 records out 2026-02-17T05:52:56.559 INFO:teuthology.orchestra.run.trial016.stderr:512 bytes copied, 0.0001458 s, 3.5 MB/s 2026-02-17T05:52:56.560 DEBUG:teuthology.orchestra.run.trial016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-17T05:52:56.618 DEBUG:teuthology.orchestra.run.trial016:> stat /dev/vg_nvme/lv_3 2026-02-17T05:52:56.677 INFO:teuthology.orchestra.run.trial016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-17T05:52:56.677 INFO:teuthology.orchestra.run.trial016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:56.677 INFO:teuthology.orchestra.run.trial016.stdout:Device: 6h/6d Inode: 1293 Links: 1 2026-02-17T05:52:56.677 INFO:teuthology.orchestra.run.trial016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:56.677 INFO:teuthology.orchestra.run.trial016.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:56.677 INFO:teuthology.orchestra.run.trial016.stdout:Access: 2026-02-17 05:48:37.455660212 +0000 2026-02-17T05:52:56.677 INFO:teuthology.orchestra.run.trial016.stdout:Modify: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.678 INFO:teuthology.orchestra.run.trial016.stdout:Change: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.678 INFO:teuthology.orchestra.run.trial016.stdout: Birth: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.678 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial016.stderr:1+0 records in 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial016.stderr:1+0 records out 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial016.stderr:512 bytes copied, 0.000145249 s, 3.5 MB/s 2026-02-17T05:52:56.739 DEBUG:teuthology.orchestra.run.trial016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-17T05:52:56.797 DEBUG:teuthology.orchestra.run.trial016:> stat /dev/vg_nvme/lv_4 2026-02-17T05:52:56.852 INFO:teuthology.orchestra.run.trial016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-17T05:52:56.852 INFO:teuthology.orchestra.run.trial016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:56.852 INFO:teuthology.orchestra.run.trial016.stdout:Device: 6h/6d Inode: 1292 Links: 1 2026-02-17T05:52:56.852 INFO:teuthology.orchestra.run.trial016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:56.852 INFO:teuthology.orchestra.run.trial016.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:56.852 INFO:teuthology.orchestra.run.trial016.stdout:Access: 2026-02-17 05:48:37.456660194 +0000 2026-02-17T05:52:56.852 INFO:teuthology.orchestra.run.trial016.stdout:Modify: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.852 INFO:teuthology.orchestra.run.trial016.stdout:Change: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.852 INFO:teuthology.orchestra.run.trial016.stdout: Birth: 2026-02-17 05:48:04.403230805 +0000 2026-02-17T05:52:56.853 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-17T05:52:56.914 INFO:teuthology.orchestra.run.trial016.stderr:1+0 records in 2026-02-17T05:52:56.914 INFO:teuthology.orchestra.run.trial016.stderr:1+0 records out 2026-02-17T05:52:56.914 INFO:teuthology.orchestra.run.trial016.stderr:512 bytes copied, 0.000184371 s, 2.8 MB/s 2026-02-17T05:52:56.915 DEBUG:teuthology.orchestra.run.trial016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-17T05:52:56.976 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-17T05:52:56.976 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:52:56.976 DEBUG:teuthology.orchestra.run.trial128:> dd if=/scratch_devs of=/dev/stdout 2026-02-17T05:52:56.995 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-17T05:52:56.995 DEBUG:teuthology.orchestra.run.trial128:> stat /dev/vg_nvme/lv_1 2026-02-17T05:52:57.052 INFO:teuthology.orchestra.run.trial128.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-17T05:52:57.053 INFO:teuthology.orchestra.run.trial128.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:57.053 INFO:teuthology.orchestra.run.trial128.stdout:Device: 6h/6d Inode: 1317 Links: 1 2026-02-17T05:52:57.053 INFO:teuthology.orchestra.run.trial128.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:57.053 INFO:teuthology.orchestra.run.trial128.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:57.053 INFO:teuthology.orchestra.run.trial128.stdout:Access: 2026-02-17 05:48:41.314054771 +0000 2026-02-17T05:52:57.053 INFO:teuthology.orchestra.run.trial128.stdout:Modify: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.053 INFO:teuthology.orchestra.run.trial128.stdout:Change: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.053 INFO:teuthology.orchestra.run.trial128.stdout: Birth: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.054 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-17T05:52:57.116 INFO:teuthology.orchestra.run.trial128.stderr:1+0 records in 2026-02-17T05:52:57.116 INFO:teuthology.orchestra.run.trial128.stderr:1+0 records out 2026-02-17T05:52:57.116 INFO:teuthology.orchestra.run.trial128.stderr:512 bytes copied, 0.000148504 s, 3.4 MB/s 2026-02-17T05:52:57.118 DEBUG:teuthology.orchestra.run.trial128:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-17T05:52:57.175 DEBUG:teuthology.orchestra.run.trial128:> stat /dev/vg_nvme/lv_2 2026-02-17T05:52:57.231 INFO:teuthology.orchestra.run.trial128.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-17T05:52:57.231 INFO:teuthology.orchestra.run.trial128.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:57.231 INFO:teuthology.orchestra.run.trial128.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-17T05:52:57.231 INFO:teuthology.orchestra.run.trial128.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:57.231 INFO:teuthology.orchestra.run.trial128.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:57.231 INFO:teuthology.orchestra.run.trial128.stdout:Access: 2026-02-17 05:48:41.314054771 +0000 2026-02-17T05:52:57.231 INFO:teuthology.orchestra.run.trial128.stdout:Modify: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.232 INFO:teuthology.orchestra.run.trial128.stdout:Change: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.232 INFO:teuthology.orchestra.run.trial128.stdout: Birth: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.232 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-17T05:52:57.293 INFO:teuthology.orchestra.run.trial128.stderr:1+0 records in 2026-02-17T05:52:57.293 INFO:teuthology.orchestra.run.trial128.stderr:1+0 records out 2026-02-17T05:52:57.293 INFO:teuthology.orchestra.run.trial128.stderr:512 bytes copied, 0.0002003 s, 2.6 MB/s 2026-02-17T05:52:57.294 DEBUG:teuthology.orchestra.run.trial128:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-17T05:52:57.352 DEBUG:teuthology.orchestra.run.trial128:> stat /dev/vg_nvme/lv_3 2026-02-17T05:52:57.410 INFO:teuthology.orchestra.run.trial128.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-17T05:52:57.410 INFO:teuthology.orchestra.run.trial128.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:57.410 INFO:teuthology.orchestra.run.trial128.stdout:Device: 6h/6d Inode: 1310 Links: 1 2026-02-17T05:52:57.410 INFO:teuthology.orchestra.run.trial128.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:57.410 INFO:teuthology.orchestra.run.trial128.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:57.410 INFO:teuthology.orchestra.run.trial128.stdout:Access: 2026-02-17 05:48:41.314054771 +0000 2026-02-17T05:52:57.410 INFO:teuthology.orchestra.run.trial128.stdout:Modify: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.410 INFO:teuthology.orchestra.run.trial128.stdout:Change: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.410 INFO:teuthology.orchestra.run.trial128.stdout: Birth: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.411 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-17T05:52:57.474 INFO:teuthology.orchestra.run.trial128.stderr:1+0 records in 2026-02-17T05:52:57.474 INFO:teuthology.orchestra.run.trial128.stderr:1+0 records out 2026-02-17T05:52:57.474 INFO:teuthology.orchestra.run.trial128.stderr:512 bytes copied, 0.000148084 s, 3.5 MB/s 2026-02-17T05:52:57.475 DEBUG:teuthology.orchestra.run.trial128:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-17T05:52:57.533 DEBUG:teuthology.orchestra.run.trial128:> stat /dev/vg_nvme/lv_4 2026-02-17T05:52:57.588 INFO:teuthology.orchestra.run.trial128.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-17T05:52:57.588 INFO:teuthology.orchestra.run.trial128.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:57.588 INFO:teuthology.orchestra.run.trial128.stdout:Device: 6h/6d Inode: 1303 Links: 1 2026-02-17T05:52:57.588 INFO:teuthology.orchestra.run.trial128.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:57.588 INFO:teuthology.orchestra.run.trial128.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:57.589 INFO:teuthology.orchestra.run.trial128.stdout:Access: 2026-02-17 05:48:41.314054771 +0000 2026-02-17T05:52:57.589 INFO:teuthology.orchestra.run.trial128.stdout:Modify: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.589 INFO:teuthology.orchestra.run.trial128.stdout:Change: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.589 INFO:teuthology.orchestra.run.trial128.stdout: Birth: 2026-02-17 05:48:06.440652903 +0000 2026-02-17T05:52:57.589 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-17T05:52:57.652 INFO:teuthology.orchestra.run.trial128.stderr:1+0 records in 2026-02-17T05:52:57.652 INFO:teuthology.orchestra.run.trial128.stderr:1+0 records out 2026-02-17T05:52:57.653 INFO:teuthology.orchestra.run.trial128.stderr:512 bytes copied, 0.000141292 s, 3.6 MB/s 2026-02-17T05:52:57.654 DEBUG:teuthology.orchestra.run.trial128:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-17T05:52:57.710 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-17T05:52:57.711 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:52:57.711 DEBUG:teuthology.orchestra.run.trial148:> dd if=/scratch_devs of=/dev/stdout 2026-02-17T05:52:57.729 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-17T05:52:57.729 DEBUG:teuthology.orchestra.run.trial148:> stat /dev/vg_nvme/lv_1 2026-02-17T05:52:57.787 INFO:teuthology.orchestra.run.trial148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-17T05:52:57.787 INFO:teuthology.orchestra.run.trial148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:57.787 INFO:teuthology.orchestra.run.trial148.stdout:Device: 6h/6d Inode: 1306 Links: 1 2026-02-17T05:52:57.787 INFO:teuthology.orchestra.run.trial148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:57.787 INFO:teuthology.orchestra.run.trial148.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:57.787 INFO:teuthology.orchestra.run.trial148.stdout:Access: 2026-02-17 05:48:42.702865326 +0000 2026-02-17T05:52:57.787 INFO:teuthology.orchestra.run.trial148.stdout:Modify: 2026-02-17 05:48:03.441533369 +0000 2026-02-17T05:52:57.787 INFO:teuthology.orchestra.run.trial148.stdout:Change: 2026-02-17 05:48:03.441533369 +0000 2026-02-17T05:52:57.788 INFO:teuthology.orchestra.run.trial148.stdout: Birth: 2026-02-17 05:48:03.441533369 +0000 2026-02-17T05:52:57.788 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-17T05:52:57.848 INFO:teuthology.orchestra.run.trial148.stderr:1+0 records in 2026-02-17T05:52:57.849 INFO:teuthology.orchestra.run.trial148.stderr:1+0 records out 2026-02-17T05:52:57.849 INFO:teuthology.orchestra.run.trial148.stderr:512 bytes copied, 0.000146183 s, 3.5 MB/s 2026-02-17T05:52:57.850 DEBUG:teuthology.orchestra.run.trial148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-17T05:52:57.907 DEBUG:teuthology.orchestra.run.trial148:> stat /dev/vg_nvme/lv_2 2026-02-17T05:52:57.963 INFO:teuthology.orchestra.run.trial148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-17T05:52:57.964 INFO:teuthology.orchestra.run.trial148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:57.964 INFO:teuthology.orchestra.run.trial148.stdout:Device: 6h/6d Inode: 1313 Links: 1 2026-02-17T05:52:57.964 INFO:teuthology.orchestra.run.trial148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:57.964 INFO:teuthology.orchestra.run.trial148.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:57.964 INFO:teuthology.orchestra.run.trial148.stdout:Access: 2026-02-17 05:48:42.702865326 +0000 2026-02-17T05:52:57.964 INFO:teuthology.orchestra.run.trial148.stdout:Modify: 2026-02-17 05:48:03.441533369 +0000 2026-02-17T05:52:57.964 INFO:teuthology.orchestra.run.trial148.stdout:Change: 2026-02-17 05:48:03.441533369 +0000 2026-02-17T05:52:57.964 INFO:teuthology.orchestra.run.trial148.stdout: Birth: 2026-02-17 05:48:03.441533369 +0000 2026-02-17T05:52:57.965 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-17T05:52:58.026 INFO:teuthology.orchestra.run.trial148.stderr:1+0 records in 2026-02-17T05:52:58.026 INFO:teuthology.orchestra.run.trial148.stderr:1+0 records out 2026-02-17T05:52:58.027 INFO:teuthology.orchestra.run.trial148.stderr:512 bytes copied, 0.000146002 s, 3.5 MB/s 2026-02-17T05:52:58.028 DEBUG:teuthology.orchestra.run.trial148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-17T05:52:58.085 DEBUG:teuthology.orchestra.run.trial148:> stat /dev/vg_nvme/lv_3 2026-02-17T05:52:58.141 INFO:teuthology.orchestra.run.trial148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-17T05:52:58.141 INFO:teuthology.orchestra.run.trial148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:58.141 INFO:teuthology.orchestra.run.trial148.stdout:Device: 6h/6d Inode: 1301 Links: 1 2026-02-17T05:52:58.141 INFO:teuthology.orchestra.run.trial148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:58.141 INFO:teuthology.orchestra.run.trial148.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:58.142 INFO:teuthology.orchestra.run.trial148.stdout:Access: 2026-02-17 05:48:42.702865326 +0000 2026-02-17T05:52:58.142 INFO:teuthology.orchestra.run.trial148.stdout:Modify: 2026-02-17 05:48:03.441533369 +0000 2026-02-17T05:52:58.142 INFO:teuthology.orchestra.run.trial148.stdout:Change: 2026-02-17 05:48:03.441533369 +0000 2026-02-17T05:52:58.142 INFO:teuthology.orchestra.run.trial148.stdout: Birth: 2026-02-17 05:48:03.441533369 +0000 2026-02-17T05:52:58.142 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-17T05:52:58.204 INFO:teuthology.orchestra.run.trial148.stderr:1+0 records in 2026-02-17T05:52:58.204 INFO:teuthology.orchestra.run.trial148.stderr:1+0 records out 2026-02-17T05:52:58.204 INFO:teuthology.orchestra.run.trial148.stderr:512 bytes copied, 0.00015503 s, 3.3 MB/s 2026-02-17T05:52:58.205 DEBUG:teuthology.orchestra.run.trial148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-17T05:52:58.264 DEBUG:teuthology.orchestra.run.trial148:> stat /dev/vg_nvme/lv_4 2026-02-17T05:52:58.323 INFO:teuthology.orchestra.run.trial148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-17T05:52:58.323 INFO:teuthology.orchestra.run.trial148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:58.323 INFO:teuthology.orchestra.run.trial148.stdout:Device: 6h/6d Inode: 1293 Links: 1 2026-02-17T05:52:58.323 INFO:teuthology.orchestra.run.trial148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:58.323 INFO:teuthology.orchestra.run.trial148.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:58.323 INFO:teuthology.orchestra.run.trial148.stdout:Access: 2026-02-17 05:48:42.703865309 +0000 2026-02-17T05:52:58.323 INFO:teuthology.orchestra.run.trial148.stdout:Modify: 2026-02-17 05:48:03.440533385 +0000 2026-02-17T05:52:58.324 INFO:teuthology.orchestra.run.trial148.stdout:Change: 2026-02-17 05:48:03.440533385 +0000 2026-02-17T05:52:58.324 INFO:teuthology.orchestra.run.trial148.stdout: Birth: 2026-02-17 05:48:03.440533385 +0000 2026-02-17T05:52:58.324 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-17T05:52:58.385 INFO:teuthology.orchestra.run.trial148.stderr:1+0 records in 2026-02-17T05:52:58.385 INFO:teuthology.orchestra.run.trial148.stderr:1+0 records out 2026-02-17T05:52:58.385 INFO:teuthology.orchestra.run.trial148.stderr:512 bytes copied, 0.000156903 s, 3.3 MB/s 2026-02-17T05:52:58.386 DEBUG:teuthology.orchestra.run.trial148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-17T05:52:58.444 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-17T05:52:58.444 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:52:58.444 DEBUG:teuthology.orchestra.run.trial164:> dd if=/scratch_devs of=/dev/stdout 2026-02-17T05:52:58.461 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-17T05:52:58.461 DEBUG:teuthology.orchestra.run.trial164:> stat /dev/vg_nvme/lv_1 2026-02-17T05:52:58.517 INFO:teuthology.orchestra.run.trial164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-17T05:52:58.518 INFO:teuthology.orchestra.run.trial164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:58.518 INFO:teuthology.orchestra.run.trial164.stdout:Device: 6h/6d Inode: 1312 Links: 1 2026-02-17T05:52:58.518 INFO:teuthology.orchestra.run.trial164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:58.518 INFO:teuthology.orchestra.run.trial164.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:58.518 INFO:teuthology.orchestra.run.trial164.stdout:Access: 2026-02-17 05:48:42.504311321 +0000 2026-02-17T05:52:58.518 INFO:teuthology.orchestra.run.trial164.stdout:Modify: 2026-02-17 05:48:08.882231972 +0000 2026-02-17T05:52:58.518 INFO:teuthology.orchestra.run.trial164.stdout:Change: 2026-02-17 05:48:08.882231972 +0000 2026-02-17T05:52:58.518 INFO:teuthology.orchestra.run.trial164.stdout: Birth: 2026-02-17 05:48:08.882231972 +0000 2026-02-17T05:52:58.519 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-17T05:52:58.580 INFO:teuthology.orchestra.run.trial164.stderr:1+0 records in 2026-02-17T05:52:58.580 INFO:teuthology.orchestra.run.trial164.stderr:1+0 records out 2026-02-17T05:52:58.580 INFO:teuthology.orchestra.run.trial164.stderr:512 bytes copied, 0.000163866 s, 3.1 MB/s 2026-02-17T05:52:58.582 DEBUG:teuthology.orchestra.run.trial164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-17T05:52:58.640 DEBUG:teuthology.orchestra.run.trial164:> stat /dev/vg_nvme/lv_2 2026-02-17T05:52:58.696 INFO:teuthology.orchestra.run.trial164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-17T05:52:58.697 INFO:teuthology.orchestra.run.trial164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:58.697 INFO:teuthology.orchestra.run.trial164.stdout:Device: 6h/6d Inode: 1315 Links: 1 2026-02-17T05:52:58.697 INFO:teuthology.orchestra.run.trial164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:58.697 INFO:teuthology.orchestra.run.trial164.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:58.697 INFO:teuthology.orchestra.run.trial164.stdout:Access: 2026-02-17 05:48:42.505311294 +0000 2026-02-17T05:52:58.697 INFO:teuthology.orchestra.run.trial164.stdout:Modify: 2026-02-17 05:48:08.883231945 +0000 2026-02-17T05:52:58.697 INFO:teuthology.orchestra.run.trial164.stdout:Change: 2026-02-17 05:48:08.883231945 +0000 2026-02-17T05:52:58.697 INFO:teuthology.orchestra.run.trial164.stdout: Birth: 2026-02-17 05:48:08.883231945 +0000 2026-02-17T05:52:58.698 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-17T05:52:58.760 INFO:teuthology.orchestra.run.trial164.stderr:1+0 records in 2026-02-17T05:52:58.760 INFO:teuthology.orchestra.run.trial164.stderr:1+0 records out 2026-02-17T05:52:58.760 INFO:teuthology.orchestra.run.trial164.stderr:512 bytes copied, 0.000147925 s, 3.5 MB/s 2026-02-17T05:52:58.761 DEBUG:teuthology.orchestra.run.trial164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-17T05:52:58.818 DEBUG:teuthology.orchestra.run.trial164:> stat /dev/vg_nvme/lv_3 2026-02-17T05:52:58.875 INFO:teuthology.orchestra.run.trial164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-17T05:52:58.876 INFO:teuthology.orchestra.run.trial164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:58.876 INFO:teuthology.orchestra.run.trial164.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-17T05:52:58.876 INFO:teuthology.orchestra.run.trial164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:58.876 INFO:teuthology.orchestra.run.trial164.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:58.876 INFO:teuthology.orchestra.run.trial164.stdout:Access: 2026-02-17 05:48:42.505311294 +0000 2026-02-17T05:52:58.876 INFO:teuthology.orchestra.run.trial164.stdout:Modify: 2026-02-17 05:48:08.882231972 +0000 2026-02-17T05:52:58.876 INFO:teuthology.orchestra.run.trial164.stdout:Change: 2026-02-17 05:48:08.882231972 +0000 2026-02-17T05:52:58.876 INFO:teuthology.orchestra.run.trial164.stdout: Birth: 2026-02-17 05:48:08.882231972 +0000 2026-02-17T05:52:58.877 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-17T05:52:58.940 INFO:teuthology.orchestra.run.trial164.stderr:1+0 records in 2026-02-17T05:52:58.940 INFO:teuthology.orchestra.run.trial164.stderr:1+0 records out 2026-02-17T05:52:58.940 INFO:teuthology.orchestra.run.trial164.stderr:512 bytes copied, 0.000162613 s, 3.1 MB/s 2026-02-17T05:52:58.941 DEBUG:teuthology.orchestra.run.trial164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-17T05:52:58.998 DEBUG:teuthology.orchestra.run.trial164:> stat /dev/vg_nvme/lv_4 2026-02-17T05:52:59.055 INFO:teuthology.orchestra.run.trial164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-17T05:52:59.055 INFO:teuthology.orchestra.run.trial164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:52:59.055 INFO:teuthology.orchestra.run.trial164.stdout:Device: 6h/6d Inode: 1309 Links: 1 2026-02-17T05:52:59.055 INFO:teuthology.orchestra.run.trial164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:52:59.055 INFO:teuthology.orchestra.run.trial164.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:52:59.055 INFO:teuthology.orchestra.run.trial164.stdout:Access: 2026-02-17 05:48:42.505311294 +0000 2026-02-17T05:52:59.055 INFO:teuthology.orchestra.run.trial164.stdout:Modify: 2026-02-17 05:48:08.882231972 +0000 2026-02-17T05:52:59.055 INFO:teuthology.orchestra.run.trial164.stdout:Change: 2026-02-17 05:48:08.882231972 +0000 2026-02-17T05:52:59.056 INFO:teuthology.orchestra.run.trial164.stdout: Birth: 2026-02-17 05:48:08.882231972 +0000 2026-02-17T05:52:59.056 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-17T05:52:59.117 INFO:teuthology.orchestra.run.trial164.stderr:1+0 records in 2026-02-17T05:52:59.117 INFO:teuthology.orchestra.run.trial164.stderr:1+0 records out 2026-02-17T05:52:59.118 INFO:teuthology.orchestra.run.trial164.stderr:512 bytes copied, 0.000160329 s, 3.2 MB/s 2026-02-17T05:52:59.119 DEBUG:teuthology.orchestra.run.trial164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-17T05:52:59.176 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-17T05:52:59.176 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial128.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial148.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial164.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2026-02-17T05:52:59.177 INFO:tasks.ceph:Generating config... 2026-02-17T05:52:59.178 INFO:tasks.ceph:[client] debug monc = 20 2026-02-17T05:52:59.178 INFO:tasks.ceph:[client] debug ms = 1 2026-02-17T05:52:59.178 INFO:tasks.ceph:[client] debug objecter = 20 2026-02-17T05:52:59.178 INFO:tasks.ceph:[client] debug rados = 20 2026-02-17T05:52:59.178 INFO:tasks.ceph:[global] mon client directed command retry = 5 2026-02-17T05:52:59.178 INFO:tasks.ceph:[global] mon election default strategy = 1 2026-02-17T05:52:59.178 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2026-02-17T05:52:59.179 INFO:tasks.ceph:[global] ms type = async 2026-02-17T05:52:59.179 INFO:tasks.ceph:[global] osd heartbeat grace = 160 2026-02-17T05:52:59.179 INFO:tasks.ceph:[global] osd op thread timeout = 150 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2026-02-17T05:52:59.179 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] bdev async discard = True 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] bdev enable discard = True 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] debug bluefs = 1 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] debug bluestore = 1 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-17T05:52:59.179 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] debug rocksdb = 10 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd class default list = * 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd class load list = * 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd client watch timeout = 240 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd fast shutdown = False 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-17T05:52:59.180 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-17T05:52:59.180 INFO:tasks.ceph:Setting up mon.a... 2026-02-17T05:52:59.180 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-17T05:52:59.216 INFO:teuthology.orchestra.run.trial016.stdout:creating /etc/ceph/ceph.keyring 2026-02-17T05:52:59.218 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-17T05:52:59.294 DEBUG:teuthology.orchestra.run.trial016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T05:52:59.357 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.16'), ('mon.b', '10.20.193.128'), ('mon.c', '10.20.193.148')] 2026-02-17T05:52:59.357 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.16,10.20.193.128,10.20.193.148', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2026-02-17T05:52:59.359 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:52:59.359 DEBUG:teuthology.orchestra.run.trial016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-17T05:52:59.415 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.16 --add b 10.20.193.128 --add c 10.20.193.148 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:52:59.489 INFO:teuthology.orchestra.run.trial016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:52:59.489 INFO:teuthology.orchestra.run.trial016.stdout:monmaptool: generated fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:52:59.489 INFO:teuthology.orchestra.run.trial016.stdout:setting min_mon_release = quincy 2026-02-17T05:52:59.489 INFO:teuthology.orchestra.run.trial016.stdout:epoch 0 2026-02-17T05:52:59.489 INFO:teuthology.orchestra.run.trial016.stdout:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:52:59.489 INFO:teuthology.orchestra.run.trial016.stdout:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:52:59.489 INFO:teuthology.orchestra.run.trial016.stdout:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:52:59.490 INFO:teuthology.orchestra.run.trial016.stdout:min_mon_release 17 (quincy) 2026-02-17T05:52:59.490 INFO:teuthology.orchestra.run.trial016.stdout:election_strategy: 1 2026-02-17T05:52:59.490 INFO:teuthology.orchestra.run.trial016.stdout:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:52:59.490 INFO:teuthology.orchestra.run.trial016.stdout:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:52:59.490 INFO:teuthology.orchestra.run.trial016.stdout:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:52:59.490 INFO:teuthology.orchestra.run.trial016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2026-02-17T05:52:59.492 DEBUG:teuthology.orchestra.run.trial016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-17T05:52:59.547 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6... 2026-02-17T05:52:59.548 DEBUG:teuthology.orchestra.run.trial016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-17T05:52:59.590 DEBUG:teuthology.orchestra.run.trial128:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-17T05:52:59.592 DEBUG:teuthology.orchestra.run.trial148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-17T05:52:59.595 DEBUG:teuthology.orchestra.run.trial164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-17T05:52:59.626 INFO:teuthology.orchestra.run.trial016.stdout:[global] 2026-02-17T05:52:59.626 INFO:teuthology.orchestra.run.trial016.stdout: chdir = "" 2026-02-17T05:52:59.626 INFO:teuthology.orchestra.run.trial016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-17T05:52:59.626 INFO:teuthology.orchestra.run.trial016.stdout: auth supported = cephx 2026-02-17T05:52:59.626 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: filestore xattr use omap = true 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: mon clock drift allowed = 1.000 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: osd crush chooseleaf type = 0 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: auth debug = true 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: ms die on old message = true 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: ms die on bug = true 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.627 INFO:teuthology.orchestra.run.trial016.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: mon pg warn max object skew = 0 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: # disable pg_autoscaler by default for new pools 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: osd pool default size = 2 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: mon osd allow primary affinity = true 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: mon osd allow pg remap = true 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: mon warn on legacy crush tunables = false 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: mon warn on crush straw calc version zero = false 2026-02-17T05:52:59.628 INFO:teuthology.orchestra.run.trial016.stdout: mon warn on no sortbitwise = false 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: mon warn on osd down out interval zero = false 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: mon warn on too few osds = false 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: mon_allow_pool_size_one = true 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: osd default data pool replay window = 5 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: mon allow pool delete = true 2026-02-17T05:52:59.629 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: mon cluster log file level = debug 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: debug asserts on shutdown = true 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: mon health detail to clog = false 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: mon host = "10.20.193.16,10.20.193.128,10.20.193.148" 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: mon client directed command retry = 5 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: mon election default strategy = 1 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: ms inject socket failures = 5000 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: ms type = async 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: osd heartbeat grace = 160 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: osd op thread timeout = 150 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: fsid = bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:52:59.630 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout:[osd] 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: osd journal size = 100 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: osd scrub load threshold = 5.0 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: osd scrub max interval = 600 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: osd mclock profile = high_recovery_ops 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: osd recover clone overlap = true 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: osd recovery max chunk = 1048576 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: osd debug shutdown = true 2026-02-17T05:52:59.631 INFO:teuthology.orchestra.run.trial016.stdout: osd debug op order = true 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: osd debug verify stray on activate = true 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: osd debug trim objects = true 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: osd open classes on start = true 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: osd debug pg log writeout = true 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: osd deep scrub update digest min age = 30 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: osd map max advance = 10 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: journal zero on create = true 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.632 INFO:teuthology.orchestra.run.trial016.stdout: filestore ondisk finisher threads = 3 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: filestore apply finisher threads = 3 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: bdev debug aio = true 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: osd debug misdirected ops = true 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: bdev async discard = True 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: bdev enable discard = True 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: bluestore allocator = bitmap 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: bluestore block size = 96636764160 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: bluestore fsck on mount = True 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: bluestore zero block detection = True 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: debug bluefs = 1 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: debug bluestore = 1 2026-02-17T05:52:59.633 INFO:teuthology.orchestra.run.trial016.stdout: debug monc = 20 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: debug ms = 1 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: debug osd = 20 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: debug rocksdb = 10 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: mon osd backfillfull_ratio = 0.85 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: mon osd full ratio = 0.9 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: mon osd nearfull ratio = 0.8 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: osd class default list = * 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: osd class load list = * 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: osd client watch timeout = 240 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: osd debug verify cached snaps = True 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: osd debug verify missing on start = True 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: osd failsafe full ratio = 0.95 2026-02-17T05:52:59.634 INFO:teuthology.orchestra.run.trial016.stdout: osd fast shutdown = False 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: osd mclock override recovery settings = True 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: osd objectstore = bluestore 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: osd op queue = debug_random 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: osd op queue cut off = debug_random 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: osd shutdown pgref assert = True 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout:[mgr] 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: debug ms = 1 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: debug mgr = 20 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: debug mon = 20 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: debug auth = 20 2026-02-17T05:52:59.635 INFO:teuthology.orchestra.run.trial016.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: mgr/telemetry/nag = false 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout:[mon] 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: debug ms = 1 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: debug mon = 20 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: debug paxos = 20 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: debug auth = 20 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: mon data avail warn = 5 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: mon mgr mkfs grace = 240 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: mon osd reporter subtree level = osd 2026-02-17T05:52:59.636 INFO:teuthology.orchestra.run.trial016.stdout: mon osd prime pg temp = true 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: auth mon ticket ttl = 660 # 11m 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: auth service ticket ttl = 240 # 4m 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: # don't complain about insecure global_id in the test suite 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: # 1m isn't quite enough 2026-02-17T05:52:59.637 INFO:teuthology.orchestra.run.trial016.stdout: mon_down_mkfs_grace = 2m 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: mon_warn_on_filestore_osds = false 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: mon min osdmap epochs = 50 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: mon osd crush smoke test = False 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: mon osdmap full prune interval = 2 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: mon osdmap full prune min = 15 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: mon osdmap full prune txsize = 2 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: mon scrub interval = 300 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: mon warn on pool no app = False 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: paxos service trim min = 10 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout:[client] 2026-02-17T05:52:59.638 INFO:teuthology.orchestra.run.trial016.stdout: rgw cache enabled = true 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout: rgw enable ops log = true 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout: rgw enable usage log = true 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout: debug monc = 20 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout: debug ms = 1 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout: debug objecter = 20 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout: debug rados = 20 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout:[mon.a] 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout:[mon.b] 2026-02-17T05:52:59.639 INFO:teuthology.orchestra.run.trial016.stdout:[mon.c] 2026-02-17T05:52:59.640 INFO:teuthology.orchestra.run.trial164.stdout:[global] 2026-02-17T05:52:59.641 INFO:teuthology.orchestra.run.trial164.stdout: chdir = "" 2026-02-17T05:52:59.641 INFO:teuthology.orchestra.run.trial164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-17T05:52:59.641 INFO:teuthology.orchestra.run.trial164.stdout: auth supported = cephx 2026-02-17T05:52:59.641 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.641 INFO:teuthology.orchestra.run.trial164.stdout: filestore xattr use omap = true 2026-02-17T05:52:59.641 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.641 INFO:teuthology.orchestra.run.trial164.stdout: mon clock drift allowed = 1.000 2026-02-17T05:52:59.641 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.641 INFO:teuthology.orchestra.run.trial164.stdout: osd crush chooseleaf type = 0 2026-02-17T05:52:59.641 INFO:teuthology.orchestra.run.trial164.stdout: auth debug = true 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: ms die on old message = true 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: ms die on bug = true 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: mon pg warn max object skew = 0 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: # disable pg_autoscaler by default for new pools 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: osd pool default size = 2 2026-02-17T05:52:59.642 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: mon osd allow primary affinity = true 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: mon osd allow pg remap = true 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: mon warn on legacy crush tunables = false 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: mon warn on crush straw calc version zero = false 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: mon warn on no sortbitwise = false 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: mon warn on osd down out interval zero = false 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: mon warn on too few osds = false 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: mon_allow_pool_size_one = true 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.643 INFO:teuthology.orchestra.run.trial164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: osd default data pool replay window = 5 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: mon allow pool delete = true 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: mon cluster log file level = debug 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: debug asserts on shutdown = true 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: mon health detail to clog = false 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: mon host = "10.20.193.16,10.20.193.128,10.20.193.148" 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: mon client directed command retry = 5 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: mon election default strategy = 1 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: ms inject socket failures = 5000 2026-02-17T05:52:59.644 INFO:teuthology.orchestra.run.trial164.stdout: ms type = async 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: osd heartbeat grace = 160 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: osd op thread timeout = 150 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: fsid = bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout:[osd] 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: osd journal size = 100 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: osd scrub load threshold = 5.0 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: osd scrub max interval = 600 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: osd mclock profile = high_recovery_ops 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.645 INFO:teuthology.orchestra.run.trial164.stdout: osd recover clone overlap = true 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: osd recovery max chunk = 1048576 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: osd debug shutdown = true 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: osd debug op order = true 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: osd debug verify stray on activate = true 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: osd debug trim objects = true 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: osd open classes on start = true 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: osd debug pg log writeout = true 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: osd deep scrub update digest min age = 30 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.646 INFO:teuthology.orchestra.run.trial164.stdout: osd map max advance = 10 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: journal zero on create = true 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: filestore ondisk finisher threads = 3 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: filestore apply finisher threads = 3 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: bdev debug aio = true 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: osd debug misdirected ops = true 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: bdev async discard = True 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: bdev enable discard = True 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: bluestore allocator = bitmap 2026-02-17T05:52:59.647 INFO:teuthology.orchestra.run.trial164.stdout: bluestore block size = 96636764160 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: bluestore fsck on mount = True 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: bluestore zero block detection = True 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: debug bluefs = 1 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: debug bluestore = 1 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: debug monc = 20 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: debug ms = 1 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: debug osd = 20 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: debug rocksdb = 10 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: mon osd backfillfull_ratio = 0.85 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: mon osd full ratio = 0.9 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: mon osd nearfull ratio = 0.8 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: osd class default list = * 2026-02-17T05:52:59.648 INFO:teuthology.orchestra.run.trial164.stdout: osd class load list = * 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: osd client watch timeout = 240 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: osd debug verify cached snaps = True 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: osd debug verify missing on start = True 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: osd failsafe full ratio = 0.95 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: osd fast shutdown = False 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: osd mclock override recovery settings = True 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: osd objectstore = bluestore 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: osd op queue = debug_random 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: osd op queue cut off = debug_random 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: osd shutdown pgref assert = True 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.649 INFO:teuthology.orchestra.run.trial164.stdout:[mgr] 2026-02-17T05:52:59.650 INFO:teuthology.orchestra.run.trial164.stdout: debug ms = 1 2026-02-17T05:52:59.650 INFO:teuthology.orchestra.run.trial164.stdout: debug mgr = 20 2026-02-17T05:52:59.650 INFO:teuthology.orchestra.run.trial164.stdout: debug mon = 20 2026-02-17T05:52:59.650 INFO:teuthology.orchestra.run.trial164.stdout: debug auth = 20 2026-02-17T05:52:59.650 INFO:teuthology.orchestra.run.trial164.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:52:59.650 INFO:teuthology.orchestra.run.trial164.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:52:59.650 INFO:teuthology.orchestra.run.trial164.stdout: mgr/telemetry/nag = false 2026-02-17T05:52:59.650 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.650 INFO:teuthology.orchestra.run.trial164.stdout:[mon] 2026-02-17T05:52:59.650 INFO:teuthology.orchestra.run.trial164.stdout: debug ms = 1 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: debug mon = 20 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: debug paxos = 20 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: debug auth = 20 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: mon data avail warn = 5 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: mon mgr mkfs grace = 240 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: mon osd reporter subtree level = osd 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: mon osd prime pg temp = true 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-17T05:52:59.651 INFO:teuthology.orchestra.run.trial164.stdout: auth mon ticket ttl = 660 # 11m 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: auth service ticket ttl = 240 # 4m 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: # don't complain about insecure global_id in the test suite 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: # 1m isn't quite enough 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: mon_down_mkfs_grace = 2m 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: mon_warn_on_filestore_osds = false 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: mon min osdmap epochs = 50 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: mon osd crush smoke test = False 2026-02-17T05:52:59.652 INFO:teuthology.orchestra.run.trial164.stdout: mon osdmap full prune interval = 2 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: mon osdmap full prune min = 15 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: mon osdmap full prune txsize = 2 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: mon scrub interval = 300 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: mon warn on pool no app = False 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: paxos service trim min = 10 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout:[client] 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: rgw cache enabled = true 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: rgw enable ops log = true 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: rgw enable usage log = true 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-17T05:52:59.653 INFO:teuthology.orchestra.run.trial164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial164.stdout: debug monc = 20 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial164.stdout: debug ms = 1 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial164.stdout: debug objecter = 20 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial164.stdout: debug rados = 20 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial164.stdout:[mon.a] 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial164.stdout:[mon.b] 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial164.stdout:[mon.c] 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial148.stdout:[global] 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial148.stdout: chdir = "" 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-17T05:52:59.654 INFO:teuthology.orchestra.run.trial148.stdout: auth supported = cephx 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: filestore xattr use omap = true 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: mon clock drift allowed = 1.000 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: osd crush chooseleaf type = 0 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: auth debug = true 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: ms die on old message = true 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: ms die on bug = true 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: mon pg warn max object skew = 0 2026-02-17T05:52:59.655 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: # disable pg_autoscaler by default for new pools 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: osd pool default size = 2 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: mon osd allow primary affinity = true 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: mon osd allow pg remap = true 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: mon warn on legacy crush tunables = false 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: mon warn on crush straw calc version zero = false 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: mon warn on no sortbitwise = false 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: mon warn on osd down out interval zero = false 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: mon warn on too few osds = false 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-17T05:52:59.656 INFO:teuthology.orchestra.run.trial148.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: mon_allow_pool_size_one = true 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: osd default data pool replay window = 5 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: mon allow pool delete = true 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: mon cluster log file level = debug 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: debug asserts on shutdown = true 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: mon health detail to clog = false 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: mon host = "10.20.193.16,10.20.193.128,10.20.193.148" 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: mon client directed command retry = 5 2026-02-17T05:52:59.657 INFO:teuthology.orchestra.run.trial148.stdout: mon election default strategy = 1 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: ms inject socket failures = 5000 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: ms type = async 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: osd heartbeat grace = 160 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: osd op thread timeout = 150 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: fsid = bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout:[osd] 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: osd journal size = 100 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: osd scrub load threshold = 5.0 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: osd scrub max interval = 600 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: osd mclock profile = high_recovery_ops 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.658 INFO:teuthology.orchestra.run.trial148.stdout: osd recover clone overlap = true 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: osd recovery max chunk = 1048576 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: osd debug shutdown = true 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: osd debug op order = true 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: osd debug verify stray on activate = true 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: osd debug trim objects = true 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: osd open classes on start = true 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: osd debug pg log writeout = true 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: osd deep scrub update digest min age = 30 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: osd map max advance = 10 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: journal zero on create = true 2026-02-17T05:52:59.659 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: filestore ondisk finisher threads = 3 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: filestore apply finisher threads = 3 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: bdev debug aio = true 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: osd debug misdirected ops = true 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: bdev async discard = True 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: bdev enable discard = True 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: bluestore allocator = bitmap 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: bluestore block size = 96636764160 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: bluestore fsck on mount = True 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: bluestore zero block detection = True 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: debug bluefs = 1 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: debug bluestore = 1 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: debug monc = 20 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: debug ms = 1 2026-02-17T05:52:59.660 INFO:teuthology.orchestra.run.trial148.stdout: debug osd = 20 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: debug rocksdb = 10 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: mon osd backfillfull_ratio = 0.85 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: mon osd full ratio = 0.9 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: mon osd nearfull ratio = 0.8 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd class default list = * 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd class load list = * 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd client watch timeout = 240 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd debug verify cached snaps = True 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd debug verify missing on start = True 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd failsafe full ratio = 0.95 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd fast shutdown = False 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd mclock override recovery settings = True 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd objectstore = bluestore 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd op queue = debug_random 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd op queue cut off = debug_random 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: osd shutdown pgref assert = True 2026-02-17T05:52:59.661 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout:[mgr] 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: debug ms = 1 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: debug mgr = 20 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: debug mon = 20 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: debug auth = 20 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: mgr/telemetry/nag = false 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout:[mon] 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: debug ms = 1 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: debug mon = 20 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: debug paxos = 20 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: debug auth = 20 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: mon data avail warn = 5 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: mon mgr mkfs grace = 240 2026-02-17T05:52:59.662 INFO:teuthology.orchestra.run.trial148.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: mon osd reporter subtree level = osd 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: mon osd prime pg temp = true 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: auth mon ticket ttl = 660 # 11m 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: auth service ticket ttl = 240 # 4m 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: # don't complain about insecure global_id in the test suite 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: # 1m isn't quite enough 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: mon_down_mkfs_grace = 2m 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: mon_warn_on_filestore_osds = false 2026-02-17T05:52:59.663 INFO:teuthology.orchestra.run.trial148.stdout: mon min osdmap epochs = 50 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: mon osd crush smoke test = False 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: mon osdmap full prune interval = 2 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: mon osdmap full prune min = 15 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: mon osdmap full prune txsize = 2 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: mon scrub interval = 300 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: mon warn on pool no app = False 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: paxos service trim min = 10 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout:[client] 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: rgw cache enabled = true 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: rgw enable ops log = true 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: rgw enable usage log = true 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: debug monc = 20 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: debug ms = 1 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: debug objecter = 20 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout: debug rados = 20 2026-02-17T05:52:59.664 INFO:teuthology.orchestra.run.trial148.stdout:[mon.a] 2026-02-17T05:52:59.665 INFO:teuthology.orchestra.run.trial148.stdout:[mon.b] 2026-02-17T05:52:59.665 INFO:teuthology.orchestra.run.trial148.stdout:[mon.c] 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout:[global] 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: chdir = "" 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: auth supported = cephx 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: filestore xattr use omap = true 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: mon clock drift allowed = 1.000 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: osd crush chooseleaf type = 0 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: auth debug = true 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: ms die on old message = true 2026-02-17T05:52:59.666 INFO:teuthology.orchestra.run.trial128.stdout: ms die on bug = true 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon pg warn max object skew = 0 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: # disable pg_autoscaler by default for new pools 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: osd pool default size = 2 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon osd allow primary affinity = true 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon osd allow pg remap = true 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon warn on legacy crush tunables = false 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon warn on crush straw calc version zero = false 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon warn on no sortbitwise = false 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon warn on osd down out interval zero = false 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon warn on too few osds = false 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-17T05:52:59.667 INFO:teuthology.orchestra.run.trial128.stdout: mon_allow_pool_size_one = true 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: osd default data pool replay window = 5 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: mon allow pool delete = true 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: mon cluster log file level = debug 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: debug asserts on shutdown = true 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: mon health detail to clog = false 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: mon host = "10.20.193.16,10.20.193.128,10.20.193.148" 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: mon client directed command retry = 5 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: mon election default strategy = 1 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: ms inject socket failures = 5000 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: ms type = async 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: osd heartbeat grace = 160 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: osd op thread timeout = 150 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: fsid = bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:52:59.668 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout:[osd] 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd journal size = 100 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd scrub load threshold = 5.0 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd scrub max interval = 600 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd mclock profile = high_recovery_ops 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd recover clone overlap = true 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd recovery max chunk = 1048576 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd debug shutdown = true 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd debug op order = true 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd debug verify stray on activate = true 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd debug trim objects = true 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd open classes on start = true 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd debug pg log writeout = true 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.669 INFO:teuthology.orchestra.run.trial128.stdout: osd deep scrub update digest min age = 30 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: osd map max advance = 10 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: journal zero on create = true 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: filestore ondisk finisher threads = 3 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: filestore apply finisher threads = 3 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: bdev debug aio = true 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: osd debug misdirected ops = true 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: bdev async discard = True 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: bdev enable discard = True 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: bluestore allocator = bitmap 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: bluestore block size = 96636764160 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: bluestore fsck on mount = True 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: bluestore zero block detection = True 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: debug bluefs = 1 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: debug bluestore = 1 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: debug monc = 20 2026-02-17T05:52:59.670 INFO:teuthology.orchestra.run.trial128.stdout: debug ms = 1 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: debug osd = 20 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: debug rocksdb = 10 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: mon osd backfillfull_ratio = 0.85 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: mon osd full ratio = 0.9 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: mon osd nearfull ratio = 0.8 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd class default list = * 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd class load list = * 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd client watch timeout = 240 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd debug verify cached snaps = True 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd debug verify missing on start = True 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd failsafe full ratio = 0.95 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd fast shutdown = False 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd mclock override recovery settings = True 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd objectstore = bluestore 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd op queue = debug_random 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd op queue cut off = debug_random 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: osd shutdown pgref assert = True 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout:[mgr] 2026-02-17T05:52:59.671 INFO:teuthology.orchestra.run.trial128.stdout: debug ms = 1 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: debug mgr = 20 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: debug mon = 20 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: debug auth = 20 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: mgr/telemetry/nag = false 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout:[mon] 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: debug ms = 1 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: debug mon = 20 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: debug paxos = 20 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: debug auth = 20 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: mon data avail warn = 5 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: mon mgr mkfs grace = 240 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: mon osd reporter subtree level = osd 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: mon osd prime pg temp = true 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-17T05:52:59.672 INFO:teuthology.orchestra.run.trial128.stdout: auth mon ticket ttl = 660 # 11m 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: auth service ticket ttl = 240 # 4m 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: # don't complain about insecure global_id in the test suite 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: # 1m isn't quite enough 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon_down_mkfs_grace = 2m 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon_warn_on_filestore_osds = false 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon min osdmap epochs = 50 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon osd crush smoke test = False 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon osdmap full prune interval = 2 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon osdmap full prune min = 15 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon osdmap full prune txsize = 2 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon scrub interval = 300 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: mon warn on pool no app = False 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: paxos service trim min = 10 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout:[client] 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: rgw cache enabled = true 2026-02-17T05:52:59.673 INFO:teuthology.orchestra.run.trial128.stdout: rgw enable ops log = true 2026-02-17T05:52:59.674 INFO:teuthology.orchestra.run.trial128.stdout: rgw enable usage log = true 2026-02-17T05:52:59.674 INFO:teuthology.orchestra.run.trial128.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-17T05:52:59.674 INFO:teuthology.orchestra.run.trial128.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-17T05:52:59.674 INFO:teuthology.orchestra.run.trial128.stdout: debug monc = 20 2026-02-17T05:52:59.674 INFO:teuthology.orchestra.run.trial128.stdout: debug ms = 1 2026-02-17T05:52:59.674 INFO:teuthology.orchestra.run.trial128.stdout: debug objecter = 20 2026-02-17T05:52:59.674 INFO:teuthology.orchestra.run.trial128.stdout: debug rados = 20 2026-02-17T05:52:59.674 INFO:teuthology.orchestra.run.trial128.stdout:[mon.a] 2026-02-17T05:52:59.674 INFO:teuthology.orchestra.run.trial128.stdout:[mon.b] 2026-02-17T05:52:59.674 INFO:teuthology.orchestra.run.trial128.stdout:[mon.c] 2026-02-17T05:52:59.675 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-17T05:52:59.675 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-17T05:52:59.748 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-17T05:52:59.748 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:52:59.748 DEBUG:teuthology.orchestra.run.trial016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-17T05:52:59.805 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:52:59.806 DEBUG:teuthology.orchestra.run.trial016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-17T05:52:59.862 INFO:tasks.ceph:Sending monmap to node ubuntu@trial016.front.sepia.ceph.com 2026-02-17T05:52:59.863 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:52:59.863 DEBUG:teuthology.orchestra.run.trial016:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-17T05:52:59.863 DEBUG:teuthology.orchestra.run.trial016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T05:52:59.935 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:52:59.936 DEBUG:teuthology.orchestra.run.trial016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:52:59.993 INFO:tasks.ceph:Sending monmap to node ubuntu@trial128.front.sepia.ceph.com 2026-02-17T05:52:59.994 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:52:59.994 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-17T05:52:59.994 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T05:53:00.026 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:53:00.027 DEBUG:teuthology.orchestra.run.trial128:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:00.082 INFO:tasks.ceph:Sending monmap to node ubuntu@trial148.front.sepia.ceph.com 2026-02-17T05:53:00.083 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:53:00.083 DEBUG:teuthology.orchestra.run.trial148:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-17T05:53:00.083 DEBUG:teuthology.orchestra.run.trial148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T05:53:00.117 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:53:00.117 DEBUG:teuthology.orchestra.run.trial148:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:00.172 INFO:tasks.ceph:Sending monmap to node ubuntu@trial164.front.sepia.ceph.com 2026-02-17T05:53:00.172 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:53:00.172 DEBUG:teuthology.orchestra.run.trial164:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-17T05:53:00.172 DEBUG:teuthology.orchestra.run.trial164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T05:53:00.207 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:53:00.207 DEBUG:teuthology.orchestra.run.trial164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:00.262 INFO:tasks.ceph:Setting up mon nodes... 2026-02-17T05:53:00.262 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-17T05:53:00.263 DEBUG:teuthology.orchestra.run.trial016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2026-02-17T05:53:00.306 INFO:teuthology.orchestra.run.trial016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2026-02-17T05:53:00.308 DEBUG:teuthology.orchestra.run.trial164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-17T05:53:00.351 INFO:teuthology.orchestra.run.trial164.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-17T05:53:00.354 INFO:tasks.ceph:Setting up mds nodes... 2026-02-17T05:53:00.355 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-17T05:53:00.355 DEBUG:teuthology.orchestra.run.trial164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-17T05:53:00.388 INFO:teuthology.orchestra.run.trial164.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-17T05:53:00.399 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-17T05:53:00.399 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial128.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial148.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial164.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2026-02-17T05:53:00.400 DEBUG:teuthology.orchestra.run.trial016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-17T05:53:00.425 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:00.425 INFO:tasks.ceph:role: osd.0 2026-02-17T05:53:00.425 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial016.front.sepia.ceph.com 2026-02-17T05:53:00.425 DEBUG:teuthology.orchestra.run.trial016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-17T05:53:00.489 INFO:teuthology.orchestra.run.trial016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:00.489 INFO:teuthology.orchestra.run.trial016.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:00.489 INFO:teuthology.orchestra.run.trial016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:00.489 INFO:teuthology.orchestra.run.trial016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:00.489 INFO:teuthology.orchestra.run.trial016.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:00.489 INFO:teuthology.orchestra.run.trial016.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:00.489 INFO:teuthology.orchestra.run.trial016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:00.490 INFO:teuthology.orchestra.run.trial016.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:00.490 INFO:teuthology.orchestra.run.trial016.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:00.490 INFO:teuthology.orchestra.run.trial016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:01.068 INFO:teuthology.orchestra.run.trial016.stdout:Discarding blocks...Done. 2026-02-17T05:53:01.070 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial016.front.sepia.ceph.com -o noatime 2026-02-17T05:53:01.070 DEBUG:teuthology.orchestra.run.trial016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-17T05:53:01.138 DEBUG:teuthology.orchestra.run.trial016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-17T05:53:01.205 DEBUG:teuthology.orchestra.run.trial016:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2026-02-17T05:53:01.271 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:01.272 INFO:tasks.ceph:role: osd.4 2026-02-17T05:53:01.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial016.front.sepia.ceph.com 2026-02-17T05:53:01.272 DEBUG:teuthology.orchestra.run.trial016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-17T05:53:01.336 INFO:teuthology.orchestra.run.trial016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:01.336 INFO:teuthology.orchestra.run.trial016.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:01.336 INFO:teuthology.orchestra.run.trial016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:01.336 INFO:teuthology.orchestra.run.trial016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:01.337 INFO:teuthology.orchestra.run.trial016.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:01.337 INFO:teuthology.orchestra.run.trial016.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:01.337 INFO:teuthology.orchestra.run.trial016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:01.337 INFO:teuthology.orchestra.run.trial016.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:01.337 INFO:teuthology.orchestra.run.trial016.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:01.337 INFO:teuthology.orchestra.run.trial016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:01.894 INFO:teuthology.orchestra.run.trial016.stdout:Discarding blocks...Done. 2026-02-17T05:53:01.896 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial016.front.sepia.ceph.com -o noatime 2026-02-17T05:53:01.896 DEBUG:teuthology.orchestra.run.trial016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2026-02-17T05:53:01.963 DEBUG:teuthology.orchestra.run.trial016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2026-02-17T05:53:02.029 DEBUG:teuthology.orchestra.run.trial016:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2026-02-17T05:53:02.093 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:02.094 INFO:tasks.ceph:role: osd.8 2026-02-17T05:53:02.094 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial016.front.sepia.ceph.com 2026-02-17T05:53:02.094 DEBUG:teuthology.orchestra.run.trial016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-17T05:53:02.157 INFO:teuthology.orchestra.run.trial016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:02.157 INFO:teuthology.orchestra.run.trial016.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:02.157 INFO:teuthology.orchestra.run.trial016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:02.157 INFO:teuthology.orchestra.run.trial016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:02.157 INFO:teuthology.orchestra.run.trial016.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:02.157 INFO:teuthology.orchestra.run.trial016.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:02.157 INFO:teuthology.orchestra.run.trial016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:02.157 INFO:teuthology.orchestra.run.trial016.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:02.157 INFO:teuthology.orchestra.run.trial016.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:02.157 INFO:teuthology.orchestra.run.trial016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:02.723 INFO:teuthology.orchestra.run.trial016.stdout:Discarding blocks...Done. 2026-02-17T05:53:02.725 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial016.front.sepia.ceph.com -o noatime 2026-02-17T05:53:02.725 DEBUG:teuthology.orchestra.run.trial016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2026-02-17T05:53:02.792 DEBUG:teuthology.orchestra.run.trial016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2026-02-17T05:53:02.860 DEBUG:teuthology.orchestra.run.trial016:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2026-02-17T05:53:02.924 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:02.924 INFO:tasks.ceph:role: osd.12 2026-02-17T05:53:02.924 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial016.front.sepia.ceph.com 2026-02-17T05:53:02.924 DEBUG:teuthology.orchestra.run.trial016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-17T05:53:02.989 INFO:teuthology.orchestra.run.trial016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:02.989 INFO:teuthology.orchestra.run.trial016.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:02.989 INFO:teuthology.orchestra.run.trial016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:02.989 INFO:teuthology.orchestra.run.trial016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:02.989 INFO:teuthology.orchestra.run.trial016.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:02.990 INFO:teuthology.orchestra.run.trial016.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:02.990 INFO:teuthology.orchestra.run.trial016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:02.990 INFO:teuthology.orchestra.run.trial016.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:02.990 INFO:teuthology.orchestra.run.trial016.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:02.990 INFO:teuthology.orchestra.run.trial016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:03.546 INFO:teuthology.orchestra.run.trial016.stdout:Discarding blocks...Done. 2026-02-17T05:53:03.548 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial016.front.sepia.ceph.com -o noatime 2026-02-17T05:53:03.548 DEBUG:teuthology.orchestra.run.trial016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2026-02-17T05:53:03.626 DEBUG:teuthology.orchestra.run.trial016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2026-02-17T05:53:03.693 DEBUG:teuthology.orchestra.run.trial016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:03.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:03.771+0000 7ff57a818780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-17T05:53:03.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:03.772+0000 7ff57a818780 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-17T05:53:03.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:03.772+0000 7ff57a818780 -1 bdev(0x560f36fcd800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:03.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:03.772+0000 7ff57a818780 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-17T05:53:07.653 DEBUG:teuthology.orchestra.run.trial016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-17T05:53:07.717 DEBUG:teuthology.orchestra.run.trial016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:07.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:07.793+0000 7fc900134780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2026-02-17T05:53:07.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:07.793+0000 7fc900134780 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2026-02-17T05:53:07.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:07.793+0000 7fc900134780 -1 bdev(0x5565138f5800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:07.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:07.793+0000 7fc900134780 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2026-02-17T05:53:11.674 DEBUG:teuthology.orchestra.run.trial016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2026-02-17T05:53:11.740 DEBUG:teuthology.orchestra.run.trial016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:11.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:11.818+0000 7f0906614780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2026-02-17T05:53:11.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:11.818+0000 7f0906614780 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2026-02-17T05:53:11.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:11.819+0000 7f0906614780 -1 bdev(0x556392c83800 /var/lib/ceph/osd/ceph-8/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:11.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:11.819+0000 7f0906614780 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2026-02-17T05:53:15.695 DEBUG:teuthology.orchestra.run.trial016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2026-02-17T05:53:15.760 DEBUG:teuthology.orchestra.run.trial016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:15.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:15.838+0000 7fcbf812c780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2026-02-17T05:53:15.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:15.838+0000 7fcbf812c780 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2026-02-17T05:53:15.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:15.838+0000 7fcbf812c780 -1 bdev(0x56267efb5800 /var/lib/ceph/osd/ceph-12/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:15.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:53:15.839+0000 7fcbf812c780 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2026-02-17T05:53:19.728 DEBUG:teuthology.orchestra.run.trial016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2026-02-17T05:53:19.793 DEBUG:teuthology.orchestra.run.trial128:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-17T05:53:19.819 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:19.819 INFO:tasks.ceph:role: osd.1 2026-02-17T05:53:19.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial128.front.sepia.ceph.com 2026-02-17T05:53:19.819 DEBUG:teuthology.orchestra.run.trial128:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-17T05:53:19.885 INFO:teuthology.orchestra.run.trial128.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:19.886 INFO:teuthology.orchestra.run.trial128.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:19.886 INFO:teuthology.orchestra.run.trial128.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:19.886 INFO:teuthology.orchestra.run.trial128.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:19.886 INFO:teuthology.orchestra.run.trial128.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:19.886 INFO:teuthology.orchestra.run.trial128.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:19.886 INFO:teuthology.orchestra.run.trial128.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:19.886 INFO:teuthology.orchestra.run.trial128.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:19.886 INFO:teuthology.orchestra.run.trial128.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:19.886 INFO:teuthology.orchestra.run.trial128.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:20.458 INFO:teuthology.orchestra.run.trial128.stdout:Discarding blocks...Done. 2026-02-17T05:53:20.460 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial128.front.sepia.ceph.com -o noatime 2026-02-17T05:53:20.460 DEBUG:teuthology.orchestra.run.trial128:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2026-02-17T05:53:20.528 DEBUG:teuthology.orchestra.run.trial128:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-17T05:53:20.594 DEBUG:teuthology.orchestra.run.trial128:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2026-02-17T05:53:20.658 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:20.658 INFO:tasks.ceph:role: osd.5 2026-02-17T05:53:20.658 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial128.front.sepia.ceph.com 2026-02-17T05:53:20.658 DEBUG:teuthology.orchestra.run.trial128:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-17T05:53:20.723 INFO:teuthology.orchestra.run.trial128.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:20.724 INFO:teuthology.orchestra.run.trial128.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:20.724 INFO:teuthology.orchestra.run.trial128.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:20.724 INFO:teuthology.orchestra.run.trial128.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:20.724 INFO:teuthology.orchestra.run.trial128.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:20.724 INFO:teuthology.orchestra.run.trial128.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:20.724 INFO:teuthology.orchestra.run.trial128.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:20.724 INFO:teuthology.orchestra.run.trial128.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:20.724 INFO:teuthology.orchestra.run.trial128.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:20.724 INFO:teuthology.orchestra.run.trial128.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:21.281 INFO:teuthology.orchestra.run.trial128.stdout:Discarding blocks...Done. 2026-02-17T05:53:21.283 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial128.front.sepia.ceph.com -o noatime 2026-02-17T05:53:21.284 DEBUG:teuthology.orchestra.run.trial128:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2026-02-17T05:53:21.352 DEBUG:teuthology.orchestra.run.trial128:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2026-02-17T05:53:21.423 DEBUG:teuthology.orchestra.run.trial128:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2026-02-17T05:53:21.486 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:21.487 INFO:tasks.ceph:role: osd.9 2026-02-17T05:53:21.487 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial128.front.sepia.ceph.com 2026-02-17T05:53:21.487 DEBUG:teuthology.orchestra.run.trial128:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-17T05:53:21.551 INFO:teuthology.orchestra.run.trial128.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:21.551 INFO:teuthology.orchestra.run.trial128.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:21.551 INFO:teuthology.orchestra.run.trial128.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:21.551 INFO:teuthology.orchestra.run.trial128.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:21.551 INFO:teuthology.orchestra.run.trial128.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:21.551 INFO:teuthology.orchestra.run.trial128.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:21.551 INFO:teuthology.orchestra.run.trial128.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:21.551 INFO:teuthology.orchestra.run.trial128.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:21.551 INFO:teuthology.orchestra.run.trial128.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:21.551 INFO:teuthology.orchestra.run.trial128.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:22.109 INFO:teuthology.orchestra.run.trial128.stdout:Discarding blocks...Done. 2026-02-17T05:53:22.112 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial128.front.sepia.ceph.com -o noatime 2026-02-17T05:53:22.112 DEBUG:teuthology.orchestra.run.trial128:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2026-02-17T05:53:22.186 DEBUG:teuthology.orchestra.run.trial128:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2026-02-17T05:53:22.255 DEBUG:teuthology.orchestra.run.trial128:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2026-02-17T05:53:22.319 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:22.320 INFO:tasks.ceph:role: osd.13 2026-02-17T05:53:22.320 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial128.front.sepia.ceph.com 2026-02-17T05:53:22.320 DEBUG:teuthology.orchestra.run.trial128:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-17T05:53:22.384 INFO:teuthology.orchestra.run.trial128.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:22.385 INFO:teuthology.orchestra.run.trial128.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:22.385 INFO:teuthology.orchestra.run.trial128.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:22.385 INFO:teuthology.orchestra.run.trial128.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:22.385 INFO:teuthology.orchestra.run.trial128.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:22.385 INFO:teuthology.orchestra.run.trial128.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:22.385 INFO:teuthology.orchestra.run.trial128.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:22.385 INFO:teuthology.orchestra.run.trial128.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:22.385 INFO:teuthology.orchestra.run.trial128.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:22.386 INFO:teuthology.orchestra.run.trial128.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:22.942 INFO:teuthology.orchestra.run.trial128.stdout:Discarding blocks...Done. 2026-02-17T05:53:22.944 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial128.front.sepia.ceph.com -o noatime 2026-02-17T05:53:22.944 DEBUG:teuthology.orchestra.run.trial128:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2026-02-17T05:53:23.019 DEBUG:teuthology.orchestra.run.trial128:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2026-02-17T05:53:23.088 DEBUG:teuthology.orchestra.run.trial128:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:23.167 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:23.167+0000 7f4432413780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-17T05:53:23.167 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:23.167+0000 7f4432413780 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-17T05:53:23.168 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:23.167+0000 7f4432413780 -1 bdev(0x558f4abe7800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:23.168 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:23.167+0000 7f4432413780 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-17T05:53:27.044 DEBUG:teuthology.orchestra.run.trial128:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-17T05:53:27.111 DEBUG:teuthology.orchestra.run.trial128:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:27.189 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:27.188+0000 7f4ec0a13780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2026-02-17T05:53:27.189 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:27.188+0000 7f4ec0a13780 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2026-02-17T05:53:27.189 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:27.189+0000 7f4ec0a13780 -1 bdev(0x55e131989800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:27.189 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:27.189+0000 7f4ec0a13780 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2026-02-17T05:53:31.073 DEBUG:teuthology.orchestra.run.trial128:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2026-02-17T05:53:31.137 DEBUG:teuthology.orchestra.run.trial128:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:31.215 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:31.214+0000 7fc229410780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2026-02-17T05:53:31.215 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:31.214+0000 7fc229410780 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2026-02-17T05:53:31.215 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:31.214+0000 7fc229410780 -1 bdev(0x55f5a5ecb800 /var/lib/ceph/osd/ceph-9/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:31.215 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:31.215+0000 7fc229410780 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2026-02-17T05:53:35.092 DEBUG:teuthology.orchestra.run.trial128:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2026-02-17T05:53:35.159 DEBUG:teuthology.orchestra.run.trial128:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:35.236+0000 7f7abf03f780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:35.236+0000 7f7abf03f780 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:35.236+0000 7f7abf03f780 -1 bdev(0x5627bc5a9800 /var/lib/ceph/osd/ceph-13/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial128.stderr:2026-02-17T05:53:35.236+0000 7f7abf03f780 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2026-02-17T05:53:39.134 DEBUG:teuthology.orchestra.run.trial128:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2026-02-17T05:53:39.199 DEBUG:teuthology.orchestra.run.trial148:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-17T05:53:39.225 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:39.225 INFO:tasks.ceph:role: osd.2 2026-02-17T05:53:39.226 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial148.front.sepia.ceph.com 2026-02-17T05:53:39.226 DEBUG:teuthology.orchestra.run.trial148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-17T05:53:39.290 INFO:teuthology.orchestra.run.trial148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:39.291 INFO:teuthology.orchestra.run.trial148.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:39.291 INFO:teuthology.orchestra.run.trial148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:39.291 INFO:teuthology.orchestra.run.trial148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:39.291 INFO:teuthology.orchestra.run.trial148.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:39.291 INFO:teuthology.orchestra.run.trial148.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:39.291 INFO:teuthology.orchestra.run.trial148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:39.291 INFO:teuthology.orchestra.run.trial148.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:39.291 INFO:teuthology.orchestra.run.trial148.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:39.291 INFO:teuthology.orchestra.run.trial148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:39.869 INFO:teuthology.orchestra.run.trial148.stdout:Discarding blocks...Done. 2026-02-17T05:53:39.871 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial148.front.sepia.ceph.com -o noatime 2026-02-17T05:53:39.871 DEBUG:teuthology.orchestra.run.trial148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2026-02-17T05:53:39.941 DEBUG:teuthology.orchestra.run.trial148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-17T05:53:40.007 DEBUG:teuthology.orchestra.run.trial148:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2026-02-17T05:53:40.071 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:40.071 INFO:tasks.ceph:role: osd.6 2026-02-17T05:53:40.072 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial148.front.sepia.ceph.com 2026-02-17T05:53:40.072 DEBUG:teuthology.orchestra.run.trial148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-17T05:53:40.136 INFO:teuthology.orchestra.run.trial148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:40.137 INFO:teuthology.orchestra.run.trial148.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:40.137 INFO:teuthology.orchestra.run.trial148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:40.137 INFO:teuthology.orchestra.run.trial148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:40.137 INFO:teuthology.orchestra.run.trial148.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:40.137 INFO:teuthology.orchestra.run.trial148.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:40.137 INFO:teuthology.orchestra.run.trial148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:40.137 INFO:teuthology.orchestra.run.trial148.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:40.137 INFO:teuthology.orchestra.run.trial148.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:40.137 INFO:teuthology.orchestra.run.trial148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:40.696 INFO:teuthology.orchestra.run.trial148.stdout:Discarding blocks...Done. 2026-02-17T05:53:40.698 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial148.front.sepia.ceph.com -o noatime 2026-02-17T05:53:40.698 DEBUG:teuthology.orchestra.run.trial148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2026-02-17T05:53:40.765 DEBUG:teuthology.orchestra.run.trial148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2026-02-17T05:53:40.832 DEBUG:teuthology.orchestra.run.trial148:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2026-02-17T05:53:40.898 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:40.899 INFO:tasks.ceph:role: osd.10 2026-02-17T05:53:40.899 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial148.front.sepia.ceph.com 2026-02-17T05:53:40.899 DEBUG:teuthology.orchestra.run.trial148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-17T05:53:40.963 INFO:teuthology.orchestra.run.trial148.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:40.963 INFO:teuthology.orchestra.run.trial148.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:40.963 INFO:teuthology.orchestra.run.trial148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:40.963 INFO:teuthology.orchestra.run.trial148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:40.963 INFO:teuthology.orchestra.run.trial148.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:40.964 INFO:teuthology.orchestra.run.trial148.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:40.964 INFO:teuthology.orchestra.run.trial148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:40.964 INFO:teuthology.orchestra.run.trial148.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:40.964 INFO:teuthology.orchestra.run.trial148.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:40.964 INFO:teuthology.orchestra.run.trial148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:41.528 INFO:teuthology.orchestra.run.trial148.stdout:Discarding blocks...Done. 2026-02-17T05:53:41.531 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial148.front.sepia.ceph.com -o noatime 2026-02-17T05:53:41.531 DEBUG:teuthology.orchestra.run.trial148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2026-02-17T05:53:41.597 DEBUG:teuthology.orchestra.run.trial148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2026-02-17T05:53:41.664 DEBUG:teuthology.orchestra.run.trial148:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2026-02-17T05:53:41.730 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:41.730 INFO:tasks.ceph:role: osd.14 2026-02-17T05:53:41.731 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial148.front.sepia.ceph.com 2026-02-17T05:53:41.731 DEBUG:teuthology.orchestra.run.trial148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-17T05:53:41.795 INFO:teuthology.orchestra.run.trial148.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:41.795 INFO:teuthology.orchestra.run.trial148.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:41.795 INFO:teuthology.orchestra.run.trial148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:41.795 INFO:teuthology.orchestra.run.trial148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:41.796 INFO:teuthology.orchestra.run.trial148.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:41.796 INFO:teuthology.orchestra.run.trial148.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:41.796 INFO:teuthology.orchestra.run.trial148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:41.796 INFO:teuthology.orchestra.run.trial148.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:41.796 INFO:teuthology.orchestra.run.trial148.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:41.796 INFO:teuthology.orchestra.run.trial148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:42.353 INFO:teuthology.orchestra.run.trial148.stdout:Discarding blocks...Done. 2026-02-17T05:53:42.355 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial148.front.sepia.ceph.com -o noatime 2026-02-17T05:53:42.355 DEBUG:teuthology.orchestra.run.trial148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2026-02-17T05:53:42.437 DEBUG:teuthology.orchestra.run.trial148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2026-02-17T05:53:42.505 DEBUG:teuthology.orchestra.run.trial148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:42.585 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:42.584+0000 7f730e0d7780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-17T05:53:42.586 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:42.584+0000 7f730e0d7780 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-17T05:53:42.586 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:42.584+0000 7f730e0d7780 -1 bdev(0x55ca726cf800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:42.586 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:42.584+0000 7f730e0d7780 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-17T05:53:46.466 DEBUG:teuthology.orchestra.run.trial148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-17T05:53:46.532 DEBUG:teuthology.orchestra.run.trial148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:46.611 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:46.610+0000 7f1c71a15780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2026-02-17T05:53:46.612 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:46.610+0000 7f1c71a15780 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2026-02-17T05:53:46.612 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:46.611+0000 7f1c71a15780 -1 bdev(0x55f74ea11800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:46.612 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:46.611+0000 7f1c71a15780 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2026-02-17T05:53:50.497 DEBUG:teuthology.orchestra.run.trial148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2026-02-17T05:53:50.563 DEBUG:teuthology.orchestra.run.trial148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:50.642 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:50.641+0000 7f9d0b9ad780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2026-02-17T05:53:50.642 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:50.641+0000 7f9d0b9ad780 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2026-02-17T05:53:50.642 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:50.641+0000 7f9d0b9ad780 -1 bdev(0x55814b6e7800 /var/lib/ceph/osd/ceph-10/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:50.642 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:50.641+0000 7f9d0b9ad780 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2026-02-17T05:53:54.523 DEBUG:teuthology.orchestra.run.trial148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2026-02-17T05:53:54.590 DEBUG:teuthology.orchestra.run.trial148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:53:54.668 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:54.667+0000 7fa146e11780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2026-02-17T05:53:54.668 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:54.667+0000 7fa146e11780 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2026-02-17T05:53:54.668 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:54.667+0000 7fa146e11780 -1 bdev(0x560a4a0d9800 /var/lib/ceph/osd/ceph-14/block) open stat got: (1) Operation not permitted 2026-02-17T05:53:54.668 INFO:teuthology.orchestra.run.trial148.stderr:2026-02-17T05:53:54.667+0000 7fa146e11780 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2026-02-17T05:53:58.550 DEBUG:teuthology.orchestra.run.trial148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2026-02-17T05:53:58.615 DEBUG:teuthology.orchestra.run.trial164:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-17T05:53:58.640 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:58.640 INFO:tasks.ceph:role: osd.3 2026-02-17T05:53:58.640 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial164.front.sepia.ceph.com 2026-02-17T05:53:58.640 DEBUG:teuthology.orchestra.run.trial164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-17T05:53:58.704 INFO:teuthology.orchestra.run.trial164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:58.704 INFO:teuthology.orchestra.run.trial164.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:58.704 INFO:teuthology.orchestra.run.trial164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:58.704 INFO:teuthology.orchestra.run.trial164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:58.704 INFO:teuthology.orchestra.run.trial164.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:58.704 INFO:teuthology.orchestra.run.trial164.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:58.705 INFO:teuthology.orchestra.run.trial164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:58.705 INFO:teuthology.orchestra.run.trial164.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:58.705 INFO:teuthology.orchestra.run.trial164.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:58.705 INFO:teuthology.orchestra.run.trial164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:53:59.275 INFO:teuthology.orchestra.run.trial164.stdout:Discarding blocks...Done. 2026-02-17T05:53:59.277 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial164.front.sepia.ceph.com -o noatime 2026-02-17T05:53:59.277 DEBUG:teuthology.orchestra.run.trial164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2026-02-17T05:53:59.350 DEBUG:teuthology.orchestra.run.trial164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-17T05:53:59.418 DEBUG:teuthology.orchestra.run.trial164:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2026-02-17T05:53:59.484 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-17T05:53:59.484 INFO:tasks.ceph:role: osd.7 2026-02-17T05:53:59.484 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial164.front.sepia.ceph.com 2026-02-17T05:53:59.484 DEBUG:teuthology.orchestra.run.trial164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-17T05:53:59.549 INFO:teuthology.orchestra.run.trial164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:53:59.549 INFO:teuthology.orchestra.run.trial164.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:53:59.549 INFO:teuthology.orchestra.run.trial164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:53:59.549 INFO:teuthology.orchestra.run.trial164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:53:59.549 INFO:teuthology.orchestra.run.trial164.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:53:59.549 INFO:teuthology.orchestra.run.trial164.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:53:59.549 INFO:teuthology.orchestra.run.trial164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:53:59.549 INFO:teuthology.orchestra.run.trial164.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:53:59.549 INFO:teuthology.orchestra.run.trial164.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:53:59.549 INFO:teuthology.orchestra.run.trial164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:54:00.108 INFO:teuthology.orchestra.run.trial164.stdout:Discarding blocks...Done. 2026-02-17T05:54:00.110 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial164.front.sepia.ceph.com -o noatime 2026-02-17T05:54:00.111 DEBUG:teuthology.orchestra.run.trial164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2026-02-17T05:54:00.178 DEBUG:teuthology.orchestra.run.trial164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2026-02-17T05:54:00.246 DEBUG:teuthology.orchestra.run.trial164:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2026-02-17T05:54:00.310 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-17T05:54:00.310 INFO:tasks.ceph:role: osd.11 2026-02-17T05:54:00.310 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial164.front.sepia.ceph.com 2026-02-17T05:54:00.310 DEBUG:teuthology.orchestra.run.trial164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-17T05:54:00.373 INFO:teuthology.orchestra.run.trial164.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:54:00.374 INFO:teuthology.orchestra.run.trial164.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:54:00.374 INFO:teuthology.orchestra.run.trial164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:54:00.374 INFO:teuthology.orchestra.run.trial164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:54:00.374 INFO:teuthology.orchestra.run.trial164.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:54:00.374 INFO:teuthology.orchestra.run.trial164.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:54:00.374 INFO:teuthology.orchestra.run.trial164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:54:00.374 INFO:teuthology.orchestra.run.trial164.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:54:00.374 INFO:teuthology.orchestra.run.trial164.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:54:00.374 INFO:teuthology.orchestra.run.trial164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:54:00.934 INFO:teuthology.orchestra.run.trial164.stdout:Discarding blocks...Done. 2026-02-17T05:54:00.936 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial164.front.sepia.ceph.com -o noatime 2026-02-17T05:54:00.936 DEBUG:teuthology.orchestra.run.trial164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2026-02-17T05:54:01.004 DEBUG:teuthology.orchestra.run.trial164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2026-02-17T05:54:01.070 DEBUG:teuthology.orchestra.run.trial164:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2026-02-17T05:54:01.134 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-17T05:54:01.134 INFO:tasks.ceph:role: osd.15 2026-02-17T05:54:01.134 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial164.front.sepia.ceph.com 2026-02-17T05:54:01.134 DEBUG:teuthology.orchestra.run.trial164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-17T05:54:01.200 INFO:teuthology.orchestra.run.trial164.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:54:01.200 INFO:teuthology.orchestra.run.trial164.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:54:01.200 INFO:teuthology.orchestra.run.trial164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:54:01.200 INFO:teuthology.orchestra.run.trial164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:54:01.200 INFO:teuthology.orchestra.run.trial164.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:54:01.200 INFO:teuthology.orchestra.run.trial164.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:54:01.200 INFO:teuthology.orchestra.run.trial164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:54:01.200 INFO:teuthology.orchestra.run.trial164.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:54:01.200 INFO:teuthology.orchestra.run.trial164.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:54:01.201 INFO:teuthology.orchestra.run.trial164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:54:01.758 INFO:teuthology.orchestra.run.trial164.stdout:Discarding blocks...Done. 2026-02-17T05:54:01.759 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial164.front.sepia.ceph.com -o noatime 2026-02-17T05:54:01.759 DEBUG:teuthology.orchestra.run.trial164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2026-02-17T05:54:01.834 DEBUG:teuthology.orchestra.run.trial164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2026-02-17T05:54:01.902 DEBUG:teuthology.orchestra.run.trial164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:54:01.980 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:01.979+0000 7f7cf1c12780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-17T05:54:01.981 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:01.979+0000 7f7cf1c12780 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-17T05:54:01.981 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:01.979+0000 7f7cf1c12780 -1 bdev(0x55bd0bcc9800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-17T05:54:01.981 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:01.980+0000 7f7cf1c12780 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-17T05:54:05.864 DEBUG:teuthology.orchestra.run.trial164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-17T05:54:05.930 DEBUG:teuthology.orchestra.run.trial164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:54:06.006 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:06.004+0000 7f4a9f61c780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2026-02-17T05:54:06.006 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:06.004+0000 7f4a9f61c780 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2026-02-17T05:54:06.006 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:06.005+0000 7f4a9f61c780 -1 bdev(0x55f1870b3800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2026-02-17T05:54:06.006 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:06.005+0000 7f4a9f61c780 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2026-02-17T05:54:09.894 DEBUG:teuthology.orchestra.run.trial164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2026-02-17T05:54:09.959 DEBUG:teuthology.orchestra.run.trial164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:54:10.036 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:10.035+0000 7fcc3b7f7780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2026-02-17T05:54:10.037 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:10.035+0000 7fcc3b7f7780 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2026-02-17T05:54:10.037 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:10.035+0000 7fcc3b7f7780 -1 bdev(0x5583f7ce1800 /var/lib/ceph/osd/ceph-11/block) open stat got: (1) Operation not permitted 2026-02-17T05:54:10.037 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:10.035+0000 7fcc3b7f7780 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2026-02-17T05:54:13.915 DEBUG:teuthology.orchestra.run.trial164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2026-02-17T05:54:13.980 DEBUG:teuthology.orchestra.run.trial164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:54:14.057 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:14.054+0000 7f8bc2fb2780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2026-02-17T05:54:14.058 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:14.055+0000 7f8bc2fb2780 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2026-02-17T05:54:14.058 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:14.055+0000 7f8bc2fb2780 -1 bdev(0x5628644a9800 /var/lib/ceph/osd/ceph-15/block) open stat got: (1) Operation not permitted 2026-02-17T05:54:14.058 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:14.055+0000 7f8bc2fb2780 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2026-02-17T05:54:17.938 DEBUG:teuthology.orchestra.run.trial164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2026-02-17T05:54:18.005 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-17T05:54:18.006 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:54:18.006 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2026-02-17T05:54:18.033 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:54:18.033 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-17T05:54:18.098 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:54:18.098 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2026-02-17T05:54:18.162 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:54:18.162 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2026-02-17T05:54:18.226 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:54:18.226 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2026-02-17T05:54:18.290 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:54:18.290 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-17T05:54:18.315 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:54:18.315 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2026-02-17T05:54:18.379 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:54:18.379 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2026-02-17T05:54:18.444 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:54:18.444 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2026-02-17T05:54:18.508 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:54:18.508 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-17T05:54:18.533 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:54:18.533 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2026-02-17T05:54:18.599 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:54:18.599 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2026-02-17T05:54:18.663 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:54:18.663 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2026-02-17T05:54:18.726 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:54:18.726 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-17T05:54:18.752 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:54:18.752 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-17T05:54:18.815 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:54:18.816 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2026-02-17T05:54:18.880 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:54:18.880 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2026-02-17T05:54:18.944 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:54:18.944 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2026-02-17T05:54:19.008 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:54:19.009 DEBUG:teuthology.orchestra.run.trial164:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-17T05:54:19.064 INFO:tasks.ceph:Adding keys to all mons... 2026-02-17T05:54:19.065 DEBUG:teuthology.orchestra.run.trial016:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-17T05:54:19.067 DEBUG:teuthology.orchestra.run.trial128:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-17T05:54:19.070 DEBUG:teuthology.orchestra.run.trial148:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-17T05:54:19.088 INFO:teuthology.orchestra.run.trial016.stdout:[mgr.y] 2026-02-17T05:54:19.089 INFO:teuthology.orchestra.run.trial016.stdout: key = AQA8ApRp3nxcEhAACIVzKkQaFuOcq36+AOEHBg== 2026-02-17T05:54:19.089 INFO:teuthology.orchestra.run.trial016.stdout:[osd.0] 2026-02-17T05:54:19.089 INFO:teuthology.orchestra.run.trial016.stdout: key = AQA/ApRpYqAPLhAAmKpITQQZDVKBXKm7h9u0kA== 2026-02-17T05:54:19.089 INFO:teuthology.orchestra.run.trial016.stdout:[osd.4] 2026-02-17T05:54:19.089 INFO:teuthology.orchestra.run.trial016.stdout: key = AQBDApRpEiZZLxAALNDiS2ZxqL1a/w8WS8adng== 2026-02-17T05:54:19.089 INFO:teuthology.orchestra.run.trial016.stdout:[osd.8] 2026-02-17T05:54:19.089 INFO:teuthology.orchestra.run.trial016.stdout: key = AQBHApRp83zbMBAAcFEfeUmOayVkPCRjLF6d1A== 2026-02-17T05:54:19.089 INFO:teuthology.orchestra.run.trial016.stdout:[osd.12] 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout: key = AQBLApRpNhwLMhAAgYz8ZSV7Q5TLa7SSiOrpQg== 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout:[osd.1] 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout: key = AQBTApRpKNMBChAASSL1+MWQlLQRkLdxCmJ0FA== 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout:[osd.5] 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout: key = AQBXApRp6gNLCxAAuorij+mpckC7T/kfxdj5BQ== 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout:[osd.9] 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout: key = AQBbApRpKNTWDBAAe/NlYNOBGFYrr0K8+kbtzw== 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout:[osd.13] 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout: key = AQBfApRp3doeDhAA+3EF46OS6rIS1r1HPltqDA== 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout:[osd.2] 2026-02-17T05:54:19.090 INFO:teuthology.orchestra.run.trial016.stdout: key = AQBmApRphardIhAA1SEUKJiize3ixrJopod9kw== 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout:[osd.6] 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout: key = AQBqApRpBrRyJBAAxpbRhTbpEqUPEM2bXS8viw== 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout:[osd.10] 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout: key = AQBuApRpRTFBJhAA5No7YCllAhWcxR4QEo41HQ== 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout:[osd.14] 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout: key = AQByApRpJWTTJxAARsgtrEUaEyDzhdTwA3roRw== 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout:[mgr.x] 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout: key = AQA8ApRpG+byFBAAByNDOzlFqLVq7UGO9n5jdA== 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout:[osd.3] 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout: key = AQB5ApRpaghrOhAALRkUoFAJkTm+niSGgeIKbQ== 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout:[osd.7] 2026-02-17T05:54:19.091 INFO:teuthology.orchestra.run.trial016.stdout: key = AQB+ApRpDFlQABAAmwIWgEupUcequwQ69KCdeA== 2026-02-17T05:54:19.092 INFO:teuthology.orchestra.run.trial016.stdout:[osd.11] 2026-02-17T05:54:19.092 INFO:teuthology.orchestra.run.trial016.stdout: key = AQCCApRpEcQhAhAAIALrrdge00nzs2bkxcQuiQ== 2026-02-17T05:54:19.092 INFO:teuthology.orchestra.run.trial016.stdout:[osd.15] 2026-02-17T05:54:19.092 INFO:teuthology.orchestra.run.trial016.stdout: key = AQCGApRpg65bAxAAqjiI50UU6ebwSTw0lnMOWA== 2026-02-17T05:54:19.092 INFO:teuthology.orchestra.run.trial016.stdout:[client.0] 2026-02-17T05:54:19.092 INFO:teuthology.orchestra.run.trial016.stdout: key = AQA8ApRpPLgjFxAANx4TlWxMqGxYT6QMVm59+g== 2026-02-17T05:54:19.094 INFO:teuthology.orchestra.run.trial128.stdout:[mgr.y] 2026-02-17T05:54:19.094 INFO:teuthology.orchestra.run.trial128.stdout: key = AQA8ApRp3nxcEhAACIVzKkQaFuOcq36+AOEHBg== 2026-02-17T05:54:19.094 INFO:teuthology.orchestra.run.trial128.stdout:[osd.0] 2026-02-17T05:54:19.094 INFO:teuthology.orchestra.run.trial128.stdout: key = AQA/ApRpYqAPLhAAmKpITQQZDVKBXKm7h9u0kA== 2026-02-17T05:54:19.094 INFO:teuthology.orchestra.run.trial128.stdout:[osd.4] 2026-02-17T05:54:19.094 INFO:teuthology.orchestra.run.trial128.stdout: key = AQBDApRpEiZZLxAALNDiS2ZxqL1a/w8WS8adng== 2026-02-17T05:54:19.094 INFO:teuthology.orchestra.run.trial128.stdout:[osd.8] 2026-02-17T05:54:19.094 INFO:teuthology.orchestra.run.trial128.stdout: key = AQBHApRp83zbMBAAcFEfeUmOayVkPCRjLF6d1A== 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout:[osd.12] 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout: key = AQBLApRpNhwLMhAAgYz8ZSV7Q5TLa7SSiOrpQg== 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout:[osd.1] 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout: key = AQBTApRpKNMBChAASSL1+MWQlLQRkLdxCmJ0FA== 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout:[osd.5] 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout: key = AQBXApRp6gNLCxAAuorij+mpckC7T/kfxdj5BQ== 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout:[osd.9] 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout: key = AQBbApRpKNTWDBAAe/NlYNOBGFYrr0K8+kbtzw== 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout:[osd.13] 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout: key = AQBfApRp3doeDhAA+3EF46OS6rIS1r1HPltqDA== 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout:[osd.2] 2026-02-17T05:54:19.095 INFO:teuthology.orchestra.run.trial128.stdout: key = AQBmApRphardIhAA1SEUKJiize3ixrJopod9kw== 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout:[osd.6] 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout: key = AQBqApRpBrRyJBAAxpbRhTbpEqUPEM2bXS8viw== 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout:[osd.10] 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout: key = AQBuApRpRTFBJhAA5No7YCllAhWcxR4QEo41HQ== 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout:[osd.14] 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout: key = AQByApRpJWTTJxAARsgtrEUaEyDzhdTwA3roRw== 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout:[mgr.x] 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout: key = AQA8ApRpG+byFBAAByNDOzlFqLVq7UGO9n5jdA== 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout:[osd.3] 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout: key = AQB5ApRpaghrOhAALRkUoFAJkTm+niSGgeIKbQ== 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout:[osd.7] 2026-02-17T05:54:19.096 INFO:teuthology.orchestra.run.trial128.stdout: key = AQB+ApRpDFlQABAAmwIWgEupUcequwQ69KCdeA== 2026-02-17T05:54:19.097 INFO:teuthology.orchestra.run.trial128.stdout:[osd.11] 2026-02-17T05:54:19.097 INFO:teuthology.orchestra.run.trial128.stdout: key = AQCCApRpEcQhAhAAIALrrdge00nzs2bkxcQuiQ== 2026-02-17T05:54:19.097 INFO:teuthology.orchestra.run.trial128.stdout:[osd.15] 2026-02-17T05:54:19.097 INFO:teuthology.orchestra.run.trial128.stdout: key = AQCGApRpg65bAxAAqjiI50UU6ebwSTw0lnMOWA== 2026-02-17T05:54:19.097 INFO:teuthology.orchestra.run.trial128.stdout:[client.0] 2026-02-17T05:54:19.097 INFO:teuthology.orchestra.run.trial128.stdout: key = AQA8ApRpPLgjFxAANx4TlWxMqGxYT6QMVm59+g== 2026-02-17T05:54:19.098 INFO:teuthology.orchestra.run.trial148.stdout:[mgr.y] 2026-02-17T05:54:19.098 INFO:teuthology.orchestra.run.trial148.stdout: key = AQA8ApRp3nxcEhAACIVzKkQaFuOcq36+AOEHBg== 2026-02-17T05:54:19.098 INFO:teuthology.orchestra.run.trial148.stdout:[osd.0] 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout: key = AQA/ApRpYqAPLhAAmKpITQQZDVKBXKm7h9u0kA== 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout:[osd.4] 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout: key = AQBDApRpEiZZLxAALNDiS2ZxqL1a/w8WS8adng== 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout:[osd.8] 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout: key = AQBHApRp83zbMBAAcFEfeUmOayVkPCRjLF6d1A== 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout:[osd.12] 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout: key = AQBLApRpNhwLMhAAgYz8ZSV7Q5TLa7SSiOrpQg== 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout:[osd.1] 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout: key = AQBTApRpKNMBChAASSL1+MWQlLQRkLdxCmJ0FA== 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout:[osd.5] 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout: key = AQBXApRp6gNLCxAAuorij+mpckC7T/kfxdj5BQ== 2026-02-17T05:54:19.099 INFO:teuthology.orchestra.run.trial148.stdout:[osd.9] 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout: key = AQBbApRpKNTWDBAAe/NlYNOBGFYrr0K8+kbtzw== 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout:[osd.13] 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout: key = AQBfApRp3doeDhAA+3EF46OS6rIS1r1HPltqDA== 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout:[osd.2] 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout: key = AQBmApRphardIhAA1SEUKJiize3ixrJopod9kw== 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout:[osd.6] 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout: key = AQBqApRpBrRyJBAAxpbRhTbpEqUPEM2bXS8viw== 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout:[osd.10] 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout: key = AQBuApRpRTFBJhAA5No7YCllAhWcxR4QEo41HQ== 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout:[osd.14] 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout: key = AQByApRpJWTTJxAARsgtrEUaEyDzhdTwA3roRw== 2026-02-17T05:54:19.100 INFO:teuthology.orchestra.run.trial148.stdout:[mgr.x] 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout: key = AQA8ApRpG+byFBAAByNDOzlFqLVq7UGO9n5jdA== 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout:[osd.3] 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout: key = AQB5ApRpaghrOhAALRkUoFAJkTm+niSGgeIKbQ== 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout:[osd.7] 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout: key = AQB+ApRpDFlQABAAmwIWgEupUcequwQ69KCdeA== 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout:[osd.11] 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout: key = AQCCApRpEcQhAhAAIALrrdge00nzs2bkxcQuiQ== 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout:[osd.15] 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout: key = AQCGApRpg65bAxAAqjiI50UU6ebwSTw0lnMOWA== 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout:[client.0] 2026-02-17T05:54:19.101 INFO:teuthology.orchestra.run.trial148.stdout: key = AQA8ApRpPLgjFxAANx4TlWxMqGxYT6QMVm59+g== 2026-02-17T05:54:19.102 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-17T05:54:19.137 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-17T05:54:19.140 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-17T05:54:19.189 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.219 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.223 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.271 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.300 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.303 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.349 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.379 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.383 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.429 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.459 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.462 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.507 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.538 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.541 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.588 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.616 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.620 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.670 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.696 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.700 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.749 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.776 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.779 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.828 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.858 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.861 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.907 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.937 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.939 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:19.986 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.017 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.019 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.066 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.096 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.097 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.143 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-17T05:54:20.177 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-17T05:54:20.179 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-17T05:54:20.223 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.256 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.259 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.302 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.335 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.339 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.383 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.415 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.418 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.462 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.496 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.499 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:54:20.541 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-17T05:54:20.576 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-17T05:54:20.580 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-17T05:54:20.621 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-17T05:54:20.622 DEBUG:teuthology.orchestra.run.trial016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-17T05:54:20.679 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-17T05:54:20.766 DEBUG:teuthology.orchestra.run.trial016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-17T05:54:20.792 DEBUG:teuthology.orchestra.run.trial128:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2026-02-17T05:54:20.817 DEBUG:teuthology.orchestra.run.trial128:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-17T05:54:20.904 DEBUG:teuthology.orchestra.run.trial128:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2026-02-17T05:54:20.929 DEBUG:teuthology.orchestra.run.trial148:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2026-02-17T05:54:20.954 DEBUG:teuthology.orchestra.run.trial148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-17T05:54:21.040 DEBUG:teuthology.orchestra.run.trial148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2026-02-17T05:54:21.063 DEBUG:teuthology.orchestra.run.trial016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:54:21.065 DEBUG:teuthology.orchestra.run.trial128:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:54:21.067 DEBUG:teuthology.orchestra.run.trial148:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:54:21.119 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-17T05:54:21.119 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-17T05:54:21.119 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T05:54:21.119 DEBUG:teuthology.orchestra.run.trial016:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2026-02-17T05:54:21.123 INFO:tasks.ceph.mon.a:Started 2026-02-17T05:54:21.123 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-17T05:54:21.123 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T05:54:21.123 DEBUG:teuthology.orchestra.run.trial128:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2026-02-17T05:54:21.124 INFO:tasks.ceph.mon.b:Started 2026-02-17T05:54:21.125 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-17T05:54:21.125 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-17T05:54:21.125 DEBUG:teuthology.orchestra.run.trial148:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2026-02-17T05:54:21.161 INFO:tasks.ceph.mon.c:Started 2026-02-17T05:54:21.161 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-17T05:54:21.161 INFO:tasks.ceph.mgr.y:Restarting daemon 2026-02-17T05:54:21.161 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2026-02-17T05:54:21.164 INFO:tasks.ceph.mgr.y:Started 2026-02-17T05:54:21.164 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-17T05:54:21.164 DEBUG:teuthology.orchestra.run.trial164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2026-02-17T05:54:21.167 INFO:tasks.ceph.mgr.x:Started 2026-02-17T05:54:21.167 INFO:tasks.ceph:Setting crush tunables to default 2026-02-17T05:54:21.168 DEBUG:teuthology.orchestra.run.trial016:> sudo ceph --cluster ceph osd crush tunables default 2026-02-17T05:54:21.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.236+0000 7f36e01f7640 10 monclient: get_monmap_and_config 2026-02-17T05:54:21.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.236+0000 7f36e01f7640 10 monclient: build_initial_monmap 2026-02-17T05:54:21.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.236+0000 7f36e01f7640 10 monclient: monmap: 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:54:21.237541+0000 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:54:21.237541+0000 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:21.238 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:21.239 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:21.239 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:21.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 1 Processor -- start 2026-02-17T05:54:21.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 1 -- start start 2026-02-17T05:54:21.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 10 monclient: init 2026-02-17T05:54:21.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 10 monclient: _add_conns ranks=[0,2,3,4,1,5] 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f36d8155ce0 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f36d81560c0 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 0x7f36d815f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 10 monclient(hunting): picked mon.noname-a con 0x7f36d806e430 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d80705b0 con 0x7f36d8155ce0 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d806cbb0 con 0x7f36d81560c0 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f36d806ce90 con 0x7f36d806e430 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:21.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36e01f7640 10 monclient(hunting): authenticate will time out at 913.806627s 2026-02-17T05:54:21.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36ddf6c640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36d8155ce0 legacy=0x7f36d806fea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:21.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36de76d640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 msgr2=0x7f36d815f290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:21.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.237+0000 7f36de76d640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 0x7f36d815f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:21.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.242+0000 7f36dd76b640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36d81560c0 legacy=0x7f36d80c9f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:21.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.438+0000 7f36de76d640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 msgr2=0x7f36d815f290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:21.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.438+0000 7f36ddf6c640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36d8155ce0 legacy=0x7f36d806fea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:21.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.438+0000 7f36de76d640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 0x7f36d815f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:21.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.443+0000 7f36dd76b640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36d81560c0 legacy=0x7f36d80c9f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:21.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.839+0000 7f36de76d640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 msgr2=0x7f36d815f290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:21.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.839+0000 7f36ddf6c640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36d8155ce0 legacy=0x7f36d806fea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:21.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.839+0000 7f36de76d640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 0x7f36d815f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:21.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:21.844+0000 7f36dd76b640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36d81560c0 legacy=0x7f36d80c9f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:22.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:22.640+0000 7f36ddf6c640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36d8155ce0 legacy=0x7f36d806fea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:22.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:22.640+0000 7f36de76d640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 msgr2=0x7f36d815f290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:22.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:22.640+0000 7f36de76d640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 0x7f36d815f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:22.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:22.645+0000 7f36dd76b640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36d81560c0 legacy=0x7f36d80c9f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 10 monclient(hunting): tick 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 msgr2=0x7f36d815f290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36d806e430 0x7f36d815f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36d81560c0 legacy=0x7f36d80c9f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36d8155ce0 legacy=0x7f36d806fea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 10 monclient: _add_conns ranks=[1,4,5] 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f36bc0037d0 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 0x7f36bc009910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-b con 0x7f36bc007450 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 0x7f36bc00c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-c con 0x7f36bc009ff0 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d806d170 con 0x7f36bc0037d0 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f36d806ce90 con 0x7f36bc007450 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f36bc00dc50 con 0x7f36bc009ff0 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36c77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36dd76b640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 msgr2=0x7f36bc009910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36de76d640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 msgr2=0x7f36bc00c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36dd76b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 0x7f36bc009910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36de76d640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 0x7f36bc00c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.237+0000 7f36ddf6c640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc0037d0 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:24.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.438+0000 7f36dd76b640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 msgr2=0x7f36bc009910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:24.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.438+0000 7f36de76d640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 msgr2=0x7f36bc00c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:24.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.438+0000 7f36dd76b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 0x7f36bc009910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:24.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.438+0000 7f36de76d640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 0x7f36bc00c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:24.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.438+0000 7f36ddf6c640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc0037d0 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:24.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.839+0000 7f36de76d640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 msgr2=0x7f36bc00c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:24.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.839+0000 7f36dd76b640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 msgr2=0x7f36bc009910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:24.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.839+0000 7f36ddf6c640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc0037d0 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:24.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.839+0000 7f36de76d640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 0x7f36bc00c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:24.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:24.839+0000 7f36dd76b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 0x7f36bc009910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:25.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:25.640+0000 7f36de76d640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 msgr2=0x7f36bc00c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:25.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:25.640+0000 7f36dd76b640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 msgr2=0x7f36bc009910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:25.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:25.640+0000 7f36de76d640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 0x7f36bc00c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:25.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:25.640+0000 7f36dd76b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 0x7f36bc009910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:25.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:25.640+0000 7f36ddf6c640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc0037d0 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:27.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 10 monclient(hunting): tick 2026-02-17T05:54:27.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:54:27.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:54:27.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:54:27.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 msgr2=0x7f36bc00c4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:27.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc009ff0 0x7f36bc00c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:27.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 msgr2=0x7f36bc009910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:27.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc007450 0x7f36bc009910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:27.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc0037d0 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:27.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36de76d640 1 -- reap_dead start 2026-02-17T05:54:27.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 10 monclient: _add_conns ranks=[0,3,1,4,5,2] 2026-02-17T05:54:27.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f36bc00cc50 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:27.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 0x7f36bc009910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:27.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-a con 0x7f36bc00d7e0 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:27.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f36bc00c580 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:27.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d80705b0 con 0x7f36bc00cc50 2026-02-17T05:54:27.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d806d170 con 0x7f36bc00c580 2026-02-17T05:54:27.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:27.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f36d806ce90 con 0x7f36bc00d7e0 2026-02-17T05:54:27.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36c77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:27.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36ddf6c640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36bc00cc50 legacy=0x7f36bc00d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:27.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36dd76b640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 msgr2=0x7f36bc009910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:27.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36dd76b640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 0x7f36bc009910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:27.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.237+0000 7f36de76d640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc00c580 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:27.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.438+0000 7f36ddf6c640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36bc00cc50 legacy=0x7f36bc00d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:27.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.438+0000 7f36dd76b640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 msgr2=0x7f36bc009910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:27.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.438+0000 7f36dd76b640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 0x7f36bc009910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:27.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.438+0000 7f36de76d640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc00c580 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:27.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.839+0000 7f36dd76b640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 msgr2=0x7f36bc009910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:27.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.839+0000 7f36ddf6c640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36bc00cc50 legacy=0x7f36bc00d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:27.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.839+0000 7f36dd76b640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 0x7f36bc009910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:27.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:27.839+0000 7f36de76d640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc00c580 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:28.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:28.640+0000 7f36dd76b640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 msgr2=0x7f36bc009910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:28.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:28.640+0000 7f36ddf6c640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36bc00cc50 legacy=0x7f36bc00d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:28.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:28.640+0000 7f36dd76b640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 0x7f36bc009910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:28.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:28.640+0000 7f36de76d640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc00c580 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:30.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 10 monclient(hunting): tick 2026-02-17T05:54:30.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:54:30.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:54:30.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:54:30.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 msgr2=0x7f36bc009910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:30.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc00d7e0 0x7f36bc009910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:30.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc00c580 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:30.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36bc00cc50 legacy=0x7f36bc00d030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:30.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 10 monclient: _add_conns ranks=[2,5,4] 2026-02-17T05:54:30.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f36bc00f180 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:30.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 0x7f36bc0151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:30.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-c con 0x7f36bc010d20 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:30.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc017d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:30.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-b con 0x7f36bc0158b0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:30.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d806cbb0 con 0x7f36bc00f180 2026-02-17T05:54:30.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:30.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f36d806ce90 con 0x7f36bc0158b0 2026-02-17T05:54:30.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:30.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f36bc00dc50 con 0x7f36bc010d20 2026-02-17T05:54:30.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.237+0000 7f36c77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:30.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.238+0000 7f36dd76b640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 msgr2=0x7f36bc0151e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:30.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.238+0000 7f36dd76b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 0x7f36bc0151e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:30.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.238+0000 7f36de76d640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 msgr2=0x7f36bc017d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:30.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.238+0000 7f36de76d640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc017d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:30.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.238+0000 7f36ddf6c640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:30.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.439+0000 7f36ddf6c640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:30.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.439+0000 7f36de76d640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 msgr2=0x7f36bc017d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:30.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.439+0000 7f36de76d640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc017d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:30.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.439+0000 7f36dd76b640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 msgr2=0x7f36bc0151e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:30.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.439+0000 7f36dd76b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 0x7f36bc0151e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:30.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.840+0000 7f36ddf6c640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:30.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.840+0000 7f36de76d640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 msgr2=0x7f36bc017d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:30.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.840+0000 7f36de76d640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc017d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:30.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.840+0000 7f36dd76b640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 msgr2=0x7f36bc0151e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:30.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:30.840+0000 7f36dd76b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 0x7f36bc0151e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:31.561 INFO:tasks.ceph.mon.a.trial016.stderr:--00:00:00:10.389 44427-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/47/c8811f81c8519af972c31a943fbe60655b3649.debug 2026-02-17T05:54:31.573 INFO:tasks.ceph.mon.b.trial128.stderr:--00:00:00:10.400 44359-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/47/c8811f81c8519af972c31a943fbe60655b3649.debug 2026-02-17T05:54:31.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:31.641+0000 7f36dd76b640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 msgr2=0x7f36bc0151e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:31.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:31.641+0000 7f36de76d640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 msgr2=0x7f36bc017d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:31.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:31.641+0000 7f36ddf6c640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:31.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:31.641+0000 7f36dd76b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 0x7f36bc0151e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:31.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:31.641+0000 7f36de76d640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc017d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:31.738 INFO:tasks.ceph.mon.c.trial148.stderr:--00:00:00:10.527 44463-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/47/c8811f81c8519af972c31a943fbe60655b3649.debug 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.237+0000 7f36c77fe640 10 monclient(hunting): tick 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.237+0000 7f36c77fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.237+0000 7f36c77fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.237+0000 7f36c77fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.237+0000 7f36c77fe640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 msgr2=0x7f36bc0151e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.237+0000 7f36c77fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc010d20 0x7f36bc0151e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 msgr2=0x7f36bc017d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc017d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36de76d640 1 -- reap_dead start 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 10 monclient: _add_conns ranks=[4,2,1,3,0,5] 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc00cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-b con 0x7f36bc0158b0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f36bc00f180 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f36bc00c580 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d80705b0 con 0x7f36bc00c580 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d806cbb0 con 0x7f36bc00f180 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f36d806ce90 con 0x7f36bc0158b0 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36c77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36ddf6c640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 msgr2=0x7f36bc00cc30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36ddf6c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc00cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36de76d640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc00c580 legacy=0x7f36bc0183f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.238+0000 7f36dd76b640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:33.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.439+0000 7f36de76d640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc00c580 legacy=0x7f36bc0183f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:33.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.439+0000 7f36ddf6c640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 msgr2=0x7f36bc00cc30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:33.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.439+0000 7f36ddf6c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc00cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:33.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.439+0000 7f36dd76b640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:33.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.840+0000 7f36de76d640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc00c580 legacy=0x7f36bc0183f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:33.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.840+0000 7f36ddf6c640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 msgr2=0x7f36bc00cc30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:33.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.840+0000 7f36ddf6c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc00cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:33.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:33.840+0000 7f36dd76b640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:34.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:34.641+0000 7f36de76d640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc00c580 legacy=0x7f36bc0183f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.128:6789/0 2026-02-17T05:54:34.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:34.641+0000 7f36ddf6c640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 msgr2=0x7f36bc00cc30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.128:3300/0 2026-02-17T05:54:34.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:34.641+0000 7f36ddf6c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc00cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:34.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:34.641+0000 7f36dd76b640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.148:6789/0 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 10 monclient(hunting): tick 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 1 -- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 msgr2=0x7f36bc00cc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc0158b0 0x7f36bc00cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 1 -- >> v1:10.20.193.148:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 1 -- >> v1:10.20.193.128:6789/0 conn(0x7f36bc00c580 legacy=0x7f36bc0183f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 10 monclient: _add_conns ranks=[3,5,0] 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc00d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-a con 0x7f36bc009910 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 0x7f36bc018b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-c con 0x7f36bc017ca0 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f36bc019040 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d806d170 con 0x7f36bc019040 2026-02-17T05:54:36.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:36.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f36d806ce90 con 0x7f36bc009910 2026-02-17T05:54:36.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:36.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f36bc00dc50 con 0x7f36bc017ca0 2026-02-17T05:54:36.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36c77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:36.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36ddf6c640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc00d170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:36.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36de76d640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36bc019040 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:36.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36ddf6c640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc00d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:36.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36dd76b640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 msgr2=0x7f36bc018b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:36.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.238+0000 7f36dd76b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 0x7f36bc018b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.439+0000 7f36de76d640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36bc019040 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.439+0000 7f36ddf6c640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc00d170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.439+0000 7f36ddf6c640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc00d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.439+0000 7f36dd76b640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 msgr2=0x7f36bc018b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.439+0000 7f36dd76b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 0x7f36bc018b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:36.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.840+0000 7f36de76d640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36bc019040 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:36.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.840+0000 7f36ddf6c640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc00d170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:36.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.840+0000 7f36ddf6c640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc00d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:36.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.840+0000 7f36dd76b640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 msgr2=0x7f36bc018b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:36.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:36.840+0000 7f36dd76b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 0x7f36bc018b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:37.166 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:00:15.992 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:54:37.169 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:00:15.997 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:54:37.316 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:00:16.142 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:54:37.319 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:00:16.147 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:54:37.356 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:00:16.145 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:54:37.507 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:00:16.296 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:54:37.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:37.641+0000 7f36de76d640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36bc019040 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.16:6789/0 2026-02-17T05:54:37.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:37.641+0000 7f36ddf6c640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc00d170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.16:3300/0 2026-02-17T05:54:37.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:37.641+0000 7f36ddf6c640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc00d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:37.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:37.641+0000 7f36dd76b640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 msgr2=0x7f36bc018b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.148:3300/0 2026-02-17T05:54:37.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:37.641+0000 7f36dd76b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 0x7f36bc018b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:39.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient(hunting): tick 2026-02-17T05:54:39.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:54:39.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:54:39.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:54:39.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 -- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 msgr2=0x7f36bc018b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:39.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc017ca0 0x7f36bc018b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:39.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 -- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc00d170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:39.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc00d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:39.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 -- >> v1:10.20.193.16:6789/0 conn(0x7f36bc019040 legacy=0x7f36bc005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:39.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36de76d640 1 -- reap_dead start 2026-02-17T05:54:39.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36de76d640 1 -- reap_dead start 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient: _add_conns ranks=[5,4,3,1,0,2] 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-c con 0x7f36bc00c580 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-b con 0x7f36bc00cc10 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-a con 0x7f36bc009910 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f36d806ce90 con 0x7f36bc009910 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f36bc00dc50 con 0x7f36bc00cc10 2026-02-17T05:54:39.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:39.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f36bc017da0 con 0x7f36bc00c580 2026-02-17T05:54:39.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.238+0000 7f36c77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:39.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.241+0000 7f36de76d640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:39.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:39.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:37756/0 (socket says 10.20.193.16:37756) 2026-02-17T05:54:39.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 learned_addr learned my addr 10.20.193.16:0/739958922 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request con 0x7f36bc00c580 auth_method 0 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36de76d640 10 monclient(hunting): get_auth_request con 0x7f36bc009910 auth_method 0 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36de76d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36de76d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36de76d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:39.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:39.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:39.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.242+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:39.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.243+0000 7f36dd76b640 10 monclient(hunting): get_auth_request con 0x7f36bc00cc10 auth_method 0 2026-02-17T05:54:39.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.243+0000 7f36dd76b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:39.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.243+0000 7f36dd76b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:39.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.243+0000 7f36dd76b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:39.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.243+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 msgr2=0x7f36bc00d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:39.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.243+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 msgr2=0x7f36bc00d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:39.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.243+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:39.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.243+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:39.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.244+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc017860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:54:39.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.244+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc017860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:39.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.244+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:39.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.244+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:39.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.443+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:39.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.443+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request con 0x7f36bc00c580 auth_method 0 2026-02-17T05:54:39.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.443+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:39.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.443+0000 7f36ddf6c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:39.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.443+0000 7f36ddf6c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:39.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.444+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:39.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.444+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:39.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.444+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:39.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.444+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:39.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.444+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:39.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.444+0000 7f36dd76b640 10 monclient(hunting): get_auth_request con 0x7f36bc00cc10 auth_method 0 2026-02-17T05:54:39.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.444+0000 7f36dd76b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:39.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.444+0000 7f36dd76b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:39.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.444+0000 7f36dd76b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:39.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 msgr2=0x7f36bc00d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:39.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 msgr2=0x7f36bc00d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:39.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:39.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:39.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:39.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36de76d640 10 monclient(hunting): get_auth_request con 0x7f36bc009910 auth_method 0 2026-02-17T05:54:39.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36de76d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:39.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36de76d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:39.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36de76d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:39.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc017860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:39.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc017860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:39.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:39.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.445+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:39.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.845+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:39.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.845+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request con 0x7f36bc00c580 auth_method 0 2026-02-17T05:54:39.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.845+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:39.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.845+0000 7f36ddf6c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:39.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.845+0000 7f36ddf6c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:39.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.845+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:39.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.845+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:39.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.845+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:39.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.845+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:39.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.846+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:39.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.846+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:39.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.846+0000 7f36de76d640 10 monclient(hunting): get_auth_request con 0x7f36bc009910 auth_method 0 2026-02-17T05:54:39.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.846+0000 7f36de76d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:39.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.846+0000 7f36de76d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:39.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.846+0000 7f36de76d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:39.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.846+0000 7f36dd76b640 10 monclient(hunting): get_auth_request con 0x7f36bc00cc10 auth_method 0 2026-02-17T05:54:39.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.846+0000 7f36dd76b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:39.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.846+0000 7f36dd76b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:39.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.846+0000 7f36dd76b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:39.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.847+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 msgr2=0x7f36bc00d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:39.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.847+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 msgr2=0x7f36bc00d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:39.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.847+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:39.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.847+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:39.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.851+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc017860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:39.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.851+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc017860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:39.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.851+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:39.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:39.851+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:40.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.647+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:40.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.647+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request con 0x7f36bc00c580 auth_method 0 2026-02-17T05:54:40.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.647+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:40.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.647+0000 7f36ddf6c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:40.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.647+0000 7f36ddf6c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:40.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.647+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:40.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.647+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:40.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.647+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:40.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.647+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:40.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.648+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:40.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.648+0000 7f36dd76b640 10 monclient(hunting): get_auth_request con 0x7f36bc00cc10 auth_method 0 2026-02-17T05:54:40.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.648+0000 7f36dd76b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:40.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.648+0000 7f36dd76b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:40.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.648+0000 7f36dd76b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:40.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.649+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 msgr2=0x7f36bc00d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:40.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.649+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 msgr2=0x7f36bc00d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:40.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.649+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:40.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.649+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:40.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.652+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:40.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.652+0000 7f36de76d640 10 monclient(hunting): get_auth_request con 0x7f36bc009910 auth_method 0 2026-02-17T05:54:40.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.652+0000 7f36de76d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:40.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.652+0000 7f36de76d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:40.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.652+0000 7f36de76d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:40.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.652+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc017860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:40.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.652+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc017860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:40.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.652+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:40.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:40.653+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:42.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 10 monclient(hunting): tick 2026-02-17T05:54:42.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:54:42.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:54:42.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:54:42.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:42.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc0151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:42.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 msgr2=0x7f36bc00d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:42.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00cc10 0x7f36bc00d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:42.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 msgr2=0x7f36bc017860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:42.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009910 0x7f36bc017860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:42.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:54:42.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f36bc00e020 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:42.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f36bc00eb10 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:42.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f36bc00f180 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:42.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d80705b0 con 0x7f36bc00e020 2026-02-17T05:54:42.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d806d170 con 0x7f36bc00f180 2026-02-17T05:54:42.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d806cbb0 con 0x7f36bc00eb10 2026-02-17T05:54:42.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:42.239+0000 7f36c77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:45.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 10 monclient(hunting): tick 2026-02-17T05:54:45.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:54:45.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:54:45.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:54:45.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 >> v1:10.20.193.148:6789/0 conn(0x7f36bc00eb10 legacy=0x7f36bc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:45.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 >> v1:10.20.193.128:6789/0 conn(0x7f36bc00f180 legacy=0x7f36bc00f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:45.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 >> v1:10.20.193.16:6789/0 conn(0x7f36bc00e020 legacy=0x7f36bc00e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:45.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 reap_dead start 2026-02-17T05:54:45.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 10 monclient: _add_conns ranks=[4,5,0,2,1,3] 2026-02-17T05:54:45.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:45.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-b con 0x7f36bc00f180 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-c con 0x7f36bc00c580 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f36bc00e020 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36bc00eef0 con 0x7f36bc00e020 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f36d806ce90 con 0x7f36bc00f180 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f36bc00dc50 con 0x7f36bc00c580 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36c77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:45.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request con 0x7f36bc00f180 auth_method 0 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36ddf6c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.239+0000 7f36ddf6c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36dd76b640 10 monclient(hunting): get_auth_request con 0x7f36bc00c580 auth_method 0 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36dd76b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36dd76b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36dd76b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:45.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:45.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc00cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:45.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc00cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:45.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:45.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.240+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:45.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.441+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:45.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.441+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:45.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.441+0000 7f36dd76b640 10 monclient(hunting): get_auth_request con 0x7f36bc00c580 auth_method 0 2026-02-17T05:54:45.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.441+0000 7f36dd76b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:45.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.441+0000 7f36dd76b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:45.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.441+0000 7f36dd76b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:45.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.441+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request con 0x7f36bc00f180 auth_method 0 2026-02-17T05:54:45.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.441+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:45.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.441+0000 7f36ddf6c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:45.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.441+0000 7f36ddf6c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:45.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.442+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:45.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.442+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:45.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.442+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:45.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.442+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:45.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.442+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc00cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:45.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.442+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc00cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:45.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.442+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:45.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.442+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:45.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.843+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:45.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.843+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:45.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.843+0000 7f36dd76b640 10 monclient(hunting): get_auth_request con 0x7f36bc00c580 auth_method 0 2026-02-17T05:54:45.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.843+0000 7f36dd76b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:45.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.843+0000 7f36dd76b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:45.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.843+0000 7f36dd76b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:45.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.843+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request con 0x7f36bc00f180 auth_method 0 2026-02-17T05:54:45.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.843+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:45.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.843+0000 7f36ddf6c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:45.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.843+0000 7f36ddf6c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:45.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.844+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc00cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:45.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.844+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc00cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:45.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.844+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:45.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.844+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:45.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.844+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:45.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.844+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:45.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.844+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:45.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:45.844+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:46.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.645+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:46.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.645+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:46.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.645+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request con 0x7f36bc00f180 auth_method 0 2026-02-17T05:54:46.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.645+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:46.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.645+0000 7f36ddf6c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:46.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.645+0000 7f36ddf6c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:46.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.645+0000 7f36dd76b640 10 monclient(hunting): get_auth_request con 0x7f36bc00c580 auth_method 0 2026-02-17T05:54:46.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.645+0000 7f36dd76b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:46.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.645+0000 7f36dd76b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:46.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.645+0000 7f36dd76b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:46.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.646+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc00cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:46.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.646+0000 7f36dd76b640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc00cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:46.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.646+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:46.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.646+0000 7f36dd76b640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:46.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.646+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:46.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.646+0000 7f36ddf6c640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:46.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.646+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:46.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:46.646+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:48.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 10 monclient(hunting): tick 2026-02-17T05:54:48.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:54:48.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:54:48.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:54:48.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 msgr2=0x7f36bc00cff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:48.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.148:3300/0 conn(0x7f36bc00c580 0x7f36bc00cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 msgr2=0x7f36bc0151b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.128:3300/0 conn(0x7f36bc00f180 0x7f36bc0151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 >> v1:10.20.193.16:6789/0 conn(0x7f36bc00e020 legacy=0x7f36bc00e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 10 monclient: _add_conns ranks=[1,2,3] 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f36bc00cc10 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f36bc00eb10 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 10 monclient(hunting): picked mon.noname-a con 0x7f36bc009890 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36bc0177c0 con 0x7f36bc00cc10 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36bc0102e0 con 0x7f36bc00eb10 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 1 -- 10.20.193.16:0/739958922 --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f36d806ce90 con 0x7f36bc009890 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36c77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.239+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.240+0000 7f36de76d640 10 monclient(hunting): get_auth_request con 0x7f36bc009890 auth_method 0 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.240+0000 7f36de76d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.240+0000 7f36de76d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.240+0000 7f36de76d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.240+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 msgr2=0x7f36bc00fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.240+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 msgr2=0x7f36bc00fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.240+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:48.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.240+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.441+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.441+0000 7f36de76d640 10 monclient(hunting): get_auth_request con 0x7f36bc009890 auth_method 0 2026-02-17T05:54:48.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.441+0000 7f36de76d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.441+0000 7f36de76d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.441+0000 7f36de76d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:48.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.441+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 msgr2=0x7f36bc00fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:54:48.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.441+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 msgr2=0x7f36bc00fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:48.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.441+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:48.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.441+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:48.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.842+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.842+0000 7f36de76d640 10 monclient(hunting): get_auth_request con 0x7f36bc009890 auth_method 0 2026-02-17T05:54:48.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.842+0000 7f36de76d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.842+0000 7f36de76d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.842+0000 7f36de76d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:48.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.842+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 msgr2=0x7f36bc00fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:54:48.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.842+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 msgr2=0x7f36bc00fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:48.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.842+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:48.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:48.842+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:49.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.559+0000 7f36dcf6a640 1 -- 10.20.193.16:0/739958922 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2785598967 0 0) 0x7f36bc0102e0 con 0x7f36bc00eb10 2026-02-17T05:54:49.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.559+0000 7f36dcf6a640 10 monclient(hunting): _negotiate 2026-02-17T05:54:49.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.559+0000 7f36dcf6a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:49.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.559+0000 7f36dcf6a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:49.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.559+0000 7f36dcf6a640 10 monclient(hunting): my global_id is 4103 2026-02-17T05:54:49.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.559+0000 7f36dcf6a640 1 -- 10.20.193.16:0/739958922 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36c0003620 con 0x7f36bc00eb10 2026-02-17T05:54:49.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.639+0000 7f36dcf6a640 1 -- 10.20.193.16:0/739958922 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3923443446 0 0) 0x7f36bc0177c0 con 0x7f36bc00cc10 2026-02-17T05:54:49.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.639+0000 7f36dcf6a640 10 monclient(hunting): _negotiate 2026-02-17T05:54:49.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.639+0000 7f36dcf6a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:49.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.639+0000 7f36dcf6a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:49.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.639+0000 7f36dcf6a640 10 monclient(hunting): my global_id is 4099 2026-02-17T05:54:49.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.639+0000 7f36dcf6a640 1 -- 10.20.193.16:0/739958922 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36bc0102e0 con 0x7f36bc00cc10 2026-02-17T05:54:49.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.645+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> v2:10.20.193.16:3300/0 conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:49.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.646+0000 7f36de76d640 10 monclient(hunting): get_auth_request con 0x7f36bc009890 auth_method 0 2026-02-17T05:54:49.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.646+0000 7f36de76d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:49.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.646+0000 7f36de76d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:49.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.646+0000 7f36de76d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:49.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.648+0000 7f36de76d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:49.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.648+0000 7f36de76d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:49.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.648+0000 7f36de76d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.649+0000 7f36de76d640 10 monclient(hunting): handle_auth_done global_id 4107 payload 274 2026-02-17T05:54:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.649+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v1:10.20.193.148:6789/0 conn(0x7f36bc00eb10 legacy=0x7f36bc006bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.649+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 >> v1:10.20.193.128:6789/0 conn(0x7f36bc00cc10 legacy=0x7f36bc00f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.649+0000 7f36de76d640 10 monclient: _finish_hunting 0 2026-02-17T05:54:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.649+0000 7f36de76d640 1 monclient: found mon.noname-a 2026-02-17T05:54:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.649+0000 7f36de76d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.649+0000 7f36de76d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.649+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc0104b0 con 0x7f36bc009890 2026-02-17T05:54:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.650+0000 7f36de76d640 1 -- 10.20.193.16:0/739958922 reap_dead start 2026-02-17T05:54:49.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.674+0000 7f36de76d640 1 --2- 10.20.193.16:0/739958922 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f36bc009890 0x7f36bc00fda0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f36d8073fa0 tx=0x7f36d4030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fed0eed092a3b8e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 1 -- 10.20.193.16:0/739958922 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4035480 con 0x7f36bc009890 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 10 monclient: dump: 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 10 monclient: _finish_auth 0 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 10 monclient: _check_auth_tickets 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 1 -- 10.20.193.16:0/739958922 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d4035640 con 0x7f36bc009890 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36e01f7640 5 monclient: authenticate success, global_id 4107 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36e01f7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36e01f7640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 1 -- 10.20.193.16:0/739958922 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4035a30 con 0x7f36bc009890 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36e01f7640 4 set_mon_vals no callback set 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.685+0000 7f36dcf6a640 10 monclient: dump: 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:54:49.686 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 10 monclient: shutdown 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 1 -- 10.20.193.16:0/739958922 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f36bc009890 msgr2=0x7f36bc00fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 1 --2- 10.20.193.16:0/739958922 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f36bc009890 0x7f36bc00fda0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f36d8073fa0 tx=0x7f36d4030ad0 comp rx=0 tx=0).stop 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 1 -- 10.20.193.16:0/739958922 shutdown_connections 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 1 --2- 10.20.193.16:0/739958922 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f36bc009890 0x7f36bc00fda0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 1 -- 10.20.193.16:0/739958922 >> 10.20.193.16:0/739958922 conn(0x7f36d80cf600 msgr2=0x7f36d80cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 1 -- 10.20.193.16:0/739958922 shutdown_connections 2026-02-17T05:54:49.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 1 -- 10.20.193.16:0/739958922 wait complete. 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 10 monclient: build_initial_monmap 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.686+0000 7f36e01f7640 10 monclient: monmap: 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 librados: starting msgr at 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 librados: starting objecter 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 Processor -- start 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 -- start start 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 librados: setting wanted keys 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 librados: calling monclient init 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient: init 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f36bc009890 0x7f36d8147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient(hunting): picked mon.noname-c con 0x7f36bc009890 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f36d80ca030 0x7f36d80ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient(hunting): picked mon.noname-a con 0x7f36d80ca030 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f36d81e7240 0x7f36d81e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient(hunting): picked mon.noname-b con 0x7f36d81e7240 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36d806da10 con 0x7f36d80ca030 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f36d806d730 con 0x7f36d81e7240 2026-02-17T05:54:49.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36d806dcf0 con 0x7f36bc009890 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36e01f7640 10 monclient(hunting): authenticate will time out at 942.256563s 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.687+0000 7f36de76d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f36d81e7240 0x7f36d81e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36de76d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f36d81e7240 0x7f36d81e76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:42582/0 (socket says 10.20.193.16:42582) 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36de76d640 1 -- 10.20.193.16:0/290484817 learned_addr learned my addr 10.20.193.16:0/290484817 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36de76d640 10 monclient(hunting): get_auth_request con 0x7f36d81e7240 auth_method 0 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36de76d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36de76d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36de76d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36ddf6c640 1 --2- 10.20.193.16:0/290484817 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f36bc009890 0x7f36d8147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36dd76b640 1 --2- 10.20.193.16:0/290484817 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f36d80ca030 0x7f36d80ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36de76d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36de76d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36de76d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request con 0x7f36bc009890 auth_method 0 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36ddf6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36ddf6c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36ddf6c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36dd76b640 10 monclient(hunting): get_auth_request con 0x7f36d80ca030 auth_method 0 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36dd76b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:49.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36dd76b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:49.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.688+0000 7f36dd76b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:49.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.689+0000 7f36dd76b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:49.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.689+0000 7f36dd76b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:49.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.689+0000 7f36dd76b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:49.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.689+0000 7f36ddf6c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:49.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.689+0000 7f36ddf6c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:49.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.689+0000 7f36ddf6c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:49.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.691+0000 7f36dd76b640 10 monclient(hunting): handle_auth_done global_id 4110 payload 824 2026-02-17T05:54:49.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.691+0000 7f36dd76b640 1 -- 10.20.193.16:0/290484817 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f36bc009890 msgr2=0x7f36d8147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:49.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.691+0000 7f36dd76b640 1 --2- 10.20.193.16:0/290484817 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f36bc009890 0x7f36d8147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:49.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.691+0000 7f36dd76b640 1 -- 10.20.193.16:0/290484817 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f36d81e7240 msgr2=0x7f36d81e76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:49.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.691+0000 7f36dd76b640 1 --2- 10.20.193.16:0/290484817 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f36d81e7240 0x7f36d81e76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:49.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.691+0000 7f36dd76b640 10 monclient: _finish_hunting 0 2026-02-17T05:54:49.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.691+0000 7f36dd76b640 1 monclient: found mon.noname-a 2026-02-17T05:54:49.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.691+0000 7f36dd76b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:49.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.691+0000 7f36dd76b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:49.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.691+0000 7f36dd76b640 1 -- 10.20.193.16:0/290484817 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d80ca9f0 con 0x7f36d80ca030 2026-02-17T05:54:49.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.693+0000 7f36dd76b640 1 --2- 10.20.193.16:0/290484817 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f36d80ca030 0x7f36d80ca4b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f36cc003d20 tx=0x7f36cc00bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:49.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.693+0000 7f36c67fc640 10 client.?.objecter ms_handle_connect 0x7f36d80ca030 2026-02-17T05:54:49.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.693+0000 7f36c67fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 1 -- 10.20.193.16:0/290484817 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc009b90 con 0x7f36d80ca030 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 10 monclient: dump: 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 10 monclient: _finish_auth 0 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 10 monclient: _check_auth_tickets 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36e01f7640 5 monclient: authenticate success, global_id 4110 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36e01f7640 10 monclient: _renew_subs 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36e01f7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36e01f7640 1 -- 10.20.193.16:0/290484817 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d81e7c00 con 0x7f36d80ca030 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36e01f7640 10 client.4110.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36e01f7640 10 monclient: _renew_subs 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36e01f7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36e01f7640 1 -- 10.20.193.16:0/290484817 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d8150bb0 con 0x7f36d80ca030 2026-02-17T05:54:49.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36e01f7640 1 librados: init done 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 1 -- 10.20.193.16:0/290484817 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36cc009d30 con 0x7f36d80ca030 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 1 -- 10.20.193.16:0/290484817 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc028060 con 0x7f36d80ca030 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c7fff640 4 set_mon_vals no callback set 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36c67fc640 10 monclient: dump: 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.695+0000 7f36e01f7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.696+0000 7f36e01f7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.696+0000 7f36e01f7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:49.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.696+0000 7f36e01f7640 1 -- 10.20.193.16:0/290484817 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d80cb870 con 0x7f36d80ca030 2026-02-17T05:54:49.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.772+0000 7f36c67fc640 1 -- 10.20.193.16:0/290484817 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f36cc01b9e0 con 0x7f36d80ca030 2026-02-17T05:54:49.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.772+0000 7f36c67fc640 1 -- 10.20.193.16:0/290484817 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f36cc023730 con 0x7f36d80ca030 2026-02-17T05:54:49.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.772+0000 7f36c67fc640 10 client.4110.objecter ms_dispatch2 0x7f36d8000b90 osd_map(1..1 src has 1..1) 2026-02-17T05:54:49.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.772+0000 7f36c67fc640 3 client.4110.objecter handle_osd_map got epochs [1,1] > 0 2026-02-17T05:54:49.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.772+0000 7f36c67fc640 3 client.4110.objecter handle_osd_map decoding full epoch 1 2026-02-17T05:54:49.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.772+0000 7f36c67fc640 20 client.4110.objecter dump_active .. 0 homeless 2026-02-17T05:54:49.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.772+0000 7f36c67fc640 1 -- 10.20.193.16:0/290484817 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f36d80ca9f0 con 0x7f36d80ca030 2026-02-17T05:54:49.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.772+0000 7f36c67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:49.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.772+0000 7f36c67fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:49.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.803+0000 7f36e01f7640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-17T05:54:49.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.803+0000 7f36e01f7640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-17T05:54:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.803+0000 7f36e01f7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.803+0000 7f36e01f7640 1 -- 10.20.193.16:0/290484817 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f36d806e850 con 0x7f36d80ca030 2026-02-17T05:54:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.900+0000 7f36c67fc640 1 -- 10.20.193.16:0/290484817 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f36cc01f020 con 0x7f36d80ca030 2026-02-17T05:54:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.900+0000 7f36c67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-17T05:54:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.900+0000 7f36c67fc640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2026-02-17T05:54:49.901 INFO:teuthology.orchestra.run.trial016.stderr:adjusted tunables profile to default 2026-02-17T05:54:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 10 librados: watch_flush enter 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 10 librados: watch_flush exit 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 20 client.4110.objecter shutdown clearing up homeless session... 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 10 client.4110.objecter successfully canceled tick 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 10 monclient: shutdown 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 1 -- 10.20.193.16:0/290484817 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f36d80ca030 msgr2=0x7f36d80ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 1 --2- 10.20.193.16:0/290484817 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f36d80ca030 0x7f36d80ca4b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f36cc003d20 tx=0x7f36cc00bcc0 comp rx=0 tx=0).stop 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 1 -- 10.20.193.16:0/290484817 shutdown_connections 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 1 --2- 10.20.193.16:0/290484817 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f36d81e7240 0x7f36d81e76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 1 --2- 10.20.193.16:0/290484817 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f36d80ca030 0x7f36d80ca4b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 1 --2- 10.20.193.16:0/290484817 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f36bc009890 0x7f36d8147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 1 -- 10.20.193.16:0/290484817 >> 10.20.193.16:0/290484817 conn(0x7f36d80cf600 msgr2=0x7f36d80c6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 1 -- 10.20.193.16:0/290484817 shutdown_connections 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 1 -- 10.20.193.16:0/290484817 wait complete. 2026-02-17T05:54:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:49.901+0000 7f36e01f7640 1 librados: shutdown 2026-02-17T05:54:49.910 INFO:tasks.ceph:check_enable_crimson: False 2026-02-17T05:54:49.910 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-17T05:54:49.910 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:54:49.911 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-17T05:54:49.935 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:54:49.935 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2026-02-17T05:54:49.991 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:49.990+0000 7fe672610140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-17T05:54:50.001 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:54:50.001 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2026-02-17T05:54:50.004 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.003+0000 7fcd207ce140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-17T05:54:50.025 DEBUG:teuthology.orchestra.run.trial016:> set -ex 2026-02-17T05:54:50.026 DEBUG:teuthology.orchestra.run.trial016:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2026-02-17T05:54:50.089 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:54:50.089 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-17T05:54:50.112 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.111+0000 7fe672610140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-17T05:54:50.116 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:54:50.117 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2026-02-17T05:54:50.126 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.126+0000 7fcd207ce140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-17T05:54:50.138 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.137+0000 7fe672610140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-17T05:54:50.152 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.152+0000 7fcd207ce140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-17T05:54:50.172 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.170+0000 7fe672610140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-17T05:54:50.181 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:54:50.181 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2026-02-17T05:54:50.185 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.185+0000 7fcd207ce140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-17T05:54:50.193 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.192+0000 7fe672610140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-17T05:54:50.207 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.207+0000 7fcd207ce140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-17T05:54:50.219 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.218+0000 7fe672610140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-17T05:54:50.233 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.233+0000 7fcd207ce140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-17T05:54:50.245 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T05:54:50.245 DEBUG:teuthology.orchestra.run.trial128:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2026-02-17T05:54:50.274 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.274+0000 7fe672610140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-17T05:54:50.289 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.288+0000 7fcd207ce140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-17T05:54:50.310 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:54:50.310 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-17T05:54:50.336 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:54:50.336 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2026-02-17T05:54:50.336 INFO:tasks.ceph.mgr.x.trial164.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-17T05:54:50.336 INFO:tasks.ceph.mgr.x.trial164.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-17T05:54:50.336 INFO:tasks.ceph.mgr.x.trial164.stderr: from numpy import show_config as show_numpy_config 2026-02-17T05:54:50.339 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.338+0000 7fe672610140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-17T05:54:50.349 INFO:tasks.ceph.mgr.y.trial016.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-17T05:54:50.349 INFO:tasks.ceph.mgr.y.trial016.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-17T05:54:50.349 INFO:tasks.ceph.mgr.y.trial016.stderr: from numpy import show_config as show_numpy_config 2026-02-17T05:54:50.353 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.352+0000 7fcd207ce140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-17T05:54:50.364 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.363+0000 7fe672610140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-17T05:54:50.377 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.377+0000 7fcd207ce140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-17T05:54:50.401 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:54:50.401 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2026-02-17T05:54:50.419 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.418+0000 7fe672610140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-17T05:54:50.432 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.432+0000 7fcd207ce140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-17T05:54:50.461 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.460+0000 7fe672610140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-17T05:54:50.464 DEBUG:teuthology.orchestra.run.trial148:> set -ex 2026-02-17T05:54:50.464 DEBUG:teuthology.orchestra.run.trial148:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2026-02-17T05:54:50.474 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.474+0000 7fcd207ce140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-17T05:54:50.486 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.485+0000 7fe672610140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-17T05:54:50.500 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.500+0000 7fcd207ce140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-17T05:54:50.511 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.509+0000 7fe672610140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-17T05:54:50.525 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.525+0000 7fcd207ce140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-17T05:54:50.528 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:54:50.529 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-17T05:54:50.536 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.534+0000 7fe672610140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-17T05:54:50.550 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.550+0000 7fcd207ce140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-17T05:54:50.554 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:54:50.554 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2026-02-17T05:54:50.558 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.557+0000 7fe672610140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-17T05:54:50.572 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.572+0000 7fcd207ce140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-17T05:54:50.581 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:54:50.581 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2026-02-17T05:54:50.644 DEBUG:teuthology.orchestra.run.trial164:> set -ex 2026-02-17T05:54:50.644 DEBUG:teuthology.orchestra.run.trial164:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2026-02-17T05:54:50.709 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new a1fd07ff-606a-4763-9d8d-5b703e21032a 0 2026-02-17T05:54:50.754 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.753+0000 7fe672610140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-17T05:54:50.766 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.766+0000 7fcd207ce140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-17T05:54:50.809 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.808+0000 7fe672610140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-17T05:54:50.813 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.812+0000 7fd7b86c4640 10 monclient: get_monmap_and_config 2026-02-17T05:54:50.813 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.812+0000 7fd7b86c4640 10 monclient: build_initial_monmap 2026-02-17T05:54:50.813 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.812+0000 7fd7b86c4640 10 monclient: monmap: 2026-02-17T05:54:50.813 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:50.813428+0000 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:50.813428+0000 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:50.814 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:50.817 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 1 Processor -- start 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 1 -- start start 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 10 monclient: init 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 10 monclient: _add_conns ranks=[2,0,4,1,3,5] 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd7b0073a00 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd7b011b490 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd7b011efa0 0x7fd7b0121390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 10 monclient(hunting): picked mon.noname-b con 0x7fd7b011efa0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b0121ab0 con 0x7fd7b011b490 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b0122cd0 con 0x7fd7b0073a00 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fd7b0123ed0 con 0x7fd7b011efa0 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.813+0000 7fd7b86c4640 10 monclient(hunting): authenticate will time out at 951.458464s 2026-02-17T05:54:50.818 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.814+0000 7fd7b6c3a640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd7b011efa0 0x7fd7b0121390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.814+0000 7fd7b5c38640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fd7b011b490 0x7fd7b011d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.164:53490/0 (socket says 10.20.193.164:53490) 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.814+0000 7fd7b5c38640 1 -- 10.20.193.164:0/2449637037 learned_addr learned my addr 10.20.193.164:0/2449637037 (peer_addr_for_me v1:10.20.193.164:0/0) 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.814+0000 7fd7b6c3a640 10 monclient(hunting): get_auth_request con 0x7fd7b011efa0 auth_method 0 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.814+0000 7fd7b6c3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.814+0000 7fd7b6c3a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.814+0000 7fd7b6c3a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.814+0000 7fd7b6c3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.814+0000 7fd7b6c3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.814+0000 7fd7b6c3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.815+0000 7fd7b6c3a640 10 monclient(hunting): handle_auth_done global_id 4105 payload 274 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.815+0000 7fd7b6c3a640 1 -- 10.20.193.164:0/2449637037 >> v1:10.20.193.148:6789/0 conn(0x7fd7b0073a00 legacy=0x7fd7b0073de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.815+0000 7fd7b6c3a640 1 -- 10.20.193.164:0/2449637037 >> v1:10.20.193.16:6789/0 conn(0x7fd7b011b490 legacy=0x7fd7b011d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.815+0000 7fd7b6c3a640 10 monclient: _finish_hunting 0 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.815+0000 7fd7b6c3a640 1 monclient: found mon.noname-b 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.815+0000 7fd7b6c3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.815+0000 7fd7b6c3a640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.815+0000 7fd7b6c3a640 1 -- 10.20.193.164:0/2449637037 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b0124050 con 0x7fd7b011efa0 2026-02-17T05:54:50.819 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.816+0000 7fd7b6c3a640 1 --2- 10.20.193.164:0/2449637037 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd7b011efa0 0x7fd7b0121390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009d90 tx=0x7fd7ac02f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5323804432e73b88 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:50.820 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.820+0000 7fcd207ce140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-17T05:54:50.824 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.823+0000 7fd7b5437640 1 -- 10.20.193.164:0/2449637037 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac03c070 con 0x7fd7b011efa0 2026-02-17T05:54:50.824 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.823+0000 7fd7b5437640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:50.824 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.823+0000 7fd7b5437640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:50.824 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.823+0000 7fd7b5437640 10 monclient: dump: 2026-02-17T05:54:50.824 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:50.824 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:50.824 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:50.824 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.823+0000 7fd7b5437640 10 monclient: _finish_auth 0 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.823+0000 7fd7b5437640 10 monclient: _check_auth_tickets 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.823+0000 7fd7b5437640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.823+0000 7fd7b86c4640 5 monclient: authenticate success, global_id 4105 2026-02-17T05:54:50.825 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.823+0000 7fd7b86c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:50.831 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.830+0000 7fe672610140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-17T05:54:50.835 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.834+0000 7fd7b5437640 1 -- 10.20.193.164:0/2449637037 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ac037440 con 0x7fd7b011efa0 2026-02-17T05:54:50.835 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.834+0000 7fd7b5437640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:50.835 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.834+0000 7fd7b5437640 1 -- 10.20.193.164:0/2449637037 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac037810 con 0x7fd7b011efa0 2026-02-17T05:54:50.835 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.834+0000 7fd7b5437640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.834+0000 7fd7b5437640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.834+0000 7fd7b5437640 10 monclient: dump: 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.834+0000 7fd7b86c4640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:50.836 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.834+0000 7fd7b86c4640 4 set_mon_vals no callback set 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 10 monclient: shutdown 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2449637037 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd7b011efa0 msgr2=0x7fd7b0121390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 --2- 10.20.193.164:0/2449637037 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd7b011efa0 0x7fd7b0121390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009d90 tx=0x7fd7ac02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2449637037 shutdown_connections 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 --2- 10.20.193.164:0/2449637037 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd7b011efa0 0x7fd7b0121390 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2449637037 >> 10.20.193.164:0/2449637037 conn(0x7fd7b006f550 msgr2=0x7fd7b0071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2449637037 shutdown_connections 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2449637037 wait complete. 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 10 monclient: build_initial_monmap 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 10 monclient: monmap: 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 librados: starting msgr at 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 librados: starting objecter 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 Processor -- start 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 -- start start 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 librados: setting wanted keys 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 librados: calling monclient init 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 10 monclient: init 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:50.838 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd7b0073a00 0x7fd7b01abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 10 monclient(hunting): picked mon.noname-b con 0x7fd7b0073a00 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd7b011b490 0x7fd7b01ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.835+0000 7fd7b86c4640 10 monclient(hunting): picked mon.noname-c con 0x7fd7b011b490 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b86c4640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd7b011efa0 0x7fd7b01b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b86c4640 10 monclient(hunting): picked mon.noname-a con 0x7fd7b011efa0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b86c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b86c4640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b0123360 con 0x7fd7b011efa0 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b86c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b86c4640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b01231e0 con 0x7fd7b0073a00 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b86c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b86c4640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b01234e0 con 0x7fd7b011b490 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b86c4640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b86c4640 10 monclient(hunting): authenticate will time out at 951.481018s 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6439640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd7b0073a00 0x7fd7b01abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6c3a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd7b011efa0 0x7fd7b01b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b5c38640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd7b011b490 0x7fd7b01ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6439640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd7b0073a00 0x7fd7b01abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.164:39246/0 (socket says 10.20.193.164:39246) 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6439640 1 -- 10.20.193.164:0/2248884200 learned_addr learned my addr 10.20.193.164:0/2248884200 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6439640 10 monclient(hunting): get_auth_request con 0x7fd7b0073a00 auth_method 0 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6439640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6439640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6439640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6c3a640 10 monclient(hunting): get_auth_request con 0x7fd7b011efa0 auth_method 0 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6c3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6c3a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6c3a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b5c38640 10 monclient(hunting): get_auth_request con 0x7fd7b011b490 auth_method 0 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b5c38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b5c38640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b5c38640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6439640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6439640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.836+0000 7fd7b6439640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:50.839 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.837+0000 7fd7b6c3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.837+0000 7fd7b6c3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.837+0000 7fd7b6c3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.837+0000 7fd7b5c38640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.837+0000 7fd7b5c38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.837+0000 7fd7b5c38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.838+0000 7fd7b5c38640 10 monclient(hunting): handle_auth_done global_id 4118 payload 824 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.838+0000 7fd7b5c38640 1 -- 10.20.193.164:0/2248884200 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd7b0073a00 msgr2=0x7fd7b01abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.838+0000 7fd7b5c38640 1 --2- 10.20.193.164:0/2248884200 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd7b0073a00 0x7fd7b01abb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.838+0000 7fd7b5c38640 1 -- 10.20.193.164:0/2248884200 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd7b011efa0 msgr2=0x7fd7b01b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.838+0000 7fd7b5c38640 1 --2- 10.20.193.164:0/2248884200 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd7b011efa0 0x7fd7b01b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.838+0000 7fd7b5c38640 10 monclient: _finish_hunting 0 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.838+0000 7fd7b5c38640 1 monclient: found mon.noname-c 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.838+0000 7fd7b5c38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.838+0000 7fd7b5c38640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.838+0000 7fd7b5c38640 1 -- 10.20.193.164:0/2248884200 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b01b0b40 con 0x7fd7b011b490 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.839+0000 7fd7b5c38640 1 --2- 10.20.193.164:0/2248884200 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd7b011b490 0x7fd7b01ac1e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400c500 tx=0x7fd7a400c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.839+0000 7fd796ffd640 10 client.?.objecter ms_handle_connect 0x7fd7b011b490 2026-02-17T05:54:50.840 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.839+0000 7fd796ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:50.842 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 1 -- 10.20.193.164:0/2248884200 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4017070 con 0x7fd7b011b490 2026-02-17T05:54:50.842 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:50.842 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:50.842 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 10 monclient: dump: 2026-02-17T05:54:50.842 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:50.842 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:50.842 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:50.842 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:50.842 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:50.842 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 10 monclient: _finish_auth 0 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 1 -- 10.20.193.164:0/2248884200 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a4005de0 con 0x7fd7b011b490 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd7b86c4640 5 monclient: authenticate success, global_id 4118 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd7b86c4640 10 monclient: _renew_subs 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd7b86c4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2248884200 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b01b0e30 con 0x7fd7b011b490 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 1 -- 10.20.193.164:0/2248884200 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40072b0 con 0x7fd7b011b490 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd796ffd640 10 monclient: dump: 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd7b4c36640 4 set_mon_vals no callback set 2026-02-17T05:54:50.843 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd7b86c4640 10 client.4118.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:50.844 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd7b86c4640 10 monclient: _renew_subs 2026-02-17T05:54:50.844 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd7b86c4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:50.844 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2248884200 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b01b85b0 con 0x7fd7b011b490 2026-02-17T05:54:50.844 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.841+0000 7fd7b86c4640 1 librados: init done 2026-02-17T05:54:50.844 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.842+0000 7fcd207ce140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-17T05:54:50.844 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.842+0000 7fd7b86c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:50.844 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.842+0000 7fd7b86c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:50.845 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.842+0000 7fd7b86c4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:50.845 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.842+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2248884200 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b01239c0 con 0x7fd7b011b490 2026-02-17T05:54:50.845 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.843+0000 7fd796ffd640 1 -- 10.20.193.164:0/2248884200 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd7a4007450 con 0x7fd7b011b490 2026-02-17T05:54:50.845 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.843+0000 7fd796ffd640 1 -- 10.20.193.164:0/2248884200 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fd7a4012760 con 0x7fd7b011b490 2026-02-17T05:54:50.845 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.843+0000 7fd796ffd640 10 client.4118.objecter ms_dispatch2 0x7fd7b0000b90 osd_map(2..2 src has 1..2) 2026-02-17T05:54:50.845 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.843+0000 7fd796ffd640 3 client.4118.objecter handle_osd_map got epochs [2,2] > 0 2026-02-17T05:54:50.845 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.843+0000 7fd796ffd640 3 client.4118.objecter handle_osd_map decoding full epoch 2 2026-02-17T05:54:50.845 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.843+0000 7fd796ffd640 20 client.4118.objecter dump_active .. 0 homeless 2026-02-17T05:54:50.854 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.853+0000 7fe672610140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-17T05:54:50.865 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.865+0000 7fcd207ce140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-17T05:54:50.907 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.906+0000 7fe672610140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-17T05:54:50.918 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.918+0000 7fcd207ce140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-17T05:54:50.919 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.917+0000 7fd796ffd640 1 -- 10.20.193.164:0/2248884200 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7fd7a4021020 con 0x7fd7b011b490 2026-02-17T05:54:50.919 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.917+0000 7fd796ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:50.919 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.917+0000 7fd796ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:50.927 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.926+0000 7fe672610140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-17T05:54:50.938 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.938+0000 7fcd207ce140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-17T05:54:50.951 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.949+0000 7fd7b86c4640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "a1fd07ff-606a-4763-9d8d-5b703e21032a", "id": 0}] 2026-02-17T05:54:50.951 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.949+0000 7fd7b86c4640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "a1fd07ff-606a-4763-9d8d-5b703e21032a", "id": 0}] 2026-02-17T05:54:50.951 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.949+0000 7fd7b86c4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:50.951 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:50.949+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2248884200 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a1fd07ff-606a-4763-9d8d-5b703e21032a", "id": 0} v 0) -- 0x7fd7b011ce70 con 0x7fd7b011b490 2026-02-17T05:54:50.978 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:50.977+0000 7fe672610140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-17T05:54:50.990 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:50.990+0000 7fcd207ce140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-17T05:54:51.052 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.051+0000 7fd796ffd640 1 -- 10.20.193.164:0/2248884200 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a1fd07ff-606a-4763-9d8d-5b703e21032a", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fd7a4012960 con 0x7fd7b011b490 2026-02-17T05:54:51.053 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.051+0000 7fd796ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "a1fd07ff-606a-4763-9d8d-5b703e21032a", "id": 0}] 2026-02-17T05:54:51.053 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.051+0000 7fd796ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:51.053 INFO:teuthology.orchestra.run.trial164.stdout:0 2026-02-17T05:54:51.053 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 10 librados: watch_flush enter 2026-02-17T05:54:51.053 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 10 librados: watch_flush exit 2026-02-17T05:54:51.053 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 20 client.4118.objecter shutdown clearing up homeless session... 2026-02-17T05:54:51.053 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 10 client.4118.objecter successfully canceled tick 2026-02-17T05:54:51.053 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 10 monclient: shutdown 2026-02-17T05:54:51.053 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:51.054 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2248884200 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd7b011b490 msgr2=0x7fd7b01ac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.054 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 1 --2- 10.20.193.164:0/2248884200 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd7b011b490 0x7fd7b01ac1e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400c500 tx=0x7fd7a400c9d0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.054 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2248884200 shutdown_connections 2026-02-17T05:54:51.054 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 1 --2- 10.20.193.164:0/2248884200 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd7b011efa0 0x7fd7b01b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.054 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 1 --2- 10.20.193.164:0/2248884200 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd7b011b490 0x7fd7b01ac1e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.054 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 1 --2- 10.20.193.164:0/2248884200 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd7b0073a00 0x7fd7b01abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.054 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2248884200 >> 10.20.193.164:0/2248884200 conn(0x7fd7b006f550 msgr2=0x7fd7b011a950 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:51.054 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2248884200 shutdown_connections 2026-02-17T05:54:51.054 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 1 -- 10.20.193.164:0/2248884200 wait complete. 2026-02-17T05:54:51.054 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.052+0000 7fd7b86c4640 1 librados: shutdown 2026-02-17T05:54:51.061 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new 7ef10114-33c6-4bb2-a822-d67cb7cdf725 1 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.164+0000 7fd27d72c640 10 monclient: get_monmap_and_config 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.164+0000 7fd27d72c640 10 monclient: build_initial_monmap 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.164+0000 7fd27d72c640 10 monclient: monmap: 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:51.165351+0000 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:51.165351+0000 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:51.166 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:51.167 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:51.167 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.167 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 1 Processor -- start 2026-02-17T05:54:51.167 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 1 -- start start 2026-02-17T05:54:51.167 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 10 monclient: init 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 10 monclient: _add_conns ranks=[2,1,4,3,5,0] 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd278119850 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fd27811d730 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd2781212a0 0x7fd278123760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 10 monclient(hunting): picked mon.noname-b con 0x7fd2781212a0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd278123e80 con 0x7fd27811d730 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2781250a0 con 0x7fd278119850 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fd2781262a0 con 0x7fd2781212a0 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.165+0000 7fd27d72c640 10 monclient(hunting): authenticate will time out at 951.810540s 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.166+0000 7fd276ffd640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fd278119850 0x7fd278119c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.164:60282/0 (socket says 10.20.193.164:60282) 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.166+0000 7fd276ffd640 1 -- 10.20.193.164:0/3935866698 learned_addr learned my addr 10.20.193.164:0/3935866698 (peer_addr_for_me v1:10.20.193.164:0/0) 2026-02-17T05:54:51.168 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.166+0000 7fd2777fe640 1 --2- 10.20.193.164:0/3935866698 >> v2:10.20.193.128:3300/0 conn(0x7fd2781212a0 0x7fd278123760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.169 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.166+0000 7fd2777fe640 10 monclient(hunting): get_auth_request con 0x7fd2781212a0 auth_method 0 2026-02-17T05:54:51.169 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.166+0000 7fd2777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.169 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.166+0000 7fd2777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.169 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.166+0000 7fd2777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 1 -- 10.20.193.164:0/3935866698 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318861675 0 0) 0x7fd2781250a0 con 0x7fd278119850 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 10 monclient(hunting): my global_id is 4121 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 1 -- 10.20.193.164:0/3935866698 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd264003620 con 0x7fd278119850 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd2777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd2777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd2777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 1 -- 10.20.193.164:0/3935866698 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422833735 0 0) 0x7fd278123e80 con 0x7fd27811d730 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.170 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 10 monclient(hunting): my global_id is 4111 2026-02-17T05:54:51.171 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.167+0000 7fd275ffb640 1 -- 10.20.193.164:0/3935866698 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2781250a0 con 0x7fd27811d730 2026-02-17T05:54:51.171 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.168+0000 7fd2777fe640 10 monclient(hunting): handle_auth_done global_id 4114 payload 274 2026-02-17T05:54:51.171 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.169+0000 7fd2777fe640 1 -- 10.20.193.164:0/3935866698 >> v1:10.20.193.148:6789/0 conn(0x7fd278119850 legacy=0x7fd278119c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.171 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.169+0000 7fd2777fe640 1 -- 10.20.193.164:0/3935866698 >> v1:10.20.193.128:6789/0 conn(0x7fd27811d730 legacy=0x7fd27811fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.171 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.169+0000 7fd2777fe640 10 monclient: _finish_hunting 0 2026-02-17T05:54:51.171 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.169+0000 7fd2777fe640 1 monclient: found mon.noname-b 2026-02-17T05:54:51.171 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.169+0000 7fd2777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:51.171 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.169+0000 7fd2777fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.171 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.169+0000 7fd2777fe640 1 -- 10.20.193.164:0/3935866698 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd278126420 con 0x7fd2781212a0 2026-02-17T05:54:51.171 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.170+0000 7fd2777fe640 1 --2- 10.20.193.164:0/3935866698 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 0x7fd278123760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd268009d90 tx=0x7fd26802f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295666392949652d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 1 -- 10.20.193.164:0/3935866698 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26803c070 con 0x7fd2781212a0 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 10 monclient: dump: 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 10 monclient: _finish_auth 0 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 1 -- 10.20.193.164:0/3935866698 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd268037440 con 0x7fd2781212a0 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:51.174 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd27d72c640 5 monclient: authenticate success, global_id 4114 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 1 -- 10.20.193.164:0/3935866698 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268037810 con 0x7fd2781212a0 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd27d72c640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd27d72c640 4 set_mon_vals no callback set 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.172+0000 7fd275ffb640 10 monclient: dump: 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.175 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.181 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient: shutdown 2026-02-17T05:54:51.181 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:51.181 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 -- 10.20.193.164:0/3935866698 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 msgr2=0x7fd278123760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.181 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 --2- 10.20.193.164:0/3935866698 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 0x7fd278123760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd268009d90 tx=0x7fd26802f1e0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.181 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 -- 10.20.193.164:0/3935866698 shutdown_connections 2026-02-17T05:54:51.181 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 --2- 10.20.193.164:0/3935866698 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 0x7fd278123760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.181 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 -- 10.20.193.164:0/3935866698 >> 10.20.193.164:0/3935866698 conn(0x7fd278073640 msgr2=0x7fd278073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:51.181 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 -- 10.20.193.164:0/3935866698 shutdown_connections 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 -- 10.20.193.164:0/3935866698 wait complete. 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient: build_initial_monmap 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient: monmap: 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 librados: starting msgr at 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 librados: starting objecter 2026-02-17T05:54:51.182 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 Processor -- start 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 -- start start 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 librados: setting wanted keys 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 librados: calling monclient init 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient: init 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd278119850 0x7fd27811f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient(hunting): picked mon.noname-a con 0x7fd278119850 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd27811d730 0x7fd27811fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient(hunting): picked mon.noname-c con 0x7fd27811d730 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 0x7fd2781b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient(hunting): picked mon.noname-b con 0x7fd2781212a0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2781244d0 con 0x7fd278119850 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.183 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd278124350 con 0x7fd2781212a0 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd278124650 con 0x7fd27811d730 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.173+0000 7fd27d72c640 10 monclient(hunting): authenticate will time out at 951.818981s 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2777fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 0x7fd2781b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2767fc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd27811d730 0x7fd27811fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2777fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 0x7fd2781b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.164:39262/0 (socket says 10.20.193.164:39262) 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2777fe640 1 -- 10.20.193.164:0/2463878852 learned_addr learned my addr 10.20.193.164:0/2463878852 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2777fe640 10 monclient(hunting): get_auth_request con 0x7fd2781212a0 auth_method 0 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd276ffd640 1 --2- 10.20.193.164:0/2463878852 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd278119850 0x7fd27811f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2767fc640 10 monclient(hunting): get_auth_request con 0x7fd27811d730 auth_method 0 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.184 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2767fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2767fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd2777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd276ffd640 10 monclient(hunting): get_auth_request con 0x7fd278119850 auth_method 0 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd276ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd276ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.174+0000 7fd276ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd276ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd276ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd276ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2777fe640 10 monclient(hunting): handle_auth_done global_id 4117 payload 824 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2777fe640 1 -- 10.20.193.164:0/2463878852 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd27811d730 msgr2=0x7fd27811fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2777fe640 1 --2- 10.20.193.164:0/2463878852 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd27811d730 0x7fd27811fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2777fe640 1 -- 10.20.193.164:0/2463878852 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd278119850 msgr2=0x7fd27811f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2777fe640 1 --2- 10.20.193.164:0/2463878852 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd278119850 0x7fd27811f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2777fe640 10 monclient: _finish_hunting 0 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2777fe640 1 monclient: found mon.noname-b 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2777fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.175+0000 7fd2777fe640 1 -- 10.20.193.164:0/2463878852 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2781b1b60 con 0x7fd2781212a0 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.176+0000 7fd2777fe640 1 --2- 10.20.193.164:0/2463878852 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 0x7fd2781b1340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd268009d90 tx=0x7fd268004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.176+0000 7fd2577fe640 10 client.?.objecter ms_handle_connect 0x7fd2781212a0 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.176+0000 7fd2577fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 1 -- 10.20.193.164:0/2463878852 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268047020 con 0x7fd2781212a0 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 10 monclient: dump: 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.186 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 10 monclient: _finish_auth 0 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 10 monclient: _check_auth_tickets 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 1 -- 10.20.193.164:0/2463878852 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd26803d040 con 0x7fd2781212a0 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd27d72c640 5 monclient: authenticate success, global_id 4117 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd27d72c640 10 monclient: _renew_subs 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd27d72c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd27d72c640 1 -- 10.20.193.164:0/2463878852 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2781b1d90 con 0x7fd2781212a0 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 1 -- 10.20.193.164:0/2463878852 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26803c050 con 0x7fd2781212a0 2026-02-17T05:54:51.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2757fa640 4 set_mon_vals no callback set 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd2577fe640 10 monclient: dump: 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd27d72c640 10 client.4117.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd27d72c640 10 monclient: _renew_subs 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd27d72c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd27d72c640 1 -- 10.20.193.164:0/2463878852 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2781b8410 con 0x7fd2781212a0 2026-02-17T05:54:51.188 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.178+0000 7fd27d72c640 1 librados: init done 2026-02-17T05:54:51.189 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.179+0000 7fd27d72c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:51.189 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.179+0000 7fd27d72c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:51.189 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.179+0000 7fd27d72c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.189 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.179+0000 7fd27d72c640 1 -- 10.20.193.164:0/2463878852 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd278124b30 con 0x7fd2781212a0 2026-02-17T05:54:51.191 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.190+0000 7fd2577fe640 1 -- 10.20.193.164:0/2463878852 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd268054070 con 0x7fd2781212a0 2026-02-17T05:54:51.191 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.190+0000 7fd2577fe640 1 -- 10.20.193.164:0/2463878852 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd2680439c0 con 0x7fd2781212a0 2026-02-17T05:54:51.191 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.190+0000 7fd2577fe640 10 client.4117.objecter ms_dispatch2 0x7fd278000b90 osd_map(3..3 src has 1..3) 2026-02-17T05:54:51.191 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.190+0000 7fd2577fe640 3 client.4117.objecter handle_osd_map got epochs [3,3] > 0 2026-02-17T05:54:51.191 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.190+0000 7fd2577fe640 3 client.4117.objecter handle_osd_map decoding full epoch 3 2026-02-17T05:54:51.191 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.190+0000 7fd2577fe640 20 client.4117.objecter dump_active .. 0 homeless 2026-02-17T05:54:51.265 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.264+0000 7fd2577fe640 1 -- 10.20.193.164:0/2463878852 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7fd268051020 con 0x7fd2781212a0 2026-02-17T05:54:51.266 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.264+0000 7fd2577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:51.266 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.264+0000 7fd2577fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:51.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.298+0000 7fd27d72c640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "7ef10114-33c6-4bb2-a822-d67cb7cdf725", "id": 1}] 2026-02-17T05:54:51.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.298+0000 7fd27d72c640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "7ef10114-33c6-4bb2-a822-d67cb7cdf725", "id": 1}] 2026-02-17T05:54:51.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.298+0000 7fd27d72c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.298+0000 7fd27d72c640 1 -- 10.20.193.164:0/2463878852 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7ef10114-33c6-4bb2-a822-d67cb7cdf725", "id": 1} v 0) -- 0x7fd2781b85e0 con 0x7fd2781212a0 2026-02-17T05:54:51.362 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.361+0000 7fd2577fe640 1 -- 10.20.193.164:0/2463878852 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7ef10114-33c6-4bb2-a822-d67cb7cdf725", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd268043bc0 con 0x7fd2781212a0 2026-02-17T05:54:51.363 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.361+0000 7fd2577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "7ef10114-33c6-4bb2-a822-d67cb7cdf725", "id": 1}] 2026-02-17T05:54:51.363 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.361+0000 7fd2577fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:51.363 INFO:teuthology.orchestra.run.trial164.stdout:1 2026-02-17T05:54:51.364 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 10 librados: watch_flush enter 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 10 librados: watch_flush exit 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 20 client.4117.objecter shutdown clearing up homeless session... 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 10 client.4117.objecter successfully canceled tick 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 10 monclient: shutdown 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 1 -- 10.20.193.164:0/2463878852 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 msgr2=0x7fd2781b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 1 --2- 10.20.193.164:0/2463878852 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 0x7fd2781b1340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd268009d90 tx=0x7fd268004b40 comp rx=0 tx=0).stop 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 1 -- 10.20.193.164:0/2463878852 shutdown_connections 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 1 --2- 10.20.193.164:0/2463878852 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2781212a0 0x7fd2781b1340 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 1 --2- 10.20.193.164:0/2463878852 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd27811d730 0x7fd27811fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.365 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 1 --2- 10.20.193.164:0/2463878852 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd278119850 0x7fd27811f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.366 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 1 -- 10.20.193.164:0/2463878852 >> 10.20.193.164:0/2463878852 conn(0x7fd278073640 msgr2=0x7fd27811b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:51.366 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 1 -- 10.20.193.164:0/2463878852 shutdown_connections 2026-02-17T05:54:51.366 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 1 -- 10.20.193.164:0/2463878852 wait complete. 2026-02-17T05:54:51.366 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.362+0000 7fd27d72c640 1 librados: shutdown 2026-02-17T05:54:51.372 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new a077772d-ffbf-4354-bc37-84706b589239 2 2026-02-17T05:54:51.379 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:51.378+0000 7fe672610140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-17T05:54:51.387 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:51.386+0000 7fcd207ce140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-17T05:54:51.401 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:51.400+0000 7fe672610140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-17T05:54:51.408 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:51.408+0000 7fcd207ce140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.434+0000 7fe83287e640 10 monclient: get_monmap_and_config 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.434+0000 7fe83287e640 10 monclient: build_initial_monmap 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.434+0000 7fe83287e640 10 monclient: monmap: 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:51.435392+0000 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:51.435392+0000 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:51.437 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 1 Processor -- start 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 1 -- start start 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 10 monclient: init 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 10 monclient: _add_conns ranks=[5,2,3,4,1,0] 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fe82c073870 0x7fe82c1181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 10 monclient(hunting): picked mon.noname-c con 0x7fe82c073870 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe82c073c50 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fe82c0740a0 0x7fe82c126a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 10 monclient(hunting): picked mon.noname-a con 0x7fe82c0740a0 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe82c127100 con 0x7fe82c073c50 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe82c1282e0 con 0x7fe82c0740a0 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe82c128460 con 0x7fe82c073870 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe83287e640 10 monclient(hunting): authenticate will time out at 952.080406s 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.435+0000 7fe830df4640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fe82c0740a0 0x7fe82c126a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe82b7fe640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fe82c073c50 0x7fe82c123360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.164:60294/0 (socket says 10.20.193.164:60294) 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe82b7fe640 1 -- 10.20.193.164:0/706366378 learned_addr learned my addr 10.20.193.164:0/706366378 (peer_addr_for_me v1:10.20.193.164:0/0) 2026-02-17T05:54:51.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe830df4640 10 monclient(hunting): get_auth_request con 0x7fe82c0740a0 auth_method 0 2026-02-17T05:54:51.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe830df4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe830df4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe830df4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe82bfff640 1 --2- 10.20.193.164:0/706366378 >> v2:10.20.193.148:3300/0 conn(0x7fe82c073870 0x7fe82c1181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe82bfff640 10 monclient(hunting): get_auth_request con 0x7fe82c073870 auth_method 0 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe82bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe82bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe82bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe830df4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe830df4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe830df4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe82bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe82bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.436+0000 7fe82bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.437+0000 7fe830df4640 10 monclient(hunting): handle_auth_done global_id 4122 payload 274 2026-02-17T05:54:51.441 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.437+0000 7fe830df4640 1 -- 10.20.193.164:0/706366378 >> v2:10.20.193.148:3300/0 conn(0x7fe82c073870 msgr2=0x7fe82c1181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:51.441 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.437+0000 7fe830df4640 1 --2- 10.20.193.164:0/706366378 >> v2:10.20.193.148:3300/0 conn(0x7fe82c073870 0x7fe82c1181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.441 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.437+0000 7fe830df4640 1 -- 10.20.193.164:0/706366378 >> v1:10.20.193.148:6789/0 conn(0x7fe82c073c50 legacy=0x7fe82c123360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.441 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.437+0000 7fe830df4640 10 monclient: _finish_hunting 0 2026-02-17T05:54:51.441 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.437+0000 7fe830df4640 1 monclient: found mon.noname-a 2026-02-17T05:54:51.441 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.437+0000 7fe830df4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:51.441 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.437+0000 7fe830df4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:51.441 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.437+0000 7fe830df4640 1 -- 10.20.193.164:0/706366378 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c1285e0 con 0x7fe82c0740a0 2026-02-17T05:54:51.441 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.438+0000 7fe830df4640 1 --2- 10.20.193.164:0/706366378 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe82c0740a0 0x7fe82c126a90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0766a0 tx=0x7fe82002f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e952ae60c126918 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:51.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 1 -- 10.20.193.164:0/706366378 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82003c070 con 0x7fe82c0740a0 2026-02-17T05:54:51.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:51.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 10 monclient: dump: 2026-02-17T05:54:51.446 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.446 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.446 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 10 monclient: _finish_auth 0 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 1 -- 10.20.193.164:0/706366378 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe82002fe40 con 0x7fe82c0740a0 2026-02-17T05:54:51.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 1 -- 10.20.193.164:0/706366378 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820037660 con 0x7fe82c0740a0 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 5 monclient: authenticate success, global_id 4122 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 4 set_mon_vals no callback set 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe82affd640 10 monclient: dump: 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.448 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 10 monclient: shutdown 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 1 -- 10.20.193.164:0/706366378 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe82c0740a0 msgr2=0x7fe82c126a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 1 --2- 10.20.193.164:0/706366378 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe82c0740a0 0x7fe82c126a90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0766a0 tx=0x7fe82002f130 comp rx=0 tx=0).stop 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 1 -- 10.20.193.164:0/706366378 shutdown_connections 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 1 --2- 10.20.193.164:0/706366378 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe82c0740a0 0x7fe82c126a90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 1 --2- 10.20.193.164:0/706366378 >> v2:10.20.193.148:3300/0 conn(0x7fe82c073870 0x7fe82c1181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 1 -- 10.20.193.164:0/706366378 >> 10.20.193.164:0/706366378 conn(0x7fe82c06f550 msgr2=0x7fe82c071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 1 -- 10.20.193.164:0/706366378 shutdown_connections 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.440+0000 7fe83287e640 1 -- 10.20.193.164:0/706366378 wait complete. 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient: build_initial_monmap 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient: monmap: 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.450 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 librados: starting msgr at 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 librados: starting objecter 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 Processor -- start 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 -- start start 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 librados: setting wanted keys 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 librados: calling monclient init 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient: init 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:51.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe82c073870 0x7fe82c1afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient(hunting): picked mon.noname-a con 0x7fe82c073870 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe82c073c50 0x7fe82c1b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient(hunting): picked mon.noname-c con 0x7fe82c073c50 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe82c0740a0 0x7fe82c1b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient(hunting): picked mon.noname-b con 0x7fe82c0740a0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c127790 con 0x7fe82c073870 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c127610 con 0x7fe82c0740a0 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c127910 con 0x7fe82c073c50 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe83287e640 10 monclient(hunting): authenticate will time out at 952.086615s 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.441+0000 7fe82b7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe82c073c50 0x7fe82c1b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe830df4640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe82c0740a0 0x7fe82c1b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82b7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe82c073c50 0x7fe82c1b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.164:44740/0 (socket says 10.20.193.164:44740) 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82b7fe640 1 -- 10.20.193.164:0/697438454 learned_addr learned my addr 10.20.193.164:0/697438454 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82b7fe640 10 monclient(hunting): get_auth_request con 0x7fe82c073c50 auth_method 0 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe830df4640 10 monclient(hunting): get_auth_request con 0x7fe82c0740a0 auth_method 0 2026-02-17T05:54:51.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe830df4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe830df4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe830df4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82bfff640 1 --2- 10.20.193.164:0/697438454 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe82c073870 0x7fe82c1afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe830df4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe830df4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe830df4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82bfff640 10 monclient(hunting): get_auth_request con 0x7fe82c073870 auth_method 0 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.442+0000 7fe82bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.443+0000 7fe830df4640 10 monclient(hunting): handle_auth_done global_id 4120 payload 824 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.443+0000 7fe830df4640 1 -- 10.20.193.164:0/697438454 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe82c073c50 msgr2=0x7fe82c1b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.443+0000 7fe830df4640 1 --2- 10.20.193.164:0/697438454 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe82c073c50 0x7fe82c1b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.443+0000 7fe830df4640 1 -- 10.20.193.164:0/697438454 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe82c073870 msgr2=0x7fe82c1afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.443+0000 7fe830df4640 1 --2- 10.20.193.164:0/697438454 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe82c073870 0x7fe82c1afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.443+0000 7fe830df4640 10 monclient: _finish_hunting 0 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.443+0000 7fe830df4640 1 monclient: found mon.noname-b 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.443+0000 7fe830df4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:51.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.443+0000 7fe830df4640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.443+0000 7fe830df4640 1 -- 10.20.193.164:0/697438454 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c1b4d10 con 0x7fe82c0740a0 2026-02-17T05:54:51.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.444+0000 7fe830df4640 1 --2- 10.20.193.164:0/697438454 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe82c0740a0 0x7fe82c1b4450 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe820036000 tx=0x7fe820004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:51.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.444+0000 7fe828ff9640 10 client.?.objecter ms_handle_connect 0x7fe82c0740a0 2026-02-17T05:54:51.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.445+0000 7fe828ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:51.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 1 -- 10.20.193.164:0/697438454 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820047020 con 0x7fe82c0740a0 2026-02-17T05:54:51.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:51.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 10 monclient: dump: 2026-02-17T05:54:51.458 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.458 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.458 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 10 monclient: _finish_auth 0 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:51.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 5 monclient: authenticate success, global_id 4120 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 10 monclient: _renew_subs 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 1 -- 10.20.193.164:0/697438454 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82c1b4fa0 con 0x7fe82c0740a0 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 10 client.4120.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 10 monclient: _renew_subs 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 1 -- 10.20.193.164:0/697438454 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82c1bc740 con 0x7fe82c0740a0 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 1 librados: init done 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 1 -- 10.20.193.164:0/697438454 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe820042470 con 0x7fe82c0740a0 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 1 -- 10.20.193.164:0/697438454 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82003c040 con 0x7fe82c0740a0 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:51.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe82a7fc640 4 set_mon_vals no callback set 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe828ff9640 10 monclient: dump: 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:51.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.462 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.446+0000 7fe83287e640 1 -- 10.20.193.164:0/697438454 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe82c127d20 con 0x7fe82c0740a0 2026-02-17T05:54:51.486 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.485+0000 7fe828ff9640 1 -- 10.20.193.164:0/697438454 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fe820042bf0 con 0x7fe82c0740a0 2026-02-17T05:54:51.491 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.489+0000 7fe828ff9640 1 -- 10.20.193.164:0/697438454 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fe820043840 con 0x7fe82c0740a0 2026-02-17T05:54:51.491 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.489+0000 7fe828ff9640 10 client.4120.objecter ms_dispatch2 0x7fe82c000b90 osd_map(4..4 src has 1..4) 2026-02-17T05:54:51.491 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.489+0000 7fe828ff9640 3 client.4120.objecter handle_osd_map got epochs [4,4] > 0 2026-02-17T05:54:51.491 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.489+0000 7fe828ff9640 3 client.4120.objecter handle_osd_map decoding full epoch 4 2026-02-17T05:54:51.492 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.489+0000 7fe828ff9640 20 client.4120.objecter dump_active .. 0 homeless 2026-02-17T05:54:51.492 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.490+0000 7fe828ff9640 1 -- 10.20.193.164:0/697438454 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7fe820051030 con 0x7fe82c0740a0 2026-02-17T05:54:51.492 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.490+0000 7fe828ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:51.492 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.490+0000 7fe828ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:51.523 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.522+0000 7fe83287e640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "a077772d-ffbf-4354-bc37-84706b589239", "id": 2}] 2026-02-17T05:54:51.524 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.522+0000 7fe83287e640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "a077772d-ffbf-4354-bc37-84706b589239", "id": 2}] 2026-02-17T05:54:51.524 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.522+0000 7fe83287e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:51.524 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.522+0000 7fe83287e640 1 -- 10.20.193.164:0/697438454 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a077772d-ffbf-4354-bc37-84706b589239", "id": 2} v 0) -- 0x7fe82c1bc970 con 0x7fe82c0740a0 2026-02-17T05:54:51.557 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe828ff9640 1 -- 10.20.193.164:0/697438454 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a077772d-ffbf-4354-bc37-84706b589239", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fe820055dd0 con 0x7fe82c0740a0 2026-02-17T05:54:51.557 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe828ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "a077772d-ffbf-4354-bc37-84706b589239", "id": 2}] 2026-02-17T05:54:51.557 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe828ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:51.557 INFO:teuthology.orchestra.run.trial164.stdout:2 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 10 librados: watch_flush enter 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 10 librados: watch_flush exit 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 20 client.4120.objecter shutdown clearing up homeless session... 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 10 client.4120.objecter successfully canceled tick 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 10 monclient: shutdown 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 1 -- 10.20.193.164:0/697438454 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe82c0740a0 msgr2=0x7fe82c1b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 1 --2- 10.20.193.164:0/697438454 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe82c0740a0 0x7fe82c1b4450 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe820036000 tx=0x7fe820004b40 comp rx=0 tx=0).stop 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 1 -- 10.20.193.164:0/697438454 shutdown_connections 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 1 --2- 10.20.193.164:0/697438454 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe82c0740a0 0x7fe82c1b4450 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 1 --2- 10.20.193.164:0/697438454 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe82c073c50 0x7fe82c1b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 1 --2- 10.20.193.164:0/697438454 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe82c073870 0x7fe82c1afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 1 -- 10.20.193.164:0/697438454 >> 10.20.193.164:0/697438454 conn(0x7fe82c06f550 msgr2=0x7fe82c123cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 1 -- 10.20.193.164:0/697438454 shutdown_connections 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 1 -- 10.20.193.164:0/697438454 wait complete. 2026-02-17T05:54:51.558 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.556+0000 7fe83287e640 1 librados: shutdown 2026-02-17T05:54:51.565 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new c82441ef-121c-4d28-a591-eb6ef81200c4 3 2026-02-17T05:54:51.674 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient: get_monmap_and_config 2026-02-17T05:54:51.674 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient: build_initial_monmap 2026-02-17T05:54:51.674 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient: monmap: 2026-02-17T05:54:51.674 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:51.674 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.674 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:51.670800+0000 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:51.670800+0000 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.675 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 1 Processor -- start 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 1 -- start start 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient: init 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient: _add_conns ranks=[0,5,4,2,1,3] 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd4cc073a00 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd4cc11b490 0x7fd4cc11d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient(hunting): picked mon.noname-c con 0x7fd4cc11b490 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd4cc11ddc0 0x7fd4cc1201b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient(hunting): picked mon.noname-b con 0x7fd4cc11ddc0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4cc1208d0 con 0x7fd4cc073a00 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fd4cc121af0 con 0x7fd4cc11ddc0 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd4cc121c70 con 0x7fd4cc11b490 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.670+0000 7fd4d2687640 10 monclient(hunting): authenticate will time out at 952.315821s 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4cb7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd4cc11b490 0x7fd4cc11d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4cbfff640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fd4cc073a00 0x7fd4cc073de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.164:53496/0 (socket says 10.20.193.164:53496) 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4cbfff640 1 -- 10.20.193.164:0/490234577 learned_addr learned my addr 10.20.193.164:0/490234577 (peer_addr_for_me v1:10.20.193.164:0/0) 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4d0bfd640 1 --2- 10.20.193.164:0/490234577 >> v2:10.20.193.128:3300/0 conn(0x7fd4cc11ddc0 0x7fd4cc1201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4cb7fe640 10 monclient(hunting): get_auth_request con 0x7fd4cc11b490 auth_method 0 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4cb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.677 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4cb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4d0bfd640 10 monclient(hunting): get_auth_request con 0x7fd4cc11ddc0 auth_method 0 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4d0bfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4d0bfd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4d0bfd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4d0bfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4d0bfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.671+0000 7fd4d0bfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.672+0000 7fd4cb7fe640 10 monclient(hunting): handle_auth_done global_id 4136 payload 274 2026-02-17T05:54:51.678 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.672+0000 7fd4cb7fe640 1 -- 10.20.193.164:0/490234577 >> v2:10.20.193.128:3300/0 conn(0x7fd4cc11ddc0 msgr2=0x7fd4cc1201b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:51.679 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.672+0000 7fd4cb7fe640 1 --2- 10.20.193.164:0/490234577 >> v2:10.20.193.128:3300/0 conn(0x7fd4cc11ddc0 0x7fd4cc1201b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.679 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.672+0000 7fd4cb7fe640 1 -- 10.20.193.164:0/490234577 >> v1:10.20.193.16:6789/0 conn(0x7fd4cc073a00 legacy=0x7fd4cc073de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.679 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.672+0000 7fd4cb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:54:51.679 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.672+0000 7fd4cb7fe640 1 monclient: found mon.noname-c 2026-02-17T05:54:51.679 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.672+0000 7fd4cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:51.679 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.672+0000 7fd4cb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:51.679 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.672+0000 7fd4cb7fe640 1 -- 10.20.193.164:0/490234577 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc121e20 con 0x7fd4cc11b490 2026-02-17T05:54:51.686 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.673+0000 7fd4cb7fe640 1 --2- 10.20.193.164:0/490234577 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 0x7fd4cc11d880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0745b0 tx=0x7fd4bc02ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff59734215da30c9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:51.686 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 1 -- 10.20.193.164:0/490234577 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc03c070 con 0x7fd4cc11b490 2026-02-17T05:54:51.686 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.686 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:51.686 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 10 monclient: dump: 2026-02-17T05:54:51.686 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.686 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 10 monclient: _finish_auth 0 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:51.687 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 1 -- 10.20.193.164:0/490234577 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc02fa60 con 0x7fd4cc11b490 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 1 -- 10.20.193.164:0/490234577 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc02fe30 con 0x7fd4cc11b490 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4caffd640 10 monclient: dump: 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.688 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 5 monclient: authenticate success, global_id 4136 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 4 set_mon_vals no callback set 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 10 monclient: shutdown 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 1 -- 10.20.193.164:0/490234577 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 msgr2=0x7fd4cc11d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 1 --2- 10.20.193.164:0/490234577 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 0x7fd4cc11d880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0745b0 tx=0x7fd4bc02ed50 comp rx=0 tx=0).stop 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 1 -- 10.20.193.164:0/490234577 shutdown_connections 2026-02-17T05:54:51.689 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 1 --2- 10.20.193.164:0/490234577 >> v2:10.20.193.128:3300/0 conn(0x7fd4cc11ddc0 0x7fd4cc1201b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 1 --2- 10.20.193.164:0/490234577 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 0x7fd4cc11d880 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 1 -- 10.20.193.164:0/490234577 >> 10.20.193.164:0/490234577 conn(0x7fd4cc06f550 msgr2=0x7fd4cc071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 1 -- 10.20.193.164:0/490234577 shutdown_connections 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.675+0000 7fd4d2687640 1 -- 10.20.193.164:0/490234577 wait complete. 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient: build_initial_monmap 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient: monmap: 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.690 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 librados: starting msgr at 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 librados: starting objecter 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 Processor -- start 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 -- start start 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 librados: setting wanted keys 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 librados: calling monclient init 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient: init 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:51.691 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd4cc073a00 0x7fd4cc1b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient(hunting): picked mon.noname-a con 0x7fd4cc073a00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 0x7fd4cc1b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient(hunting): picked mon.noname-c con 0x7fd4cc11b490 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd4cc11ddc0 0x7fd4cc1ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient(hunting): picked mon.noname-b con 0x7fd4cc11ddc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc120f60 con 0x7fd4cc073a00 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc120de0 con 0x7fd4cc11ddc0 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:51.692 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1210e0 con 0x7fd4cc11b490 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d2687640 10 monclient(hunting): authenticate will time out at 952.321397s 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cb7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 0x7fd4cc1b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cbfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd4cc073a00 0x7fd4cc1b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4d0bfd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd4cc11ddc0 0x7fd4cc1ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cb7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 0x7fd4cc1b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.164:44752/0 (socket says 10.20.193.164:44752) 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cb7fe640 1 -- 10.20.193.164:0/1990194041 learned_addr learned my addr 10.20.193.164:0/1990194041 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cb7fe640 10 monclient(hunting): get_auth_request con 0x7fd4cc11b490 auth_method 0 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.693 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cbfff640 10 monclient(hunting): get_auth_request con 0x7fd4cc073a00 auth_method 0 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.676+0000 7fd4cbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4d0bfd640 10 monclient(hunting): get_auth_request con 0x7fd4cc11ddc0 auth_method 0 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4d0bfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4d0bfd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4d0bfd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.694 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4d0bfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4d0bfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.677+0000 7fd4d0bfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.678+0000 7fd4cb7fe640 10 monclient(hunting): handle_auth_done global_id 4139 payload 824 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.678+0000 7fd4cb7fe640 1 -- 10.20.193.164:0/1990194041 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd4cc11ddc0 msgr2=0x7fd4cc1ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.678+0000 7fd4cb7fe640 1 --2- 10.20.193.164:0/1990194041 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd4cc11ddc0 0x7fd4cc1ab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.678+0000 7fd4cb7fe640 1 -- 10.20.193.164:0/1990194041 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd4cc073a00 msgr2=0x7fd4cc1b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.678+0000 7fd4cb7fe640 1 --2- 10.20.193.164:0/1990194041 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd4cc073a00 0x7fd4cc1b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.678+0000 7fd4cb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.678+0000 7fd4cb7fe640 1 monclient: found mon.noname-c 2026-02-17T05:54:51.695 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.678+0000 7fd4cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.678+0000 7fd4cb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.678+0000 7fd4cb7fe640 1 -- 10.20.193.164:0/1990194041 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc1abe70 con 0x7fd4cc11b490 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.679+0000 7fd4cb7fe640 1 --2- 10.20.193.164:0/1990194041 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 0x7fd4cc1b1b30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009130 tx=0x7fd4bc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.679+0000 7fd4c8ff9640 10 client.?.objecter ms_handle_connect 0x7fd4cc11b490 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.679+0000 7fd4c8ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 1 -- 10.20.193.164:0/1990194041 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc03c040 con 0x7fd4cc11b490 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 10 monclient: dump: 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.696 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 10 monclient: _finish_auth 0 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 1 -- 10.20.193.164:0/1990194041 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc042450 con 0x7fd4cc11b490 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:51.697 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 1 -- 10.20.193.164:0/1990194041 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc042820 con 0x7fd4cc11b490 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 5 monclient: authenticate success, global_id 4139 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 10 monclient: _renew_subs 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 1 -- 10.20.193.164:0/1990194041 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc1ac100 con 0x7fd4cc11b490 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4ca7fc640 4 set_mon_vals no callback set 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4c8ff9640 10 monclient: dump: 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:51.698 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 10 client.4139.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 10 monclient: _renew_subs 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 1 -- 10.20.193.164:0/1990194041 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc1b82d0 con 0x7fd4cc11b490 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 1 librados: init done 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:51.699 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:51.700 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:51.700 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.681+0000 7fd4d2687640 1 -- 10.20.193.164:0/1990194041 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc1214f0 con 0x7fd4cc11b490 2026-02-17T05:54:51.700 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.682+0000 7fd4c8ff9640 1 -- 10.20.193.164:0/1990194041 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd4bc055430 con 0x7fd4cc11b490 2026-02-17T05:54:51.700 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.682+0000 7fd4c8ff9640 1 -- 10.20.193.164:0/1990194041 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fd4bc04b8a0 con 0x7fd4cc11b490 2026-02-17T05:54:51.700 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.682+0000 7fd4c8ff9640 10 client.4139.objecter ms_dispatch2 0x7fd4cc000b90 osd_map(5..5 src has 1..5) 2026-02-17T05:54:51.700 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.682+0000 7fd4c8ff9640 3 client.4139.objecter handle_osd_map got epochs [5,5] > 0 2026-02-17T05:54:51.700 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.682+0000 7fd4c8ff9640 3 client.4139.objecter handle_osd_map decoding full epoch 5 2026-02-17T05:54:51.700 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.682+0000 7fd4c8ff9640 20 client.4139.objecter dump_active .. 0 homeless 2026-02-17T05:54:51.723 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.722+0000 7fd4c8ff9640 1 -- 10.20.193.164:0/1990194041 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7fd4bc04b410 con 0x7fd4cc11b490 2026-02-17T05:54:51.723 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.722+0000 7fd4c8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:51.723 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.722+0000 7fd4c8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:51.747 INFO:tasks.ceph.mgr.x.trial164.stderr:2026-02-17T05:54:51.746+0000 7fe672610140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-17T05:54:51.749 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:51.749+0000 7fcd207ce140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-17T05:54:51.755 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.754+0000 7fd4d2687640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c82441ef-121c-4d28-a591-eb6ef81200c4", "id": 3}] 2026-02-17T05:54:51.756 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.754+0000 7fd4d2687640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c82441ef-121c-4d28-a591-eb6ef81200c4", "id": 3}] 2026-02-17T05:54:51.756 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.754+0000 7fd4d2687640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:51.756 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.754+0000 7fd4d2687640 1 -- 10.20.193.164:0/1990194041 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c82441ef-121c-4d28-a591-eb6ef81200c4", "id": 3} v 0) -- 0x7fd4cc11aad0 con 0x7fd4cc11b490 2026-02-17T05:54:51.887 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.885+0000 7fd4c8ff9640 1 -- 10.20.193.164:0/1990194041 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mgrmap(e 2) ==== 50049+0+0 (secure 0 0 0) 0x7fd4bc04baa0 con 0x7fd4cc11b490 2026-02-17T05:54:51.931 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.929+0000 7fd4c8ff9640 1 -- 10.20.193.164:0/1990194041 <== mon.2 v2:10.20.193.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c82441ef-121c-4d28-a591-eb6ef81200c4", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fd4bc050e40 con 0x7fd4cc11b490 2026-02-17T05:54:51.931 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.929+0000 7fd4c8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c82441ef-121c-4d28-a591-eb6ef81200c4", "id": 3}] 2026-02-17T05:54:51.931 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.929+0000 7fd4c8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:51.931 INFO:teuthology.orchestra.run.trial164.stdout:3 2026-02-17T05:54:51.933 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 10 librados: watch_flush enter 2026-02-17T05:54:51.933 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 10 librados: watch_flush exit 2026-02-17T05:54:51.933 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 20 client.4139.objecter shutdown clearing up homeless session... 2026-02-17T05:54:51.933 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 10 client.4139.objecter successfully canceled tick 2026-02-17T05:54:51.934 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 10 monclient: shutdown 2026-02-17T05:54:51.934 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:51.934 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 1 -- 10.20.193.164:0/1990194041 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 msgr2=0x7fd4cc1b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:51.934 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 1 --2- 10.20.193.164:0/1990194041 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 0x7fd4cc1b1b30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009130 tx=0x7fd4bc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.934 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 1 -- 10.20.193.164:0/1990194041 shutdown_connections 2026-02-17T05:54:51.934 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 1 --2- 10.20.193.164:0/1990194041 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd4cc11ddc0 0x7fd4cc1ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.934 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 1 --2- 10.20.193.164:0/1990194041 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd4cc11b490 0x7fd4cc1b1b30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.934 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 1 --2- 10.20.193.164:0/1990194041 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd4cc073a00 0x7fd4cc1b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:51.934 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 1 -- 10.20.193.164:0/1990194041 >> 10.20.193.164:0/1990194041 conn(0x7fd4cc06f550 msgr2=0x7fd4cc11e470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:51.934 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 1 -- 10.20.193.164:0/1990194041 shutdown_connections 2026-02-17T05:54:51.935 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 1 -- 10.20.193.164:0/1990194041 wait complete. 2026-02-17T05:54:51.935 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:51.930+0000 7fd4d2687640 1 librados: shutdown 2026-02-17T05:54:51.941 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new 5bf489c2-40d8-490c-9c10-34a9c6fc407c 4 2026-02-17T05:54:52.006 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.003+0000 7f94d0187640 10 monclient: get_monmap_and_config 2026-02-17T05:54:52.006 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.003+0000 7f94d0187640 10 monclient: build_initial_monmap 2026-02-17T05:54:52.006 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.003+0000 7f94d0187640 10 monclient: monmap: 2026-02-17T05:54:52.006 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:52.006 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.007 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:52.004239+0000 2026-02-17T05:54:52.007 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:52.004239+0000 2026-02-17T05:54:52.007 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:52.007 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.007 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:52.007 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:52.007 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:52.007 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:52.007 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 1 Processor -- start 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 1 -- start start 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 10 monclient: init 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 10 monclient: _add_conns ranks=[2,5,4,0,3,1] 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f94c81149e0 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f94c811d600 0x7f94c811f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 10 monclient(hunting): picked mon.noname-c con 0x7f94c811d600 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:52.008 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f94c811ffc0 0x7f94c8122480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 10 monclient(hunting): picked mon.noname-b con 0x7f94c811ffc0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c8122ba0 con 0x7f94c81149e0 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f94c8123dc0 con 0x7f94c811ffc0 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f94c8123f40 con 0x7f94c811d600 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.004+0000 7f94d0187640 10 monclient(hunting): authenticate will time out at 952.649349s 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.005+0000 7f94cd6fb640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f94c811d600 0x7f94c811f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.005+0000 7f94cd6fb640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f94c811d600 0x7f94c811f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.164:44766/0 (socket says 10.20.193.164:44766) 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.005+0000 7f94cd6fb640 1 -- 10.20.193.164:0/1150956114 learned_addr learned my addr 10.20.193.164:0/1150956114 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.005+0000 7f94cd6fb640 10 monclient(hunting): get_auth_request con 0x7f94c811d600 auth_method 0 2026-02-17T05:54:52.009 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.005+0000 7f94cd6fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.010 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.005+0000 7f94cd6fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.010 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.005+0000 7f94cd6fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.011 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.005+0000 7f94cd6fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.011 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.005+0000 7f94cd6fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.011 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.005+0000 7f94cd6fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.011 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.006+0000 7f94cd6fb640 10 monclient(hunting): handle_auth_done global_id 4148 payload 274 2026-02-17T05:54:52.011 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.006+0000 7f94cd6fb640 1 -- 10.20.193.164:0/1150956114 >> v2:10.20.193.128:3300/0 conn(0x7f94c811ffc0 msgr2=0x7f94c8122480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:52.011 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.006+0000 7f94cd6fb640 1 --2- 10.20.193.164:0/1150956114 >> v2:10.20.193.128:3300/0 conn(0x7f94c811ffc0 0x7f94c8122480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.012 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.006+0000 7f94cd6fb640 1 -- 10.20.193.164:0/1150956114 >> v1:10.20.193.148:6789/0 conn(0x7f94c81149e0 legacy=0x7f94c8114dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.012 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.006+0000 7f94cd6fb640 10 monclient: _finish_hunting 0 2026-02-17T05:54:52.012 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.006+0000 7f94cd6fb640 1 monclient: found mon.noname-c 2026-02-17T05:54:52.012 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.006+0000 7f94cd6fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:52.012 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.006+0000 7f94cd6fb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.012 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.006+0000 7f94cd6fb640 1 -- 10.20.193.164:0/1150956114 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c81240f0 con 0x7f94c811d600 2026-02-17T05:54:52.012 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.010+0000 7f94cd6fb640 1 --2- 10.20.193.164:0/1150956114 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 0x7f94c811f9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f94c81146a0 tx=0x7f94bc02eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ecdf891adbeac95 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:52.017 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 1 -- 10.20.193.164:0/1150956114 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc03c070 con 0x7f94c811d600 2026-02-17T05:54:52.017 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.017 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:52.017 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 10 monclient: dump: 2026-02-17T05:54:52.017 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.017 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.017 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.018 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.018 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.018 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.018 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.018 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.018 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.018 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.018 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 10 monclient: _finish_auth 0 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 10 monclient: _check_auth_tickets 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 1 -- 10.20.193.164:0/1150956114 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94bc02fbe0 con 0x7f94c811d600 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94d0187640 5 monclient: authenticate success, global_id 4148 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 1 -- 10.20.193.164:0/1150956114 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc037440 con 0x7f94c811d600 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94ccefa640 10 monclient: dump: 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.019 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94d0187640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.013+0000 7f94d0187640 4 set_mon_vals no callback set 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 10 monclient: shutdown 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:52.020 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 -- 10.20.193.164:0/1150956114 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 msgr2=0x7f94c811f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 --2- 10.20.193.164:0/1150956114 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 0x7f94c811f9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f94c81146a0 tx=0x7f94bc02eed0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 -- 10.20.193.164:0/1150956114 shutdown_connections 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 --2- 10.20.193.164:0/1150956114 >> v2:10.20.193.128:3300/0 conn(0x7f94c811ffc0 0x7f94c8122480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 --2- 10.20.193.164:0/1150956114 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 0x7f94c811f9f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 -- 10.20.193.164:0/1150956114 >> 10.20.193.164:0/1150956114 conn(0x7f94c8073430 msgr2=0x7f94c8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 -- 10.20.193.164:0/1150956114 shutdown_connections 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 -- 10.20.193.164:0/1150956114 wait complete. 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 10 monclient: build_initial_monmap 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 10 monclient: monmap: 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:52.021 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:52.022 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:52.022 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:52.022 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.022 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:52.022 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:52.022 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:52.022 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.022 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 librados: starting msgr at 2026-02-17T05:54:52.022 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 librados: starting objecter 2026-02-17T05:54:52.022 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 Processor -- start 2026-02-17T05:54:52.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 -- start start 2026-02-17T05:54:52.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 librados: setting wanted keys 2026-02-17T05:54:52.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 1 librados: calling monclient init 2026-02-17T05:54:52.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.014+0000 7f94d0187640 10 monclient: init 2026-02-17T05:54:52.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:52.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:54:52.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f94c81149e0 0x7f94c81ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 10 monclient(hunting): picked mon.noname-b con 0x7f94c81149e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:52.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 0x7f94c81abed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 10 monclient(hunting): picked mon.noname-c con 0x7f94c811d600 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:52.024 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f94c811ffc0 0x7f94c81b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.024 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 10 monclient(hunting): picked mon.noname-a con 0x7f94c811ffc0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:52.024 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.024 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81231f0 con 0x7f94c811ffc0 2026-02-17T05:54:52.024 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.024 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8123070 con 0x7f94c81149e0 2026-02-17T05:54:52.024 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.024 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8123370 con 0x7f94c811d600 2026-02-17T05:54:52.024 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:52.024 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.015+0000 7f94d0187640 10 monclient(hunting): authenticate will time out at 952.660061s 2026-02-17T05:54:52.028 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.016+0000 7f94cd6fb640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 0x7f94c81abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.028 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.016+0000 7f94cd6fb640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 0x7f94c81abed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.164:44774/0 (socket says 10.20.193.164:44774) 2026-02-17T05:54:52.028 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.016+0000 7f94cd6fb640 1 -- 10.20.193.164:0/1707808215 learned_addr learned my addr 10.20.193.164:0/1707808215 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:52.029 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.016+0000 7f94cd6fb640 10 monclient(hunting): get_auth_request con 0x7f94c811d600 auth_method 0 2026-02-17T05:54:52.029 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.016+0000 7f94cd6fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.029 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.016+0000 7f94cd6fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.029 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.016+0000 7f94cd6fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.029 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.017+0000 7f94cd6fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.029 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.017+0000 7f94cd6fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.029 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.017+0000 7f94cd6fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.029 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.018+0000 7f94cd6fb640 10 monclient(hunting): handle_auth_done global_id 4157 payload 824 2026-02-17T05:54:52.029 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.018+0000 7f94cd6fb640 1 -- 10.20.193.164:0/1707808215 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f94c81149e0 msgr2=0x7f94c81ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.018+0000 7f94cd6fb640 1 --2- 10.20.193.164:0/1707808215 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f94c81149e0 0x7f94c81ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.018+0000 7f94cd6fb640 1 -- 10.20.193.164:0/1707808215 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f94c811ffc0 msgr2=0x7f94c81b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.018+0000 7f94cd6fb640 1 --2- 10.20.193.164:0/1707808215 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f94c811ffc0 0x7f94c81b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.018+0000 7f94cd6fb640 10 monclient: _finish_hunting 0 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.018+0000 7f94cd6fb640 1 monclient: found mon.noname-c 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.018+0000 7f94cd6fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.018+0000 7f94cd6fb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.018+0000 7f94cd6fb640 1 -- 10.20.193.164:0/1707808215 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c81b0950 con 0x7f94c811d600 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.022+0000 7f94cd6fb640 1 --2- 10.20.193.164:0/1707808215 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 0x7f94c81abed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009a70 tx=0x7f94bc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.022+0000 7f94b67fc640 10 client.?.objecter ms_handle_connect 0x7f94c811d600 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.022+0000 7f94b67fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:52.030 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94b67fc640 1 -- 10.20.193.164:0/1707808215 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc047070 con 0x7f94c811d600 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94b67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94b67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94b67fc640 10 monclient: dump: 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.031 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94b67fc640 10 monclient: _finish_auth 0 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94b67fc640 10 monclient: _check_auth_tickets 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94d0187640 5 monclient: authenticate success, global_id 4157 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94d0187640 10 monclient: _renew_subs 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94d0187640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94d0187640 1 -- 10.20.193.164:0/1707808215 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c81b0be0 con 0x7f94c811d600 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94d0187640 10 client.4157.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94d0187640 10 monclient: _renew_subs 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94d0187640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.032 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94d0187640 1 -- 10.20.193.164:0/1707808215 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c81b84d0 con 0x7f94c811d600 2026-02-17T05:54:52.033 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.024+0000 7f94d0187640 1 librados: init done 2026-02-17T05:54:52.033 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94d0187640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.033 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94d0187640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.033 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94d0187640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.033 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94d0187640 1 -- 10.20.193.164:0/1707808215 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c8123800 con 0x7f94c811d600 2026-02-17T05:54:52.033 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94b67fc640 1 -- 10.20.193.164:0/1707808215 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94bc02fbe0 con 0x7f94c811d600 2026-02-17T05:54:52.033 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94b67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:52.033 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94b67fc640 1 -- 10.20.193.164:0/1707808215 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc03c040 con 0x7f94c811d600 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94b67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94b67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94b7fff640 4 set_mon_vals no callback set 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.025+0000 7f94b67fc640 10 monclient: dump: 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.034 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.035 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.035 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.076 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.075+0000 7f94b67fc640 1 -- 10.20.193.164:0/1707808215 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 2) ==== 50049+0+0 (secure 0 0 0) 0x7f94bc042830 con 0x7f94c811d600 2026-02-17T05:54:52.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.083+0000 7f94b67fc640 1 -- 10.20.193.164:0/1707808215 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f94bc03f8e0 con 0x7f94c811d600 2026-02-17T05:54:52.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.083+0000 7f94b67fc640 10 client.4157.objecter ms_dispatch2 0x7f94c8000b90 osd_map(6..6 src has 1..6) 2026-02-17T05:54:52.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.083+0000 7f94b67fc640 3 client.4157.objecter handle_osd_map got epochs [6,6] > 0 2026-02-17T05:54:52.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.083+0000 7f94b67fc640 3 client.4157.objecter handle_osd_map decoding full epoch 6 2026-02-17T05:54:52.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.083+0000 7f94b67fc640 20 client.4157.objecter dump_active .. 0 homeless 2026-02-17T05:54:52.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.083+0000 7f94b67fc640 1 -- 10.20.193.164:0/1707808215 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f94bc052080 con 0x7f94c811d600 2026-02-17T05:54:52.086 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.083+0000 7f94b67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.086 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.083+0000 7f94b67fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:52.118 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.116+0000 7f94d0187640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "5bf489c2-40d8-490c-9c10-34a9c6fc407c", "id": 4}] 2026-02-17T05:54:52.118 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.116+0000 7f94d0187640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "5bf489c2-40d8-490c-9c10-34a9c6fc407c", "id": 4}] 2026-02-17T05:54:52.118 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.116+0000 7f94d0187640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.118 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.116+0000 7f94d0187640 1 -- 10.20.193.164:0/1707808215 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5bf489c2-40d8-490c-9c10-34a9c6fc407c", "id": 4} v 0) -- 0x7f94c81b86a0 con 0x7f94c811d600 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94b67fc640 1 -- 10.20.193.164:0/1707808215 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5bf489c2-40d8-490c-9c10-34a9c6fc407c", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f94bc04be20 con 0x7f94c811d600 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94b67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "5bf489c2-40d8-490c-9c10-34a9c6fc407c", "id": 4}] 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94b67fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stdout:4 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 10 librados: watch_flush enter 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 10 librados: watch_flush exit 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 20 client.4157.objecter shutdown clearing up homeless session... 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 10 client.4157.objecter successfully canceled tick 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 10 monclient: shutdown 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 1 -- 10.20.193.164:0/1707808215 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 msgr2=0x7f94c81abed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.216 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 1 --2- 10.20.193.164:0/1707808215 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 0x7f94c81abed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009a70 tx=0x7f94bc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.217 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 1 -- 10.20.193.164:0/1707808215 shutdown_connections 2026-02-17T05:54:52.217 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 1 --2- 10.20.193.164:0/1707808215 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f94c811ffc0 0x7f94c81b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.217 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 1 --2- 10.20.193.164:0/1707808215 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f94c811d600 0x7f94c81abed0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.217 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 1 --2- 10.20.193.164:0/1707808215 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f94c81149e0 0x7f94c81ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.217 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 1 -- 10.20.193.164:0/1707808215 >> 10.20.193.164:0/1707808215 conn(0x7f94c8073430 msgr2=0x7f94c8112b60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:52.217 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 1 -- 10.20.193.164:0/1707808215 shutdown_connections 2026-02-17T05:54:52.217 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 1 -- 10.20.193.164:0/1707808215 wait complete. 2026-02-17T05:54:52.217 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.215+0000 7f94d0187640 1 librados: shutdown 2026-02-17T05:54:52.224 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new 3b254810-f671-42ee-8ae9-45fd888d08a7 5 2026-02-17T05:54:52.296 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.285+0000 7fca0e824640 10 monclient: get_monmap_and_config 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.285+0000 7fca0e824640 10 monclient: build_initial_monmap 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.285+0000 7fca0e824640 10 monclient: monmap: 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:52.286068+0000 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:52.286068+0000 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.285+0000 7fca0e824640 1 Processor -- start 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.285+0000 7fca0e824640 1 -- start start 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.285+0000 7fca0e824640 10 monclient: init 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 10 monclient: _add_conns ranks=[1,3,5,4,0,2] 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fca080735d0 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fca080741b0 0x7fca0811f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 10 monclient(hunting): picked mon.noname-a con 0x7fca080741b0 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fca0811fff0 0x7fca081224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 10 monclient(hunting): picked mon.noname-c con 0x7fca0811fff0 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca08122bd0 con 0x7fca080735d0 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fca08123df0 con 0x7fca080741b0 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fca08123f70 con 0x7fca0811fff0 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0e824640 10 monclient(hunting): authenticate will time out at 952.931127s 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca07fff640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fca080735d0 0x7fca080739b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.164:58468/0 (socket says 10.20.193.164:58468) 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0cd9a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fca0811fff0 0x7fca081224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca07fff640 1 -- 10.20.193.164:0/110603835 learned_addr learned my addr 10.20.193.164:0/110603835 (peer_addr_for_me v1:10.20.193.164:0/0) 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0cd9a640 10 monclient(hunting): get_auth_request con 0x7fca0811fff0 auth_method 0 2026-02-17T05:54:52.297 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0cd9a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0cd9a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0cd9a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0cd9a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0cd9a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.286+0000 7fca0cd9a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.287+0000 7fca0cd9a640 10 monclient(hunting): handle_auth_done global_id 4163 payload 274 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.287+0000 7fca0cd9a640 1 -- 10.20.193.164:0/110603835 >> v2:10.20.193.16:3300/0 conn(0x7fca080741b0 msgr2=0x7fca0811f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.287+0000 7fca0cd9a640 1 --2- 10.20.193.164:0/110603835 >> v2:10.20.193.16:3300/0 conn(0x7fca080741b0 0x7fca0811f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.287+0000 7fca0cd9a640 1 -- 10.20.193.164:0/110603835 >> v1:10.20.193.128:6789/0 conn(0x7fca080735d0 legacy=0x7fca080739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.287+0000 7fca0cd9a640 10 monclient: _finish_hunting 0 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.287+0000 7fca0cd9a640 1 monclient: found mon.noname-c 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.287+0000 7fca0cd9a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.287+0000 7fca0cd9a640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.287+0000 7fca0cd9a640 1 -- 10.20.193.164:0/110603835 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca08124120 con 0x7fca0811fff0 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.288+0000 7fca0cd9a640 1 --2- 10.20.193.164:0/110603835 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca0811fff0 0x7fca081224b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fca0811ffc0 tx=0x7fc9fc02f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dac4ce45f29220 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 1 -- 10.20.193.164:0/110603835 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc03c070 con 0x7fca0811fff0 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 10 monclient: dump: 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 10 monclient: _finish_auth 0 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 1 -- 10.20.193.164:0/110603835 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9fc02fe40 con 0x7fca0811fff0 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 1 -- 10.20.193.164:0/110603835 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc037660 con 0x7fca0811fff0 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 5 monclient: authenticate success, global_id 4163 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 4 set_mon_vals no callback set 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca06ffd640 10 monclient: dump: 2026-02-17T05:54:52.298 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 10 monclient: shutdown 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 1 -- 10.20.193.164:0/110603835 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca0811fff0 msgr2=0x7fca081224b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 1 --2- 10.20.193.164:0/110603835 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca0811fff0 0x7fca081224b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fca0811ffc0 tx=0x7fc9fc02f130 comp rx=0 tx=0).stop 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 1 -- 10.20.193.164:0/110603835 shutdown_connections 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 1 --2- 10.20.193.164:0/110603835 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca0811fff0 0x7fca081224b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 1 --2- 10.20.193.164:0/110603835 >> v2:10.20.193.16:3300/0 conn(0x7fca080741b0 0x7fca0811f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 1 -- 10.20.193.164:0/110603835 >> 10.20.193.164:0/110603835 conn(0x7fca0806f550 msgr2=0x7fca08071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 1 -- 10.20.193.164:0/110603835 shutdown_connections 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 1 -- 10.20.193.164:0/110603835 wait complete. 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 10 monclient: build_initial_monmap 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 10 monclient: monmap: 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 1 librados: starting msgr at 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.290+0000 7fca0e824640 1 librados: starting objecter 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 1 Processor -- start 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 1 -- start start 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 1 librados: setting wanted keys 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 1 librados: calling monclient init 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient: init 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca080735d0 0x7fca081ab970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient(hunting): picked mon.noname-c con 0x7fca080735d0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fca080741b0 0x7fca081abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.299 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient(hunting): picked mon.noname-b con 0x7fca080741b0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fca0811fff0 0x7fca081b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient(hunting): picked mon.noname-a con 0x7fca0811fff0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca08123260 con 0x7fca0811fff0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fca081230e0 con 0x7fca080741b0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fca081233e0 con 0x7fca080735d0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca0e824640 10 monclient(hunting): authenticate will time out at 952.936272s 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca077fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fca080741b0 0x7fca081abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca07fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca080735d0 0x7fca081ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca07fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca080735d0 0x7fca081ab970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.164:33928/0 (socket says 10.20.193.164:33928) 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca07fff640 1 -- 10.20.193.164:0/1343414180 learned_addr learned my addr 10.20.193.164:0/1343414180 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca07fff640 10 monclient(hunting): get_auth_request con 0x7fca080735d0 auth_method 0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca07fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca07fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca077fe640 10 monclient(hunting): get_auth_request con 0x7fca080741b0 auth_method 0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca077fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.291+0000 7fca077fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.292+0000 7fca07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.292+0000 7fca07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.292+0000 7fca07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.292+0000 7fca077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.292+0000 7fca077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.292+0000 7fca077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.293+0000 7fca07fff640 10 monclient(hunting): handle_auth_done global_id 4166 payload 824 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.293+0000 7fca07fff640 1 -- 10.20.193.164:0/1343414180 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fca080741b0 msgr2=0x7fca081abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.293+0000 7fca07fff640 1 --2- 10.20.193.164:0/1343414180 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fca080741b0 0x7fca081abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.293+0000 7fca07fff640 1 -- 10.20.193.164:0/1343414180 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fca0811fff0 msgr2=0x7fca081b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.293+0000 7fca07fff640 1 --2- 10.20.193.164:0/1343414180 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fca0811fff0 0x7fca081b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.293+0000 7fca07fff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.293+0000 7fca07fff640 1 monclient: found mon.noname-c 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.293+0000 7fca07fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.293+0000 7fca07fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.293+0000 7fca07fff640 1 -- 10.20.193.164:0/1343414180 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca081b0920 con 0x7fca080735d0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.294+0000 7fca07fff640 1 --2- 10.20.193.164:0/1343414180 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca080735d0 0x7fca081ab970 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400c700 tx=0x7fc9f400cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.294+0000 7fca04ff9640 10 client.?.objecter ms_handle_connect 0x7fca080735d0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.294+0000 7fca04ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca04ff9640 1 -- 10.20.193.164:0/1343414180 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4017070 con 0x7fca080735d0 2026-02-17T05:54:52.300 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca04ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca04ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca04ff9640 10 monclient: dump: 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca04ff9640 10 monclient: _finish_auth 0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca04ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca04ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 5 monclient: authenticate success, global_id 4166 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 10 monclient: _renew_subs 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 1 -- 10.20.193.164:0/1343414180 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca081b0c10 con 0x7fca080735d0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 10 client.4166.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 10 monclient: _renew_subs 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 1 -- 10.20.193.164:0/1343414180 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca081b8410 con 0x7fca080735d0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 1 librados: init done 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.295+0000 7fca0e824640 1 -- 10.20.193.164:0/1343414180 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca08123870 con 0x7fca080735d0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.296+0000 7fca04ff9640 1 -- 10.20.193.164:0/1343414180 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f4005bb0 con 0x7fca080735d0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.296+0000 7fca04ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.296+0000 7fca04ff9640 1 -- 10.20.193.164:0/1343414180 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4007070 con 0x7fca080735d0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.296+0000 7fca04ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.301 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.297+0000 7fca04ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.297+0000 7fca04ff9640 10 monclient: dump: 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.302 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.297+0000 7fca067fc640 4 set_mon_vals no callback set 2026-02-17T05:54:52.357 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.356+0000 7fca04ff9640 1 -- 10.20.193.164:0/1343414180 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 2) ==== 50049+0+0 (secure 0 0 0) 0x7fc9f4007210 con 0x7fca080735d0 2026-02-17T05:54:52.360 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.359+0000 7fca04ff9640 1 -- 10.20.193.164:0/1343414180 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fc9f4007710 con 0x7fca080735d0 2026-02-17T05:54:52.360 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.359+0000 7fca04ff9640 10 client.4166.objecter ms_dispatch2 0x7fca08000b90 osd_map(7..7 src has 1..7) 2026-02-17T05:54:52.360 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.359+0000 7fca04ff9640 3 client.4166.objecter handle_osd_map got epochs [7,7] > 0 2026-02-17T05:54:52.360 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.359+0000 7fca04ff9640 3 client.4166.objecter handle_osd_map decoding full epoch 7 2026-02-17T05:54:52.360 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.359+0000 7fca04ff9640 20 client.4166.objecter dump_active .. 0 homeless 2026-02-17T05:54:52.360 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.359+0000 7fca04ff9640 1 -- 10.20.193.164:0/1343414180 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7fc9f4021020 con 0x7fca080735d0 2026-02-17T05:54:52.360 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.359+0000 7fca04ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.360 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.359+0000 7fca04ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:52.393 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.392+0000 7fca0e824640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "3b254810-f671-42ee-8ae9-45fd888d08a7", "id": 5}] 2026-02-17T05:54:52.393 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.392+0000 7fca0e824640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "3b254810-f671-42ee-8ae9-45fd888d08a7", "id": 5}] 2026-02-17T05:54:52.393 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.392+0000 7fca0e824640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.393 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.392+0000 7fca0e824640 1 -- 10.20.193.164:0/1343414180 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3b254810-f671-42ee-8ae9-45fd888d08a7", "id": 5} v 0) -- 0x7fca08119ac0 con 0x7fca080735d0 2026-02-17T05:54:52.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca04ff9640 1 -- 10.20.193.164:0/1343414180 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3b254810-f671-42ee-8ae9-45fd888d08a7", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fc9f400fce0 con 0x7fca080735d0 2026-02-17T05:54:52.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca04ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "3b254810-f671-42ee-8ae9-45fd888d08a7", "id": 5}] 2026-02-17T05:54:52.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca04ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:52.436 INFO:teuthology.orchestra.run.trial164.stdout:5 2026-02-17T05:54:52.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 10 librados: watch_flush enter 2026-02-17T05:54:52.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 10 librados: watch_flush exit 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 20 client.4166.objecter shutdown clearing up homeless session... 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 10 client.4166.objecter successfully canceled tick 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 10 monclient: shutdown 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 1 -- 10.20.193.164:0/1343414180 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca080735d0 msgr2=0x7fca081ab970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 1 --2- 10.20.193.164:0/1343414180 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca080735d0 0x7fca081ab970 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400c700 tx=0x7fc9f400cbd0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 1 -- 10.20.193.164:0/1343414180 shutdown_connections 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 1 --2- 10.20.193.164:0/1343414180 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fca080741b0 0x7fca081abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 1 --2- 10.20.193.164:0/1343414180 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fca0811fff0 0x7fca081b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 1 --2- 10.20.193.164:0/1343414180 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fca080735d0 0x7fca081ab970 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 1 -- 10.20.193.164:0/1343414180 >> 10.20.193.164:0/1343414180 conn(0x7fca0806f550 msgr2=0x7fca08071940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 1 -- 10.20.193.164:0/1343414180 shutdown_connections 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 1 -- 10.20.193.164:0/1343414180 wait complete. 2026-02-17T05:54:52.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.435+0000 7fca0e824640 1 librados: shutdown 2026-02-17T05:54:52.444 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new b8592e91-c8a9-49a0-846c-519306dc7064 6 2026-02-17T05:54:52.507 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.506+0000 7f66164fa640 10 monclient: get_monmap_and_config 2026-02-17T05:54:52.507 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.506+0000 7f66164fa640 10 monclient: build_initial_monmap 2026-02-17T05:54:52.507 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.506+0000 7f66164fa640 10 monclient: monmap: 2026-02-17T05:54:52.507 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:52.507 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.507 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:52.507205+0000 2026-02-17T05:54:52.507 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:52.507205+0000 2026-02-17T05:54:52.507 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:52.507 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.507 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:52.508 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:52.508 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:52.508 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:52.508 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:52.508 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:52.508 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 1 Processor -- start 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 1 -- start start 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient: init 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient: _add_conns ranks=[4,5,3,0,1,2] 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f6610073870 0x7f66101181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient(hunting): picked mon.noname-b con 0x7f6610073870 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6610073c50 0x7f66100740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient(hunting): picked mon.noname-c con 0x7f6610073c50 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f6610123360 0x7f66101257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient(hunting): picked mon.noname-a con 0x7f6610123360 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6610125ed0 con 0x7f6610123360 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f6610126050 con 0x7f6610073870 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6610126200 con 0x7f6610073c50 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f66164fa640 10 monclient(hunting): authenticate will time out at 953.152247s 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660f7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6610073c50 0x7f66100740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660ffff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f6610073870 0x7f66101181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660f7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6610073c50 0x7f66100740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.164:33942/0 (socket says 10.20.193.164:33942) 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660f7fe640 1 -- 10.20.193.164:0/1621701663 learned_addr learned my addr 10.20.193.164:0/1621701663 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660f7fe640 10 monclient(hunting): get_auth_request con 0x7f6610073c50 auth_method 0 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660ffff640 10 monclient(hunting): get_auth_request con 0x7f6610073870 auth_method 0 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.507+0000 7f660ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.509 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 10 monclient(hunting): handle_auth_done global_id 4169 payload 274 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 1 -- 10.20.193.164:0/1621701663 >> v2:10.20.193.128:3300/0 conn(0x7f6610073870 msgr2=0x7f66101181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 1 --2- 10.20.193.164:0/1621701663 >> v2:10.20.193.128:3300/0 conn(0x7f6610073870 0x7f66101181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 1 -- 10.20.193.164:0/1621701663 >> v2:10.20.193.16:3300/0 conn(0x7f6610123360 msgr2=0x7f66101257b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 1 --2- 10.20.193.164:0/1621701663 >> v2:10.20.193.16:3300/0 conn(0x7f6610123360 0x7f66101257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 1 monclient: found mon.noname-c 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.508+0000 7f660f7fe640 1 -- 10.20.193.164:0/1621701663 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66101263b0 con 0x7f6610073c50 2026-02-17T05:54:52.510 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.509+0000 7f660f7fe640 1 --2- 10.20.193.164:0/1621701663 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073c50 0x7f66100740a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f66100766a0 tx=0x7f660002edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8c5ac233043312 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 1 -- 10.20.193.164:0/1621701663 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660003c070 con 0x7f6610073c50 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 10 monclient: dump: 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 10 monclient: _finish_auth 0 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 1 -- 10.20.193.164:0/1621701663 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f660002fab0 con 0x7f6610073c50 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 1 -- 10.20.193.164:0/1621701663 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600037440 con 0x7f6610073c50 2026-02-17T05:54:52.512 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 5 monclient: authenticate success, global_id 4169 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f660effd640 10 monclient: dump: 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:52.513 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 4 set_mon_vals no callback set 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 10 monclient: shutdown 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 1 -- 10.20.193.164:0/1621701663 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073c50 msgr2=0x7f66100740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 1 --2- 10.20.193.164:0/1621701663 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073c50 0x7f66100740a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f66100766a0 tx=0x7f660002edf0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 1 -- 10.20.193.164:0/1621701663 shutdown_connections 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 1 --2- 10.20.193.164:0/1621701663 >> v2:10.20.193.16:3300/0 conn(0x7f6610123360 0x7f66101257b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 1 --2- 10.20.193.164:0/1621701663 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073c50 0x7f66100740a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 1 --2- 10.20.193.164:0/1621701663 >> v2:10.20.193.128:3300/0 conn(0x7f6610073870 0x7f66101181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 1 -- 10.20.193.164:0/1621701663 >> 10.20.193.164:0/1621701663 conn(0x7f661006f550 msgr2=0x7f6610071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 1 -- 10.20.193.164:0/1621701663 shutdown_connections 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.511+0000 7f66164fa640 1 -- 10.20.193.164:0/1621701663 wait complete. 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient: build_initial_monmap 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient: monmap: 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:52.514 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 librados: starting msgr at 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 librados: starting objecter 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 Processor -- start 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 -- start start 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 librados: setting wanted keys 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 librados: calling monclient init 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient: init 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073870 0x7f66101afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient(hunting): picked mon.noname-c con 0x7f6610073870 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6610073c50 0x7f66101b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient(hunting): picked mon.noname-b con 0x7f6610073c50 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6610123360 0x7f66101b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient(hunting): picked mon.noname-a con 0x7f6610123360 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6610126ac0 con 0x7f6610123360 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f6610126940 con 0x7f6610073c50 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6610126c40 con 0x7f6610073870 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f66164fa640 10 monclient(hunting): authenticate will time out at 953.157520s 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f660ffff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073870 0x7f66101afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f660f7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6610073c50 0x7f66101b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f660ffff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073870 0x7f66101afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.164:33952/0 (socket says 10.20.193.164:33952) 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f660ffff640 1 -- 10.20.193.164:0/373361472 learned_addr learned my addr 10.20.193.164:0/373361472 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f660ffff640 10 monclient(hunting): get_auth_request con 0x7f6610073870 auth_method 0 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f660ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f660ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.512+0000 7f660ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.513+0000 7f660f7fe640 10 monclient(hunting): get_auth_request con 0x7f6610073c50 auth_method 0 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.513+0000 7f660f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.515 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.513+0000 7f660f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.513+0000 7f660f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.513+0000 7f660ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.513+0000 7f660ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.513+0000 7f660ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.513+0000 7f660f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.513+0000 7f660f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.513+0000 7f660f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.514+0000 7f660ffff640 10 monclient(hunting): handle_auth_done global_id 4172 payload 824 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.514+0000 7f660ffff640 1 -- 10.20.193.164:0/373361472 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6610073c50 msgr2=0x7f66101b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.514+0000 7f660ffff640 1 --2- 10.20.193.164:0/373361472 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6610073c50 0x7f66101b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.514+0000 7f660ffff640 1 -- 10.20.193.164:0/373361472 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6610123360 msgr2=0x7f66101b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.514+0000 7f660ffff640 1 --2- 10.20.193.164:0/373361472 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6610123360 0x7f66101b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.514+0000 7f660ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.514+0000 7f660ffff640 1 monclient: found mon.noname-c 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.514+0000 7f660ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.514+0000 7f660ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.516 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.514+0000 7f660ffff640 1 -- 10.20.193.164:0/373361472 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66101b4cf0 con 0x7f6610073870 2026-02-17T05:54:52.519 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.519+0000 7f660ffff640 1 --2- 10.20.193.164:0/373361472 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073870 0x7f66101afdc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00ebe0 tx=0x7f65fc00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:52.519 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.519+0000 7f660cff9640 10 client.?.objecter ms_handle_connect 0x7f6610073870 2026-02-17T05:54:52.519 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.519+0000 7f660cff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:52.548 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 1 -- 10.20.193.164:0/373361472 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc017070 con 0x7f6610073870 2026-02-17T05:54:52.548 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.548 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:52.548 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 10 monclient: dump: 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 10 monclient: _finish_auth 0 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 10 monclient: _check_auth_tickets 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 1 -- 10.20.193.164:0/373361472 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65fc00cc40 con 0x7f6610073870 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 5 monclient: authenticate success, global_id 4172 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 10 monclient: _renew_subs 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 1 -- 10.20.193.164:0/373361472 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66101b4fe0 con 0x7f6610073870 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 1 -- 10.20.193.164:0/373361472 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc013470 con 0x7f6610073870 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660cff9640 10 monclient: dump: 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f660e7fc640 4 set_mon_vals no callback set 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 10 client.4172.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 10 monclient: _renew_subs 2026-02-17T05:54:52.549 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.550 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 1 -- 10.20.193.164:0/373361472 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66101bc900 con 0x7f6610073870 2026-02-17T05:54:52.550 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 1 librados: init done 2026-02-17T05:54:52.550 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.550 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.550 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.550 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.548+0000 7f66164fa640 1 -- 10.20.193.164:0/373361472 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66100740a0 con 0x7f6610073870 2026-02-17T05:54:52.554 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.553+0000 7f660cff9640 1 -- 10.20.193.164:0/373361472 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 2) ==== 50049+0+0 (secure 0 0 0) 0x7f65fc024060 con 0x7f6610073870 2026-02-17T05:54:52.555 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.554+0000 7f660cff9640 1 -- 10.20.193.164:0/373361472 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f65fc013d70 con 0x7f6610073870 2026-02-17T05:54:52.555 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.554+0000 7f660cff9640 10 client.4172.objecter ms_dispatch2 0x7f6610000b90 osd_map(8..8 src has 1..8) 2026-02-17T05:54:52.555 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.554+0000 7f660cff9640 3 client.4172.objecter handle_osd_map got epochs [8,8] > 0 2026-02-17T05:54:52.555 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.554+0000 7f660cff9640 3 client.4172.objecter handle_osd_map decoding full epoch 8 2026-02-17T05:54:52.555 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.554+0000 7f660cff9640 20 client.4172.objecter dump_active .. 0 homeless 2026-02-17T05:54:52.596 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.595+0000 7f660cff9640 1 -- 10.20.193.164:0/373361472 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f65fc013610 con 0x7f6610073870 2026-02-17T05:54:52.596 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.595+0000 7f660cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.596 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.595+0000 7f660cff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:52.628 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.627+0000 7f66164fa640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "b8592e91-c8a9-49a0-846c-519306dc7064", "id": 6}] 2026-02-17T05:54:52.628 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.627+0000 7f66164fa640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "b8592e91-c8a9-49a0-846c-519306dc7064", "id": 6}] 2026-02-17T05:54:52.628 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.627+0000 7f66164fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:52.628 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.627+0000 7f66164fa640 1 -- 10.20.193.164:0/373361472 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b8592e91-c8a9-49a0-846c-519306dc7064", "id": 6} v 0) -- 0x7f66101bcad0 con 0x7f6610073870 2026-02-17T05:54:52.669 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.668+0000 7f660cff9640 1 -- 10.20.193.164:0/373361472 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b8592e91-c8a9-49a0-846c-519306dc7064", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f65fc0253d0 con 0x7f6610073870 2026-02-17T05:54:52.669 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.668+0000 7f660cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "b8592e91-c8a9-49a0-846c-519306dc7064", "id": 6}] 2026-02-17T05:54:52.669 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.668+0000 7f660cff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:52.669 INFO:teuthology.orchestra.run.trial164.stdout:6 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.668+0000 7f66164fa640 10 librados: watch_flush enter 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.668+0000 7f66164fa640 10 librados: watch_flush exit 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.668+0000 7f66164fa640 20 client.4172.objecter shutdown clearing up homeless session... 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.668+0000 7f66164fa640 10 client.4172.objecter successfully canceled tick 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 10 monclient: shutdown 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 1 -- 10.20.193.164:0/373361472 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073870 msgr2=0x7f66101afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 1 --2- 10.20.193.164:0/373361472 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073870 0x7f66101afdc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00ebe0 tx=0x7f65fc00c520 comp rx=0 tx=0).stop 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 1 -- 10.20.193.164:0/373361472 shutdown_connections 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 1 --2- 10.20.193.164:0/373361472 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6610073c50 0x7f66101b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 1 --2- 10.20.193.164:0/373361472 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6610123360 0x7f66101b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 1 --2- 10.20.193.164:0/373361472 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6610073870 0x7f66101afdc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 1 -- 10.20.193.164:0/373361472 >> 10.20.193.164:0/373361472 conn(0x7f661006f550 msgr2=0x7f6610071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 1 -- 10.20.193.164:0/373361472 shutdown_connections 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 1 -- 10.20.193.164:0/373361472 wait complete. 2026-02-17T05:54:52.670 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.669+0000 7f66164fa640 1 librados: shutdown 2026-02-17T05:54:52.677 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new 00efccbd-39d3-4135-8978-22d7614c94ed 7 2026-02-17T05:54:52.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient: get_monmap_and_config 2026-02-17T05:54:52.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient: build_initial_monmap 2026-02-17T05:54:52.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient: monmap: 2026-02-17T05:54:52.745 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:52.745 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.745 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:52.740887+0000 2026-02-17T05:54:52.745 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:52.740887+0000 2026-02-17T05:54:52.745 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:52.746 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.746 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:52.746 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:52.746 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:52.746 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:52.746 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:52.746 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:52.746 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.746 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 1 Processor -- start 2026-02-17T05:54:52.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 1 -- start start 2026-02-17T05:54:52.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient: init 2026-02-17T05:54:52.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:52.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient: _add_conns ranks=[0,3,1,5,2,4] 2026-02-17T05:54:52.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f82980735d0 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:52.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f82980741b0 0x7f829811f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient(hunting): picked mon.noname-a con 0x7f82980741b0 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:52.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f829811fff0 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:52.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8298123db0 con 0x7f82980735d0 2026-02-17T05:54:52.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8298124fd0 con 0x7f829811fff0 2026-02-17T05:54:52.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f82981261d0 con 0x7f82980741b0 2026-02-17T05:54:52.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:52.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.740+0000 7f829eeb7640 10 monclient(hunting): authenticate will time out at 953.385929s 2026-02-17T05:54:52.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.741+0000 7f829d42d640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f829811fff0 0x7f82981224b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.164:58476/0 (socket says 10.20.193.164:58476) 2026-02-17T05:54:52.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.741+0000 7f829d42d640 1 -- 10.20.193.164:0/2377357286 learned_addr learned my addr 10.20.193.164:0/2377357286 (peer_addr_for_me v1:10.20.193.164:0/0) 2026-02-17T05:54:52.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.741+0000 7f828ffff640 1 --2- 10.20.193.164:0/2377357286 >> v2:10.20.193.16:3300/0 conn(0x7f82980741b0 0x7f829811f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.741+0000 7f828ffff640 10 monclient(hunting): get_auth_request con 0x7f82980741b0 auth_method 0 2026-02-17T05:54:52.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.741+0000 7f828ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.741+0000 7f828ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.741+0000 7f828ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.742+0000 7f828ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.742+0000 7f828ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.742+0000 7f828ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.742+0000 7f828f7fe640 1 -- 10.20.193.164:0/2377357286 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1421557321 0 0) 0x7f8298124fd0 con 0x7f829811fff0 2026-02-17T05:54:52.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.742+0000 7f828f7fe640 10 monclient(hunting): _negotiate 2026-02-17T05:54:52.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.742+0000 7f828f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.742+0000 7f828f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.742+0000 7f828f7fe640 10 monclient(hunting): my global_id is 4138 2026-02-17T05:54:52.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.742+0000 7f828f7fe640 1 -- 10.20.193.164:0/2377357286 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8284003620 con 0x7f829811fff0 2026-02-17T05:54:52.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.743+0000 7f828ffff640 10 monclient(hunting): handle_auth_done global_id 4143 payload 274 2026-02-17T05:54:52.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.743+0000 7f828ffff640 1 -- 10.20.193.164:0/2377357286 >> v1:10.20.193.128:6789/0 conn(0x7f829811fff0 legacy=0x7f82981224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.743+0000 7f828ffff640 1 -- 10.20.193.164:0/2377357286 >> v1:10.20.193.16:6789/0 conn(0x7f82980735d0 legacy=0x7f82980739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.743+0000 7f828ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:52.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.743+0000 7f828ffff640 1 monclient: found mon.noname-a 2026-02-17T05:54:52.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.743+0000 7f828ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:52.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.743+0000 7f828ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:52.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.743+0000 7f828ffff640 1 -- 10.20.193.164:0/2377357286 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8298126350 con 0x7f82980741b0 2026-02-17T05:54:52.759 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.745+0000 7f828ffff640 1 --2- 10.20.193.164:0/2377357286 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 0x7f829811f990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f82800098a0 tx=0x7f828002edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63b4bb484e50e62 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:52.759 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 1 -- 10.20.193.164:0/2377357286 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828003c070 con 0x7f82980741b0 2026-02-17T05:54:52.759 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.760 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:52.760 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 10 monclient: dump: 2026-02-17T05:54:52.760 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.760 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.760 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.760 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.760 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.760 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.760 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.761 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.761 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.761 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.761 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:54:52.761 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:54:52.761 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:52.761 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 5 monclient: authenticate success, global_id 4143 2026-02-17T05:54:52.761 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 1 -- 10.20.193.164:0/2377357286 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828002fae0 con 0x7f82980741b0 2026-02-17T05:54:52.761 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:52.761 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 4 set_mon_vals no callback set 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 1 -- 10.20.193.164:0/2377357286 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280037440 con 0x7f82980741b0 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f828f7fe640 10 monclient: dump: 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.762 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.763 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.763 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.763 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.763 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.763 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 10 monclient: shutdown 2026-02-17T05:54:52.763 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:52.763 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 1 -- 10.20.193.164:0/2377357286 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 msgr2=0x7f829811f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.763 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 1 --2- 10.20.193.164:0/2377357286 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 0x7f829811f990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f82800098a0 tx=0x7f828002edd0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.763 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 1 -- 10.20.193.164:0/2377357286 shutdown_connections 2026-02-17T05:54:52.763 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 1 --2- 10.20.193.164:0/2377357286 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 0x7f829811f990 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.764 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 1 -- 10.20.193.164:0/2377357286 >> 10.20.193.164:0/2377357286 conn(0x7f829806f550 msgr2=0x7f8298071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:52.764 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 1 -- 10.20.193.164:0/2377357286 shutdown_connections 2026-02-17T05:54:52.764 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.747+0000 7f829eeb7640 1 -- 10.20.193.164:0/2377357286 wait complete. 2026-02-17T05:54:52.764 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient: build_initial_monmap 2026-02-17T05:54:52.764 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient: monmap: 2026-02-17T05:54:52.764 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:52.764 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:52.764 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:52.764 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:52.764 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:52.765 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.765 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:52.765 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:52.765 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:52.765 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 librados: starting msgr at 2026-02-17T05:54:52.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 librados: starting objecter 2026-02-17T05:54:52.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 Processor -- start 2026-02-17T05:54:52.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 -- start start 2026-02-17T05:54:52.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 librados: setting wanted keys 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 librados: calling monclient init 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient: init 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f82980735d0 0x7f82981b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient(hunting): picked mon.noname-c con 0x7f82980735d0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 0x7f82981b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient(hunting): picked mon.noname-a con 0x7f82980741b0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f829811fff0 0x7f829811e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient(hunting): picked mon.noname-b con 0x7f829811fff0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:52.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8298124440 con 0x7f82980741b0 2026-02-17T05:54:52.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f82981242c0 con 0x7f829811fff0 2026-02-17T05:54:52.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:52.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f82981245c0 con 0x7f82980735d0 2026-02-17T05:54:52.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:52.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829eeb7640 10 monclient(hunting): authenticate will time out at 953.393463s 2026-02-17T05:54:52.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f828ffff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 0x7f82981b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829cc2c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f82980735d0 0x7f82981b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f828ffff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 0x7f82981b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.164:41188/0 (socket says 10.20.193.164:41188) 2026-02-17T05:54:52.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f828ffff640 1 -- 10.20.193.164:0/593220693 learned_addr learned my addr 10.20.193.164:0/593220693 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:52.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f828ffff640 10 monclient(hunting): get_auth_request con 0x7f82980741b0 auth_method 0 2026-02-17T05:54:52.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f828ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f828ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f828ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.748+0000 7f829d42d640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f829811fff0 0x7f829811e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829cc2c640 10 monclient(hunting): get_auth_request con 0x7f82980735d0 auth_method 0 2026-02-17T05:54:52.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829cc2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829cc2c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829cc2c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f828ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f828ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f828ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829d42d640 10 monclient(hunting): get_auth_request con 0x7f829811fff0 auth_method 0 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829d42d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829d42d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829d42d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829cc2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829cc2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829cc2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829d42d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829d42d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.749+0000 7f829d42d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.750+0000 7f828ffff640 10 monclient(hunting): handle_auth_done global_id 4149 payload 824 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.750+0000 7f828ffff640 1 -- 10.20.193.164:0/593220693 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f82980735d0 msgr2=0x7f82981b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.750+0000 7f828ffff640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f82980735d0 0x7f82981b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.750+0000 7f828ffff640 1 -- 10.20.193.164:0/593220693 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f829811fff0 msgr2=0x7f829811e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.750+0000 7f828ffff640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f829811fff0 0x7f829811e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.750+0000 7f828ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.750+0000 7f828ffff640 1 monclient: found mon.noname-a 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.750+0000 7f828ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:52.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.750+0000 7f828ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.750+0000 7f828ffff640 1 -- 10.20.193.164:0/593220693 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829811ebd0 con 0x7f82980741b0 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.751+0000 7f828ffff640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 0x7f82981b1c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8280009130 tx=0x7f82800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.751+0000 7f828d7fa640 10 client.?.objecter ms_handle_connect 0x7f82980741b0 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.751+0000 7f828d7fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 1 -- 10.20.193.164:0/593220693 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280046070 con 0x7f82980741b0 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 10 monclient: dump: 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.771 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.772 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.772 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.772 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.772 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.772 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.772 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.772 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.772 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 10 monclient: _finish_auth 0 2026-02-17T05:54:52.772 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 10 monclient: _check_auth_tickets 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 1 -- 10.20.193.164:0/593220693 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8280042670 con 0x7f82980741b0 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 1 -- 10.20.193.164:0/593220693 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828003c040 con 0x7f82980741b0 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f829eeb7640 5 monclient: authenticate success, global_id 4149 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828effd640 4 set_mon_vals no callback set 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f829eeb7640 10 monclient: _renew_subs 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f829eeb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f829eeb7640 1 -- 10.20.193.164:0/593220693 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829811ee00 con 0x7f82980741b0 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:52.773 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f828d7fa640 10 monclient: dump: 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:52.774 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f829eeb7640 10 client.4149.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:52.775 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f829eeb7640 10 monclient: _renew_subs 2026-02-17T05:54:52.775 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f829eeb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:52.775 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f829eeb7640 1 -- 10.20.193.164:0/593220693 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82981b8450 con 0x7f82980741b0 2026-02-17T05:54:52.775 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.752+0000 7f829eeb7640 1 librados: init done 2026-02-17T05:54:52.775 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.753+0000 7f829eeb7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.775 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.753+0000 7f829eeb7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.775 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.753+0000 7f829eeb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:52.775 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.753+0000 7f829eeb7640 1 -- 10.20.193.164:0/593220693 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8298124aa0 con 0x7f82980741b0 2026-02-17T05:54:52.776 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.759+0000 7f828d7fa640 1 -- 10.20.193.164:0/593220693 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 2) ==== 50049+0+0 (secure 0 0 0) 0x7f82800413b0 con 0x7f82980741b0 2026-02-17T05:54:52.776 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.760+0000 7f828d7fa640 1 -- 10.20.193.164:0/593220693 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f8280049360 con 0x7f82980741b0 2026-02-17T05:54:52.776 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.760+0000 7f828d7fa640 10 client.4149.objecter ms_dispatch2 0x7f8298000b90 osd_map(9..9 src has 1..9) 2026-02-17T05:54:52.777 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.760+0000 7f828d7fa640 3 client.4149.objecter handle_osd_map got epochs [9,9] > 0 2026-02-17T05:54:52.777 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.760+0000 7f828d7fa640 3 client.4149.objecter handle_osd_map decoding full epoch 9 2026-02-17T05:54:52.777 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.760+0000 7f828d7fa640 20 client.4149.objecter dump_active .. 0 homeless 2026-02-17T05:54:52.800 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.799+0000 7f828d7fa640 1 -- 10.20.193.164:0/593220693 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f82800416d0 con 0x7f82980741b0 2026-02-17T05:54:52.800 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.799+0000 7f828d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:52.801 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.799+0000 7f828d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:52.832 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.831+0000 7f829eeb7640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "00efccbd-39d3-4135-8978-22d7614c94ed", "id": 7}] 2026-02-17T05:54:52.832 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.831+0000 7f829eeb7640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "00efccbd-39d3-4135-8978-22d7614c94ed", "id": 7}] 2026-02-17T05:54:52.832 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.831+0000 7f829eeb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:52.832 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.831+0000 7f829eeb7640 1 -- 10.20.193.164:0/593220693 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00efccbd-39d3-4135-8978-22d7614c94ed", "id": 7} v 0) -- 0x7f82980725e0 con 0x7f82980741b0 2026-02-17T05:54:52.961 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.959+0000 7f828d7fa640 1 -- 10.20.193.164:0/593220693 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mgrmap(e 3) ==== 99895+0+0 (secure 0 0 0) 0x7f8280051020 con 0x7f82980741b0 2026-02-17T05:54:52.962 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.960+0000 7f828d7fa640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8268079200 0x7f826807b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:52.962 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.960+0000 7f829cc2c640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8268079200 0x7f826807b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:52.962 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.960+0000 7f829cc2c640 10 monclient: get_auth_request con 0x7f8268079200 auth_method 0 2026-02-17T05:54:52.963 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.961+0000 7f829cc2c640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8268079200 0x7f826807b5f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8288002770 tx=0x7f8288007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:52.963 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:52.961+0000 7f828d7fa640 10 client.4149.objecter ms_handle_connect 0x7f8268079200 2026-02-17T05:54:53.023 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:53.023+0000 7fcc3d48e640 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-17T05:54:53.066 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f828d7fa640 1 -- 10.20.193.164:0/593220693 <== mon.0 v2:10.20.193.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00efccbd-39d3-4135-8978-22d7614c94ed", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f8280089c70 con 0x7f82980741b0 2026-02-17T05:54:53.067 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f828d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "00efccbd-39d3-4135-8978-22d7614c94ed", "id": 7}] 2026-02-17T05:54:53.067 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f828d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:53.067 INFO:teuthology.orchestra.run.trial164.stdout:7 2026-02-17T05:54:53.069 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 10 librados: watch_flush enter 2026-02-17T05:54:53.069 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 10 librados: watch_flush exit 2026-02-17T05:54:53.069 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 20 client.4149.objecter shutdown clearing up homeless session... 2026-02-17T05:54:53.069 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 10 client.4149.objecter successfully canceled tick 2026-02-17T05:54:53.069 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 -- 10.20.193.164:0/593220693 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8268079200 msgr2=0x7f826807b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8268079200 0x7f826807b5f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8288002770 tx=0x7f8288007ef0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 10 monclient: shutdown 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 -- 10.20.193.164:0/593220693 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 msgr2=0x7f82981b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 0x7f82981b1c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8280009130 tx=0x7f82800047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 -- 10.20.193.164:0/593220693 shutdown_connections 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8268079200 0x7f826807b5f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f829811fff0 0x7f829811e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f82980741b0 0x7f82981b1c10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 --2- 10.20.193.164:0/593220693 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f82980735d0 0x7f82981b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.070 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 -- 10.20.193.164:0/593220693 >> 10.20.193.164:0/593220693 conn(0x7f829806f550 msgr2=0x7f829806f930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:53.071 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 -- 10.20.193.164:0/593220693 shutdown_connections 2026-02-17T05:54:53.071 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 -- 10.20.193.164:0/593220693 wait complete. 2026-02-17T05:54:53.071 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.066+0000 7f829eeb7640 1 librados: shutdown 2026-02-17T05:54:53.076 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new 540bfc0b-4904-4286-b210-1c06f78cd6e6 8 2026-02-17T05:54:53.087 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T05:54:53.087+0000 7fcc3d48e640 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.140+0000 7f9c9a70a640 10 monclient: get_monmap_and_config 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.140+0000 7f9c9a70a640 10 monclient: build_initial_monmap 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.140+0000 7f9c9a70a640 10 monclient: monmap: 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:53.140988+0000 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:53.140988+0000 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:53.141 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:53.142 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.142 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.140+0000 7f9c9a70a640 1 Processor -- start 2026-02-17T05:54:53.142 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.140+0000 7f9c9a70a640 1 -- start start 2026-02-17T05:54:53.142 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.140+0000 7f9c9a70a640 10 monclient: init 2026-02-17T05:54:53.142 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.140+0000 7f9c9a70a640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.140+0000 7f9c9a70a640 10 monclient: _add_conns ranks=[1,4,3,0,5,2] 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.140+0000 7f9c9a70a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f9c940735d0 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f9c940741b0 0x7f9c9411f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 10 monclient(hunting): picked mon.noname-b con 0x7f9c940741b0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f9c9411fff0 0x7f9c941224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 10 monclient(hunting): picked mon.noname-a con 0x7f9c9411fff0 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c94122bd0 con 0x7f9c940735d0 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9c94123df0 con 0x7f9c9411fff0 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f9c94123f70 con 0x7f9c940741b0 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c9a70a640 10 monclient(hunting): authenticate will time out at 953.786072s 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c93fff640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f9c940735d0 0x7f9c940739b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.164:58490/0 (socket says 10.20.193.164:58490) 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c93fff640 1 -- 10.20.193.164:0/2366429783 learned_addr learned my addr 10.20.193.164:0/2366429783 (peer_addr_for_me v1:10.20.193.164:0/0) 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c98c80640 1 --2- 10.20.193.164:0/2366429783 >> v2:10.20.193.16:3300/0 conn(0x7f9c9411fff0 0x7f9c941224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c98c80640 10 monclient(hunting): get_auth_request con 0x7f9c9411fff0 auth_method 0 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c98c80640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c98c80640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c98c80640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.141+0000 7f9c937fe640 1 --2- 10.20.193.164:0/2366429783 >> v2:10.20.193.128:3300/0 conn(0x7f9c940741b0 0x7f9c9411f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.143 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c937fe640 10 monclient(hunting): get_auth_request con 0x7f9c940741b0 auth_method 0 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c937fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 10 monclient(hunting): handle_auth_done global_id 4152 payload 274 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 1 -- 10.20.193.164:0/2366429783 >> v2:10.20.193.128:3300/0 conn(0x7f9c940741b0 msgr2=0x7f9c9411f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 1 --2- 10.20.193.164:0/2366429783 >> v2:10.20.193.128:3300/0 conn(0x7f9c940741b0 0x7f9c9411f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 1 -- 10.20.193.164:0/2366429783 >> v1:10.20.193.128:6789/0 conn(0x7f9c940735d0 legacy=0x7f9c940739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 10 monclient: _finish_hunting 0 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 1 monclient: found mon.noname-a 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.142+0000 7f9c98c80640 1 -- 10.20.193.164:0/2366429783 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c94124120 con 0x7f9c9411fff0 2026-02-17T05:54:53.144 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.143+0000 7f9c98c80640 1 --2- 10.20.193.164:0/2366429783 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c9411fff0 0x7f9c941224b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9c94074180 tx=0x7f9c8802f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb3649718570693 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 1 -- 10.20.193.164:0/2366429783 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8803c070 con 0x7f9c9411fff0 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 10 monclient: dump: 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 10 monclient: _finish_auth 0 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 1 -- 10.20.193.164:0/2366429783 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8802fe40 con 0x7f9c9411fff0 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 5 monclient: authenticate success, global_id 4152 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:53.146 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 4 set_mon_vals no callback set 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 1 -- 10.20.193.164:0/2366429783 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88037660 con 0x7f9c9411fff0 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c92ffd640 10 monclient: dump: 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.148 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 10 monclient: shutdown 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2366429783 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c9411fff0 msgr2=0x7f9c941224b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 1 --2- 10.20.193.164:0/2366429783 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c9411fff0 0x7f9c941224b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9c94074180 tx=0x7f9c8802f130 comp rx=0 tx=0).stop 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2366429783 shutdown_connections 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 1 --2- 10.20.193.164:0/2366429783 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c9411fff0 0x7f9c941224b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 1 --2- 10.20.193.164:0/2366429783 >> v2:10.20.193.128:3300/0 conn(0x7f9c940741b0 0x7f9c9411f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2366429783 >> 10.20.193.164:0/2366429783 conn(0x7f9c9406f550 msgr2=0x7f9c94071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2366429783 shutdown_connections 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.145+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2366429783 wait complete. 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient: build_initial_monmap 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient: monmap: 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 librados: starting msgr at 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 librados: starting objecter 2026-02-17T05:54:53.149 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 Processor -- start 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 -- start start 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 librados: setting wanted keys 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 librados: calling monclient init 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient: init 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c940735d0 0x7f9c941b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient(hunting): picked mon.noname-a con 0x7f9c940735d0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c940741b0 0x7f9c941b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient(hunting): picked mon.noname-b con 0x7f9c940741b0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c9411fff0 0x7f9c941ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient(hunting): picked mon.noname-c con 0x7f9c9411fff0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94123260 con 0x7f9c940735d0 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941230e0 con 0x7f9c940741b0 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941233e0 con 0x7f9c9411fff0 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c9a70a640 10 monclient(hunting): authenticate will time out at 953.791490s 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c93fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c940735d0 0x7f9c941b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c93fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c940735d0 0x7f9c941b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.164:41206/0 (socket says 10.20.193.164:41206) 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c93fff640 1 -- 10.20.193.164:0/2318389726 learned_addr learned my addr 10.20.193.164:0/2318389726 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c93fff640 10 monclient(hunting): get_auth_request con 0x7f9c940735d0 auth_method 0 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c93fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.150 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c93fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.146+0000 7f9c937fe640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c940741b0 0x7f9c941b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c98c80640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c9411fff0 0x7f9c941ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c937fe640 10 monclient(hunting): get_auth_request con 0x7f9c940741b0 auth_method 0 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c937fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c98c80640 10 monclient(hunting): get_auth_request con 0x7f9c9411fff0 auth_method 0 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c98c80640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c98c80640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c98c80640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c98c80640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c98c80640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.151 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.147+0000 7f9c98c80640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.152 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.148+0000 7f9c93fff640 10 monclient(hunting): handle_auth_done global_id 4155 payload 824 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.148+0000 7f9c93fff640 1 -- 10.20.193.164:0/2318389726 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c9411fff0 msgr2=0x7f9c941ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.148+0000 7f9c93fff640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c9411fff0 0x7f9c941ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.148+0000 7f9c93fff640 1 -- 10.20.193.164:0/2318389726 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c940741b0 msgr2=0x7f9c941b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.148+0000 7f9c93fff640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c940741b0 0x7f9c941b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.148+0000 7f9c93fff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.148+0000 7f9c93fff640 1 monclient: found mon.noname-a 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.148+0000 7f9c93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.148+0000 7f9c93fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.148+0000 7f9c93fff640 1 -- 10.20.193.164:0/2318389726 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c941abcf0 con 0x7f9c940735d0 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.149+0000 7f9c93fff640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c940735d0 0x7f9c941b1380 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c700 tx=0x7f9c8000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.149+0000 7f9c90ff9640 10 client.?.objecter ms_handle_connect 0x7f9c940735d0 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.149+0000 7f9c90ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 1 -- 10.20.193.164:0/2318389726 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80017070 con 0x7f9c940735d0 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 10 monclient: dump: 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.153 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 10 monclient: _finish_auth 0 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 1 -- 10.20.193.164:0/2318389726 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c80005bb0 con 0x7f9c940735d0 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 1 -- 10.20.193.164:0/2318389726 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80007070 con 0x7f9c940735d0 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c927fc640 4 set_mon_vals no callback set 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c90ff9640 10 monclient: dump: 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 5 monclient: authenticate success, global_id 4155 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 10 monclient: _renew_subs 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2318389726 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c941abf80 con 0x7f9c940735d0 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 10 client.4155.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 10 monclient: _renew_subs 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2318389726 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941b8310 con 0x7f9c940735d0 2026-02-17T05:54:53.154 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 1 librados: init done 2026-02-17T05:54:53.155 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:53.155 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:53.155 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.155 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.151+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2318389726 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c941237f0 con 0x7f9c940735d0 2026-02-17T05:54:53.162 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.162+0000 7f9c90ff9640 1 -- 10.20.193.164:0/2318389726 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 3) ==== 99895+0+0 (secure 0 0 0) 0x7f9c80024080 con 0x7f9c940735d0 2026-02-17T05:54:53.163 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.162+0000 7f9c90ff9640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c64078f90 0x7f9c6407b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.163 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.162+0000 7f9c937fe640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c64078f90 0x7f9c6407b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.163 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.162+0000 7f9c937fe640 10 monclient: get_auth_request con 0x7f9c64078f90 auth_method 0 2026-02-17T05:54:53.164 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.163+0000 7f9c90ff9640 10 client.4155.objecter ms_handle_connect 0x7f9c64078f90 2026-02-17T05:54:53.164 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.163+0000 7f9c937fe640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c64078f90 0x7f9c6407b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c941b0f10 tx=0x7f9c84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:53.164 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.163+0000 7f9c90ff9640 1 -- 10.20.193.164:0/2318389726 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f9c8005d730 con 0x7f9c940735d0 2026-02-17T05:54:53.164 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.163+0000 7f9c90ff9640 10 client.4155.objecter ms_dispatch2 0x7f9c94000b90 osd_map(10..10 src has 1..10) 2026-02-17T05:54:53.164 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.163+0000 7f9c90ff9640 3 client.4155.objecter handle_osd_map got epochs [10,10] > 0 2026-02-17T05:54:53.164 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.163+0000 7f9c90ff9640 3 client.4155.objecter handle_osd_map decoding full epoch 10 2026-02-17T05:54:53.164 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.163+0000 7f9c90ff9640 20 client.4155.objecter dump_active .. 0 homeless 2026-02-17T05:54:53.266 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.264+0000 7f9c90ff9640 1 -- 10.20.193.164:0/2318389726 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9c800616f0 con 0x7f9c940735d0 2026-02-17T05:54:53.266 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.264+0000 7f9c90ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:53.266 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.264+0000 7f9c90ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:53.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.352+0000 7f9c9a70a640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "540bfc0b-4904-4286-b210-1c06f78cd6e6", "id": 8}] 2026-02-17T05:54:53.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.352+0000 7f9c9a70a640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "540bfc0b-4904-4286-b210-1c06f78cd6e6", "id": 8}] 2026-02-17T05:54:53.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.352+0000 7f9c9a70a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.352+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2318389726 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "540bfc0b-4904-4286-b210-1c06f78cd6e6", "id": 8} v 0) -- 0x7f9c94119ac0 con 0x7f9c940735d0 2026-02-17T05:54:53.372 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c90ff9640 1 -- 10.20.193.164:0/2318389726 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "540bfc0b-4904-4286-b210-1c06f78cd6e6", "id": 8}]=0 v11) ==== 112+0+2 (secure 0 0 0) 0x7f9c94119ac0 con 0x7f9c940735d0 2026-02-17T05:54:53.373 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c90ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "540bfc0b-4904-4286-b210-1c06f78cd6e6", "id": 8}] 2026-02-17T05:54:53.373 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c90ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:53.373 INFO:teuthology.orchestra.run.trial164.stdout:8 2026-02-17T05:54:53.375 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 10 librados: watch_flush enter 2026-02-17T05:54:53.375 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 10 librados: watch_flush exit 2026-02-17T05:54:53.375 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 20 client.4155.objecter shutdown clearing up homeless session... 2026-02-17T05:54:53.375 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 10 client.4155.objecter successfully canceled tick 2026-02-17T05:54:53.375 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2318389726 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c64078f90 msgr2=0x7f9c6407b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c64078f90 0x7f9c6407b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c941b0f10 tx=0x7f9c84008040 comp rx=0 tx=0).stop 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 10 monclient: shutdown 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2318389726 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c940735d0 msgr2=0x7f9c941b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c940735d0 0x7f9c941b1380 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c700 tx=0x7f9c8000cbd0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2318389726 shutdown_connections 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c64078f90 0x7f9c6407b450 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c940741b0 0x7f9c941b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c9411fff0 0x7f9c941ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 --2- 10.20.193.164:0/2318389726 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c940735d0 0x7f9c941b1380 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.376 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2318389726 >> 10.20.193.164:0/2318389726 conn(0x7f9c9406f550 msgr2=0x7f9c94121250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:53.377 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2318389726 shutdown_connections 2026-02-17T05:54:53.377 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 -- 10.20.193.164:0/2318389726 wait complete. 2026-02-17T05:54:53.377 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.371+0000 7f9c9a70a640 1 librados: shutdown 2026-02-17T05:54:53.381 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new dd5ed63a-41d2-4afc-86dd-7834e0c16448 9 2026-02-17T05:54:53.444 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.442+0000 7fef85843640 10 monclient: get_monmap_and_config 2026-02-17T05:54:53.444 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.442+0000 7fef85843640 10 monclient: build_initial_monmap 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.442+0000 7fef85843640 10 monclient: monmap: 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:53.444272+0000 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:53.444272+0000 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:53.445 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 1 Processor -- start 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 1 -- start start 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 10 monclient: init 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 10 monclient: _add_conns ranks=[3,5,2,4,1,0] 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fef80073870 0x7fef801181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 10 monclient(hunting): picked mon.noname-a con 0x7fef80073870 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fef80073c50 0x7fef800740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 10 monclient(hunting): picked mon.noname-c con 0x7fef80073c50 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fef80123360 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef801270b0 con 0x7fef80123360 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fef801282d0 con 0x7fef80073870 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fef80128450 con 0x7fef80073c50 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef85843640 10 monclient(hunting): authenticate will time out at 954.089338s 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef7effd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fef80073870 0x7fef801181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef7e7fc640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fef80073c50 0x7fef800740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef7effd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fef80073870 0x7fef801181f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.164:41216/0 (socket says 10.20.193.164:41216) 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef7effd640 1 -- 10.20.193.164:0/3559896992 learned_addr learned my addr 10.20.193.164:0/3559896992 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef7effd640 10 monclient(hunting): get_auth_request con 0x7fef80073870 auth_method 0 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef7effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef7effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.443+0000 7fef7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.446 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7e7fc640 10 monclient(hunting): get_auth_request con 0x7fef80073c50 auth_method 0 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 10 monclient(hunting): handle_auth_done global_id 4158 payload 274 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 1 -- 10.20.193.164:0/3559896992 >> v2:10.20.193.148:3300/0 conn(0x7fef80073c50 msgr2=0x7fef800740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 1 --2- 10.20.193.164:0/3559896992 >> v2:10.20.193.148:3300/0 conn(0x7fef80073c50 0x7fef800740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 1 -- 10.20.193.164:0/3559896992 >> v1:10.20.193.148:6789/0 conn(0x7fef80123360 legacy=0x7fef801257b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 10 monclient: _finish_hunting 0 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 1 monclient: found mon.noname-a 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.444+0000 7fef7effd640 1 -- 10.20.193.164:0/3559896992 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef80128600 con 0x7fef80073870 2026-02-17T05:54:53.447 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.445+0000 7fef7effd640 1 --2- 10.20.193.164:0/3559896992 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073870 0x7fef801181f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fef8006c060 tx=0x7fef7402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab67f24bbd0b6955 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:53.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 1 -- 10.20.193.164:0/3559896992 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7403c070 con 0x7fef80073870 2026-02-17T05:54:53.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:53.449 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 10 monclient: dump: 2026-02-17T05:54:53.449 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.449 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.449 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 10 monclient: _finish_auth 0 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 1 -- 10.20.193.164:0/3559896992 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef7402fb30 con 0x7fef80073870 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 5 monclient: authenticate success, global_id 4158 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 1 -- 10.20.193.164:0/3559896992 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74037440 con 0x7fef80073870 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef7dffb640 10 monclient: dump: 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.450 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 4 set_mon_vals no callback set 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 10 monclient: shutdown 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 1 -- 10.20.193.164:0/3559896992 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073870 msgr2=0x7fef801181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 1 --2- 10.20.193.164:0/3559896992 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073870 0x7fef801181f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fef8006c060 tx=0x7fef7402ee60 comp rx=0 tx=0).stop 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 1 -- 10.20.193.164:0/3559896992 shutdown_connections 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 1 --2- 10.20.193.164:0/3559896992 >> v2:10.20.193.148:3300/0 conn(0x7fef80073c50 0x7fef800740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 1 --2- 10.20.193.164:0/3559896992 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073870 0x7fef801181f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 1 -- 10.20.193.164:0/3559896992 >> 10.20.193.164:0/3559896992 conn(0x7fef8006f550 msgr2=0x7fef80071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.447+0000 7fef85843640 1 -- 10.20.193.164:0/3559896992 shutdown_connections 2026-02-17T05:54:53.451 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 -- 10.20.193.164:0/3559896992 wait complete. 2026-02-17T05:54:53.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient: build_initial_monmap 2026-02-17T05:54:53.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient: monmap: 2026-02-17T05:54:53.452 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:53.452 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:53.452 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:53.452 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:53.452 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:53.452 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.452 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 librados: starting msgr at 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 librados: starting objecter 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 Processor -- start 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 -- start start 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 librados: setting wanted keys 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 librados: calling monclient init 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient: init 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fef80073870 0x7fef801afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient(hunting): picked mon.noname-b con 0x7fef80073870 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073c50 0x7fef801b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient(hunting): picked mon.noname-a con 0x7fef80073c50 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fef80123360 0x7fef801b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient(hunting): picked mon.noname-c con 0x7fef80123360 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef80127740 con 0x7fef80073c50 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fef801275c0 con 0x7fef80073870 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fef801278c0 con 0x7fef80123360 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.448+0000 7fef85843640 10 monclient(hunting): authenticate will time out at 954.094849s 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7e7fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073c50 0x7fef801b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7f7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fef80123360 0x7fef801b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7effd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fef80073870 0x7fef801afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7e7fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073c50 0x7fef801b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.164:41230/0 (socket says 10.20.193.164:41230) 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7e7fc640 1 -- 10.20.193.164:0/616778341 learned_addr learned my addr 10.20.193.164:0/616778341 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7e7fc640 10 monclient(hunting): get_auth_request con 0x7fef80073c50 auth_method 0 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7f7fe640 10 monclient(hunting): get_auth_request con 0x7fef80123360 auth_method 0 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7effd640 10 monclient(hunting): get_auth_request con 0x7fef80073870 auth_method 0 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.449+0000 7fef7effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.450+0000 7fef7e7fc640 10 monclient(hunting): handle_auth_done global_id 4161 payload 824 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.450+0000 7fef7e7fc640 1 -- 10.20.193.164:0/616778341 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fef80123360 msgr2=0x7fef801b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.450+0000 7fef7e7fc640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fef80123360 0x7fef801b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.450+0000 7fef7e7fc640 1 -- 10.20.193.164:0/616778341 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fef80073870 msgr2=0x7fef801afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.450+0000 7fef7e7fc640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fef80073870 0x7fef801afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.450+0000 7fef7e7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.450+0000 7fef7e7fc640 1 monclient: found mon.noname-a 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.450+0000 7fef7e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.450+0000 7fef7e7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.450+0000 7fef7e7fc640 1 -- 10.20.193.164:0/616778341 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef801b4cd0 con 0x7fef80073c50 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.451+0000 7fef7e7fc640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073c50 0x7fef801b02e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fef6800ebe0 tx=0x7fef6800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.451+0000 7fef57fff640 10 client.?.objecter ms_handle_connect 0x7fef80073c50 2026-02-17T05:54:53.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.451+0000 7fef57fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 1 -- 10.20.193.164:0/616778341 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68017070 con 0x7fef80073c50 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 10 monclient: dump: 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 10 monclient: _finish_auth 0 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 10 monclient: _check_auth_tickets 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 1 -- 10.20.193.164:0/616778341 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6800cc40 con 0x7fef80073c50 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 1 -- 10.20.193.164:0/616778341 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68013470 con 0x7fef80073c50 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef57fff640 10 monclient: dump: 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.455 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef7d7fa640 4 set_mon_vals no callback set 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 5 monclient: authenticate success, global_id 4161 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 10 monclient: _renew_subs 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 1 -- 10.20.193.164:0/616778341 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef801b4fc0 con 0x7fef80073c50 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 10 client.4161.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 10 monclient: _renew_subs 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 1 -- 10.20.193.164:0/616778341 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801bc8e0 con 0x7fef80073c50 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 1 librados: init done 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.456 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.453+0000 7fef85843640 1 -- 10.20.193.164:0/616778341 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef800740a0 con 0x7fef80073c50 2026-02-17T05:54:53.466 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.465+0000 7fef57fff640 1 -- 10.20.193.164:0/616778341 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 3) ==== 99895+0+0 (secure 0 0 0) 0x7fef68024060 con 0x7fef80073c50 2026-02-17T05:54:53.467 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.465+0000 7fef57fff640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef50078f90 0x7fef5007b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.467 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.465+0000 7fef7effd640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef50078f90 0x7fef5007b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.467 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.465+0000 7fef7effd640 10 monclient: get_auth_request con 0x7fef50078f90 auth_method 0 2026-02-17T05:54:53.467 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.466+0000 7fef57fff640 10 client.4161.objecter ms_handle_connect 0x7fef50078f90 2026-02-17T05:54:53.467 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.466+0000 7fef7effd640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef50078f90 0x7fef5007b450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fef801188a0 tx=0x7fef74039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:53.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.466+0000 7fef57fff640 1 -- 10.20.193.164:0/616778341 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1760+0+0 (secure 0 0 0) 0x7fef6805e410 con 0x7fef80073c50 2026-02-17T05:54:53.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.466+0000 7fef57fff640 10 client.4161.objecter ms_dispatch2 0x7fef80000b90 osd_map(11..11 src has 1..11) 2026-02-17T05:54:53.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.466+0000 7fef57fff640 3 client.4161.objecter handle_osd_map got epochs [11,11] > 0 2026-02-17T05:54:53.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.466+0000 7fef57fff640 3 client.4161.objecter handle_osd_map decoding full epoch 11 2026-02-17T05:54:53.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.466+0000 7fef57fff640 20 client.4161.objecter dump_active .. 0 homeless 2026-02-17T05:54:53.570 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.568+0000 7fef57fff640 1 -- 10.20.193.164:0/616778341 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fef680623d0 con 0x7fef80073c50 2026-02-17T05:54:53.570 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.568+0000 7fef57fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:53.570 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.568+0000 7fef57fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:53.659 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.657+0000 7fef85843640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "dd5ed63a-41d2-4afc-86dd-7834e0c16448", "id": 9}] 2026-02-17T05:54:53.659 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.657+0000 7fef85843640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "dd5ed63a-41d2-4afc-86dd-7834e0c16448", "id": 9}] 2026-02-17T05:54:53.659 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.657+0000 7fef85843640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:53.659 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.657+0000 7fef85843640 1 -- 10.20.193.164:0/616778341 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dd5ed63a-41d2-4afc-86dd-7834e0c16448", "id": 9} v 0) -- 0x7fef801bcab0 con 0x7fef80073c50 2026-02-17T05:54:53.675 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.673+0000 7fef57fff640 1 -- 10.20.193.164:0/616778341 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd5ed63a-41d2-4afc-86dd-7834e0c16448", "id": 9}]=0 v12) ==== 112+0+2 (secure 0 0 0) 0x7fef680666d0 con 0x7fef80073c50 2026-02-17T05:54:53.675 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.673+0000 7fef57fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "dd5ed63a-41d2-4afc-86dd-7834e0c16448", "id": 9}] 2026-02-17T05:54:53.675 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.673+0000 7fef57fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:53.675 INFO:teuthology.orchestra.run.trial164.stdout:9 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 10 librados: watch_flush enter 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 10 librados: watch_flush exit 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 20 client.4161.objecter shutdown clearing up homeless session... 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 10 client.4161.objecter successfully canceled tick 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 -- 10.20.193.164:0/616778341 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef50078f90 msgr2=0x7fef5007b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef50078f90 0x7fef5007b450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fef801188a0 tx=0x7fef74039040 comp rx=0 tx=0).stop 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 10 monclient: shutdown 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 -- 10.20.193.164:0/616778341 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073c50 msgr2=0x7fef801b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073c50 0x7fef801b02e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fef6800ebe0 tx=0x7fef6800c520 comp rx=0 tx=0).stop 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 -- 10.20.193.164:0/616778341 shutdown_connections 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef50078f90 0x7fef5007b450 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fef80123360 0x7fef801b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef80073c50 0x7fef801b02e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 --2- 10.20.193.164:0/616778341 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fef80073870 0x7fef801afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 -- 10.20.193.164:0/616778341 >> 10.20.193.164:0/616778341 conn(0x7fef8006f550 msgr2=0x7fef80123780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 -- 10.20.193.164:0/616778341 shutdown_connections 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 -- 10.20.193.164:0/616778341 wait complete. 2026-02-17T05:54:53.676 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.674+0000 7fef85843640 1 librados: shutdown 2026-02-17T05:54:53.683 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new d425da5d-7acd-48f9-abf7-fb4896df492d 10 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 10 monclient: get_monmap_and_config 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 10 monclient: build_initial_monmap 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 10 monclient: monmap: 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:53.747961+0000 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:53.747961+0000 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:53.748 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:53.749 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:53.749 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:53.749 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:53.749 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 1 Processor -- start 2026-02-17T05:54:53.749 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 1 -- start start 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 10 monclient: init 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 10 monclient: _add_conns ranks=[2,5,4,0,3,1] 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fad44073a00 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fad4411b490 0x7fad4411d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 10 monclient(hunting): picked mon.noname-c con 0x7fad4411b490 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fad4411ddc0 0x7fad441201b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.746+0000 7fad4b2b7640 10 monclient(hunting): picked mon.noname-b con 0x7fad4411ddc0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4b2b7640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad441208d0 con 0x7fad44073a00 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4b2b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4b2b7640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fad44121af0 con 0x7fad4411ddc0 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4b2b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4b2b7640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fad44121c70 con 0x7fad4411b490 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4b2b7640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4b2b7640 10 monclient(hunting): authenticate will time out at 954.393014s 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4982d640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fad4411ddc0 0x7fad441201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4982d640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fad4411ddc0 0x7fad441201b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.164:39712/0 (socket says 10.20.193.164:39712) 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4982d640 1 -- 10.20.193.164:0/3475539495 learned_addr learned my addr 10.20.193.164:0/3475539495 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4982d640 10 monclient(hunting): get_auth_request con 0x7fad4411ddc0 auth_method 0 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4982d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4982d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4982d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4982d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4982d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.750 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.747+0000 7fad4982d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.751 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.748+0000 7fad4982d640 10 monclient(hunting): handle_auth_done global_id 4156 payload 274 2026-02-17T05:54:53.751 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.748+0000 7fad4982d640 1 -- 10.20.193.164:0/3475539495 >> v2:10.20.193.148:3300/0 conn(0x7fad4411b490 msgr2=0x7fad4411d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:53.751 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.748+0000 7fad4982d640 1 --2- 10.20.193.164:0/3475539495 >> v2:10.20.193.148:3300/0 conn(0x7fad4411b490 0x7fad4411d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.751 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.748+0000 7fad4982d640 1 -- 10.20.193.164:0/3475539495 >> v1:10.20.193.148:6789/0 conn(0x7fad44073a00 legacy=0x7fad44073de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:53.751 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.748+0000 7fad4982d640 10 monclient: _finish_hunting 0 2026-02-17T05:54:53.751 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.748+0000 7fad4982d640 1 monclient: found mon.noname-b 2026-02-17T05:54:53.751 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.748+0000 7fad4982d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:53.751 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.748+0000 7fad4982d640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:54:53.751 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.748+0000 7fad4982d640 1 -- 10.20.193.164:0/3475539495 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad44121e20 con 0x7fad4411ddc0 2026-02-17T05:54:53.758 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.756+0000 7fad4982d640 1 --2- 10.20.193.164:0/3475539495 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad4411ddc0 0x7fad441201b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fad440745b0 tx=0x7fad4002f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e03510a076dc665d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:53.764 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 1 -- 10.20.193.164:0/3475539495 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4002fc50 con 0x7fad4411ddc0 2026-02-17T05:54:53.764 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.764 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:53.764 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 10 monclient: dump: 2026-02-17T05:54:53.764 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.764 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.764 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.764 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 10 monclient: _finish_auth 0 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 10 monclient: _check_auth_tickets 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 1 -- 10.20.193.164:0/3475539495 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad4002fdf0 con 0x7fad4411ddc0 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 1 -- 10.20.193.164:0/3475539495 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40037610 con 0x7fad4411ddc0 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 5 monclient: authenticate success, global_id 4156 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad33fff640 10 monclient: dump: 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 4 set_mon_vals no callback set 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 10 monclient: shutdown 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:53.765 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3475539495 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad4411ddc0 msgr2=0x7fad441201b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 1 --2- 10.20.193.164:0/3475539495 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad4411ddc0 0x7fad441201b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fad440745b0 tx=0x7fad4002f230 comp rx=0 tx=0).stop 2026-02-17T05:54:53.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3475539495 shutdown_connections 2026-02-17T05:54:53.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 1 --2- 10.20.193.164:0/3475539495 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad4411ddc0 0x7fad441201b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 1 --2- 10.20.193.164:0/3475539495 >> v2:10.20.193.148:3300/0 conn(0x7fad4411b490 0x7fad4411d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3475539495 >> 10.20.193.164:0/3475539495 conn(0x7fad4406f550 msgr2=0x7fad44071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:53.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3475539495 shutdown_connections 2026-02-17T05:54:53.766 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.762+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3475539495 wait complete. 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient: build_initial_monmap 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient: monmap: 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:53.767 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 librados: starting msgr at 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 librados: starting objecter 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 Processor -- start 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 -- start start 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 librados: setting wanted keys 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 librados: calling monclient init 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient: init 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad44073a00 0x7fad44112270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient(hunting): picked mon.noname-b con 0x7fad44073a00 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fad4411b490 0x7fad441127b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient(hunting): picked mon.noname-c con 0x7fad4411b490 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fad4411ddc0 0x7fad441b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient(hunting): picked mon.noname-a con 0x7fad4411ddc0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad44120f00 con 0x7fad4411ddc0 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fad44120d80 con 0x7fad44073a00 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad44121080 con 0x7fad4411b490 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4b2b7640 10 monclient(hunting): authenticate will time out at 954.409509s 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4902c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad44073a00 0x7fad44112270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4902c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad44073a00 0x7fad44112270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.164:39718/0 (socket says 10.20.193.164:39718) 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4902c640 1 -- 10.20.193.164:0/3027581354 learned_addr learned my addr 10.20.193.164:0/3027581354 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4902c640 10 monclient(hunting): get_auth_request con 0x7fad44073a00 auth_method 0 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4902c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4902c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.768 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4902c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.763+0000 7fad4882b640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fad4411b490 0x7fad441127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.764+0000 7fad4902c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.764+0000 7fad4902c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.764+0000 7fad4902c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.764+0000 7fad4882b640 10 monclient(hunting): get_auth_request con 0x7fad4411b490 auth_method 0 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.764+0000 7fad4882b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.764+0000 7fad4882b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.764+0000 7fad4882b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.764+0000 7fad4882b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.764+0000 7fad4882b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.764+0000 7fad4882b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.765+0000 7fad4902c640 10 monclient(hunting): handle_auth_done global_id 4159 payload 824 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.765+0000 7fad4902c640 1 -- 10.20.193.164:0/3027581354 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fad4411b490 msgr2=0x7fad441127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.765+0000 7fad4902c640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fad4411b490 0x7fad441127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.765+0000 7fad4902c640 1 -- 10.20.193.164:0/3027581354 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fad4411ddc0 msgr2=0x7fad441b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.765+0000 7fad4902c640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fad4411ddc0 0x7fad441b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.765+0000 7fad4902c640 10 monclient: _finish_hunting 0 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.765+0000 7fad4902c640 1 monclient: found mon.noname-b 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.765+0000 7fad4902c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.765+0000 7fad4902c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:54:53.769 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.765+0000 7fad4902c640 1 -- 10.20.193.164:0/3027581354 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad441b7ca0 con 0x7fad44073a00 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.766+0000 7fad4902c640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad44073a00 0x7fad44112270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fad3400b570 tx=0x7fad3400ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.766+0000 7fad31ffb640 10 client.?.objecter ms_handle_connect 0x7fad44073a00 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.766+0000 7fad31ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad31ffb640 1 -- 10.20.193.164:0/3027581354 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34013020 con 0x7fad44073a00 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad31ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad31ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad31ffb640 10 monclient: dump: 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.770 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad31ffb640 10 monclient: _finish_auth 0 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad31ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad31ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 5 monclient: authenticate success, global_id 4159 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 10 monclient: _renew_subs 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3027581354 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad441b7e70 con 0x7fad44073a00 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 10 client.4159.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 10 monclient: _renew_subs 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3027581354 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad441b82a0 con 0x7fad44073a00 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 1 librados: init done 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.767+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3027581354 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad44121500 con 0x7fad44073a00 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.768+0000 7fad31ffb640 1 -- 10.20.193.164:0/3027581354 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad34004480 con 0x7fad44073a00 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.768+0000 7fad31ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.768+0000 7fad31ffb640 1 -- 10.20.193.164:0/3027581354 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34009380 con 0x7fad44073a00 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.768+0000 7fad31ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.768+0000 7fad31ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.768+0000 7fad31ffb640 10 monclient: dump: 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:53.771 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:53.772 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:53.772 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.768+0000 7fad337fe640 4 set_mon_vals no callback set 2026-02-17T05:54:53.877 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.874+0000 7fad31ffb640 1 -- 10.20.193.164:0/3027581354 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 3) ==== 99895+0+0 (secure 0 0 0) 0x7fad3400f460 con 0x7fad44073a00 2026-02-17T05:54:53.877 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.875+0000 7fad31ffb640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fad14078f90 0x7fad1407b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:53.877 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.875+0000 7fad4882b640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fad14078f90 0x7fad1407b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:53.878 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.875+0000 7fad4882b640 10 monclient: get_auth_request con 0x7fad14078f90 auth_method 0 2026-02-17T05:54:53.878 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.876+0000 7fad4882b640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fad14078f90 0x7fad1407b450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fad38004640 tx=0x7fad38009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:53.878 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.876+0000 7fad31ffb640 10 client.4159.objecter ms_handle_connect 0x7fad14078f90 2026-02-17T05:54:53.884 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.883+0000 7fad31ffb640 1 -- 10.20.193.164:0/3027581354 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 1875+0+0 (secure 0 0 0) 0x7fad34004630 con 0x7fad44073a00 2026-02-17T05:54:53.885 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.883+0000 7fad31ffb640 10 client.4159.objecter ms_dispatch2 0x7fad44000b90 osd_map(12..12 src has 1..12) 2026-02-17T05:54:53.885 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.883+0000 7fad31ffb640 3 client.4159.objecter handle_osd_map got epochs [12,12] > 0 2026-02-17T05:54:53.885 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.883+0000 7fad31ffb640 3 client.4159.objecter handle_osd_map decoding full epoch 12 2026-02-17T05:54:53.885 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.883+0000 7fad31ffb640 20 client.4159.objecter dump_active .. 0 homeless 2026-02-17T05:54:53.885 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.883+0000 7fad31ffb640 1 -- 10.20.193.164:0/3027581354 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fad3401d020 con 0x7fad44073a00 2026-02-17T05:54:53.885 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.883+0000 7fad31ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:53.885 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.883+0000 7fad31ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:53.972 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.969+0000 7fad4b2b7640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "d425da5d-7acd-48f9-abf7-fb4896df492d", "id": 10}] 2026-02-17T05:54:53.972 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.969+0000 7fad4b2b7640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "d425da5d-7acd-48f9-abf7-fb4896df492d", "id": 10}] 2026-02-17T05:54:53.972 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.969+0000 7fad4b2b7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:53.972 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:53.969+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3027581354 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d425da5d-7acd-48f9-abf7-fb4896df492d", "id": 10} v 0) -- 0x7fad441155b0 con 0x7fad44073a00 2026-02-17T05:54:54.020 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad31ffb640 1 -- 10.20.193.164:0/3027581354 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d425da5d-7acd-48f9-abf7-fb4896df492d", "id": 10}]=0 v13) ==== 113+0+3 (secure 0 0 0) 0x7fad34062740 con 0x7fad44073a00 2026-02-17T05:54:54.020 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad31ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "d425da5d-7acd-48f9-abf7-fb4896df492d", "id": 10}] 2026-02-17T05:54:54.021 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad31ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:54.021 INFO:teuthology.orchestra.run.trial164.stdout:10 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 10 librados: watch_flush enter 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 10 librados: watch_flush exit 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 20 client.4159.objecter shutdown clearing up homeless session... 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 10 client.4159.objecter successfully canceled tick 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3027581354 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fad14078f90 msgr2=0x7fad1407b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fad14078f90 0x7fad1407b450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fad38004640 tx=0x7fad38009210 comp rx=0 tx=0).stop 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 10 monclient: shutdown 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3027581354 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad44073a00 msgr2=0x7fad44112270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad44073a00 0x7fad44112270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fad3400b570 tx=0x7fad3400ba40 comp rx=0 tx=0).stop 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3027581354 shutdown_connections 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fad14078f90 0x7fad1407b450 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fad4411ddc0 0x7fad441b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fad4411b490 0x7fad441127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 --2- 10.20.193.164:0/3027581354 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fad44073a00 0x7fad44112270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3027581354 >> 10.20.193.164:0/3027581354 conn(0x7fad4406f550 msgr2=0x7fad4411e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3027581354 shutdown_connections 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.019+0000 7fad4b2b7640 1 -- 10.20.193.164:0/3027581354 wait complete. 2026-02-17T05:54:54.023 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.020+0000 7fad4b2b7640 1 librados: shutdown 2026-02-17T05:54:54.031 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new 32bc4c0f-3014-4c48-b6c7-f1f09db76d35 11 2026-02-17T05:54:54.100 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient: get_monmap_and_config 2026-02-17T05:54:54.100 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient: build_initial_monmap 2026-02-17T05:54:54.101 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient: monmap: 2026-02-17T05:54:54.101 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:54.101 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.101 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:54.095755+0000 2026-02-17T05:54:54.101 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:54.095755+0000 2026-02-17T05:54:54.101 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:54.101 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.101 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:54.101 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:54.101 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 1 Processor -- start 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 1 -- start start 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient: init 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient: _add_conns ranks=[4,5,3,0,2,1] 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f0f800735d0 0x7f0f800739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient(hunting): picked mon.noname-b con 0x7f0f800735d0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:54.102 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f0f80073fe0 0x7f0f8011e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient(hunting): picked mon.noname-c con 0x7f0f80073fe0 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f0f8011eec0 0x7f0f801212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient(hunting): picked mon.noname-a con 0x7f0f8011eec0 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0f801219f0 con 0x7f0f8011eec0 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f0f80121b70 con 0x7f0f800735d0 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f0f80121d20 con 0x7f0f80073fe0 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.094+0000 7f0f866a7640 10 monclient(hunting): authenticate will time out at 954.740788s 2026-02-17T05:54:54.103 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7ffff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f0f800735d0 0x7f0f800739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7f7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f0f80073fe0 0x7f0f8011e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7ffff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f0f800735d0 0x7f0f800739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.164:39732/0 (socket says 10.20.193.164:39732) 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7ffff640 1 -- 10.20.193.164:0/1050068164 learned_addr learned my addr 10.20.193.164:0/1050068164 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7ffff640 10 monclient(hunting): get_auth_request con 0x7f0f800735d0 auth_method 0 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f84c1d640 1 --2- 10.20.193.164:0/1050068164 >> v2:10.20.193.16:3300/0 conn(0x7f0f8011eec0 0x7f0f801212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7f7fe640 10 monclient(hunting): get_auth_request con 0x7f0f80073fe0 auth_method 0 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.104 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f84c1d640 10 monclient(hunting): get_auth_request con 0x7f0f8011eec0 auth_method 0 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f84c1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f84c1d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f84c1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f84c1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f84c1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.105 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.095+0000 7f0f84c1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7ffff640 10 monclient(hunting): handle_auth_done global_id 4162 payload 274 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7ffff640 1 -- 10.20.193.164:0/1050068164 >> v2:10.20.193.148:3300/0 conn(0x7f0f80073fe0 msgr2=0x7f0f8011e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7ffff640 1 --2- 10.20.193.164:0/1050068164 >> v2:10.20.193.148:3300/0 conn(0x7f0f80073fe0 0x7f0f8011e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7ffff640 1 -- 10.20.193.164:0/1050068164 >> v2:10.20.193.16:3300/0 conn(0x7f0f8011eec0 msgr2=0x7f0f801212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7ffff640 1 --2- 10.20.193.164:0/1050068164 >> v2:10.20.193.16:3300/0 conn(0x7f0f8011eec0 0x7f0f801212d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7ffff640 1 monclient: found mon.noname-b 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7ffff640 1 -- 10.20.193.164:0/1050068164 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f80121ed0 con 0x7f0f800735d0 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.096+0000 7f0f7f7fe640 1 --2- 10.20.193.164:0/1050068164 >> v2:10.20.193.148:3300/0 conn(0x7f0f80073fe0 0x7f0f8011e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:54:54.106 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.097+0000 7f0f7ffff640 1 --2- 10.20.193.164:0/1050068164 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0f800735d0 0x7f0f800739b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f800766a0 tx=0x7f0f6c02edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11b10b5b35fd2d1a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:54.114 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 1 -- 10.20.193.164:0/1050068164 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c03c070 con 0x7f0f800735d0 2026-02-17T05:54:54.114 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.114 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:54.114 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 10 monclient: dump: 2026-02-17T05:54:54.114 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.114 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.114 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.114 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 10 monclient: _finish_auth 0 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 1 -- 10.20.193.164:0/1050068164 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6c02fab0 con 0x7f0f800735d0 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 5 monclient: authenticate success, global_id 4162 2026-02-17T05:54:54.115 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 1 -- 10.20.193.164:0/1050068164 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c037440 con 0x7f0f800735d0 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 4 set_mon_vals no callback set 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f7effd640 10 monclient: dump: 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.116 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 10 monclient: shutdown 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 -- 10.20.193.164:0/1050068164 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0f800735d0 msgr2=0x7f0f800739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 --2- 10.20.193.164:0/1050068164 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0f800735d0 0x7f0f800739b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f800766a0 tx=0x7f0f6c02edf0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 -- 10.20.193.164:0/1050068164 shutdown_connections 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 --2- 10.20.193.164:0/1050068164 >> v2:10.20.193.16:3300/0 conn(0x7f0f8011eec0 0x7f0f801212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.117 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 --2- 10.20.193.164:0/1050068164 >> v2:10.20.193.148:3300/0 conn(0x7f0f80073fe0 0x7f0f8011e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 --2- 10.20.193.164:0/1050068164 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0f800735d0 0x7f0f800739b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 -- 10.20.193.164:0/1050068164 >> 10.20.193.164:0/1050068164 conn(0x7f0f8006f550 msgr2=0x7f0f80071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 -- 10.20.193.164:0/1050068164 shutdown_connections 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 -- 10.20.193.164:0/1050068164 wait complete. 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 10 monclient: build_initial_monmap 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 10 monclient: monmap: 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:54.118 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 librados: starting msgr at 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.099+0000 7f0f866a7640 1 librados: starting objecter 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 1 Processor -- start 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 1 -- start start 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 1 librados: setting wanted keys 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 1 librados: calling monclient init 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient: init 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:54.119 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0f800735d0 0x7f0f801b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient(hunting): picked mon.noname-a con 0x7f0f800735d0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0f80073fe0 0x7f0f801b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient(hunting): picked mon.noname-c con 0x7f0f80073fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0f8011eec0 0x7f0f801ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient(hunting): picked mon.noname-b con 0x7f0f8011eec0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f801225e0 con 0x7f0f800735d0 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80122460 con 0x7f0f8011eec0 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.120 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80122760 con 0x7f0f80073fe0 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f866a7640 10 monclient(hunting): authenticate will time out at 954.746355s 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7f7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0f80073fe0 0x7f0f801b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f84c1d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0f8011eec0 0x7f0f801ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7ffff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0f800735d0 0x7f0f801b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f84c1d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0f8011eec0 0x7f0f801ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.164:39736/0 (socket says 10.20.193.164:39736) 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7f7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0f80073fe0 0x7f0f801b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.164:34042/0 (socket says 10.20.193.164:34042) 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f84c1d640 1 -- 10.20.193.164:0/3356726976 learned_addr learned my addr 10.20.193.164:0/3356726976 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f84c1d640 10 monclient(hunting): get_auth_request con 0x7f0f8011eec0 auth_method 0 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f84c1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f84c1d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.121 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f84c1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7f7fe640 10 monclient(hunting): get_auth_request con 0x7f0f80073fe0 auth_method 0 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7ffff640 10 monclient(hunting): get_auth_request con 0x7f0f800735d0 auth_method 0 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.100+0000 7f0f7ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.101+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.101+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.101+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.122 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.101+0000 7f0f84c1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.101+0000 7f0f84c1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.101+0000 7f0f84c1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.101+0000 7f0f7ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.101+0000 7f0f7ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.101+0000 7f0f7ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.102+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_done global_id 4196 payload 824 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.102+0000 7f0f7f7fe640 1 -- 10.20.193.164:0/3356726976 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0f8011eec0 msgr2=0x7f0f801ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.102+0000 7f0f7f7fe640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0f8011eec0 0x7f0f801ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.102+0000 7f0f7f7fe640 1 -- 10.20.193.164:0/3356726976 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0f800735d0 msgr2=0x7f0f801b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.102+0000 7f0f7f7fe640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0f800735d0 0x7f0f801b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.102+0000 7f0f7f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:54:54.123 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.102+0000 7f0f7f7fe640 1 monclient: found mon.noname-c 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.102+0000 7f0f7f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.102+0000 7f0f7f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.102+0000 7f0f7f7fe640 1 -- 10.20.193.164:0/3356726976 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f801abdc0 con 0x7f0f80073fe0 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.103+0000 7f0f7f7fe640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0f80073fe0 0x7f0f801b1ac0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000ebe0 tx=0x7f0f7000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.103+0000 7f0f7cff9640 10 client.?.objecter ms_handle_connect 0x7f0f80073fe0 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.103+0000 7f0f7cff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f7cff9640 1 -- 10.20.193.164:0/3356726976 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70017070 con 0x7f0f80073fe0 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f7cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f7cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f7cff9640 10 monclient: dump: 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.124 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f7cff9640 10 monclient: _finish_auth 0 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f7cff9640 10 monclient: _check_auth_tickets 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f7cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f866a7640 5 monclient: authenticate success, global_id 4196 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f866a7640 10 monclient: _renew_subs 2026-02-17T05:54:54.125 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f866a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f866a7640 1 -- 10.20.193.164:0/3356726976 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f801ac050 con 0x7f0f80073fe0 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f866a7640 10 client.4196.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f866a7640 10 monclient: _renew_subs 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f866a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f866a7640 1 -- 10.20.193.164:0/3356726976 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f801b83e0 con 0x7f0f80073fe0 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f866a7640 1 librados: init done 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f7cff9640 1 -- 10.20.193.164:0/3356726976 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f700093e0 con 0x7f0f80073fe0 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f7cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f7e7fc640 4 set_mon_vals no callback set 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.104+0000 7f0f866a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.105+0000 7f0f7cff9640 1 -- 10.20.193.164:0/3356726976 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70021060 con 0x7f0f80073fe0 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.105+0000 7f0f866a7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.105+0000 7f0f866a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:54.126 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.105+0000 7f0f866a7640 1 -- 10.20.193.164:0/3356726976 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f800739b0 con 0x7f0f80073fe0 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.105+0000 7f0f7cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.105+0000 7f0f7cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.105+0000 7f0f7cff9640 10 monclient: dump: 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.127 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.211 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.209+0000 7f0f7cff9640 1 -- 10.20.193.164:0/3356726976 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 3) ==== 99895+0+0 (secure 0 0 0) 0x7f0f70004020 con 0x7f0f80073fe0 2026-02-17T05:54:54.212 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.209+0000 7f0f7cff9640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0f50078f90 0x7f0f5007b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.212 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.210+0000 7f0f7ffff640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0f50078f90 0x7f0f5007b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.212 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.210+0000 7f0f7ffff640 10 monclient: get_auth_request con 0x7f0f50078f90 auth_method 0 2026-02-17T05:54:54.213 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.210+0000 7f0f7ffff640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0f50078f90 0x7f0f5007b450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f80074510 tx=0x7f0f6c039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:54.213 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.210+0000 7f0f7cff9640 10 client.4196.objecter ms_handle_connect 0x7f0f50078f90 2026-02-17T05:54:54.218 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.217+0000 7f0f7cff9640 1 -- 10.20.193.164:0/3356726976 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 1990+0+0 (secure 0 0 0) 0x7f0f7005e0f0 con 0x7f0f80073fe0 2026-02-17T05:54:54.219 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.217+0000 7f0f7cff9640 10 client.4196.objecter ms_dispatch2 0x7f0f80000b90 osd_map(13..13 src has 1..13) 2026-02-17T05:54:54.219 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.217+0000 7f0f7cff9640 3 client.4196.objecter handle_osd_map got epochs [13,13] > 0 2026-02-17T05:54:54.219 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.217+0000 7f0f7cff9640 3 client.4196.objecter handle_osd_map decoding full epoch 13 2026-02-17T05:54:54.219 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.217+0000 7f0f7cff9640 20 client.4196.objecter dump_active .. 0 homeless 2026-02-17T05:54:54.219 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.217+0000 7f0f7cff9640 1 -- 10.20.193.164:0/3356726976 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0f700620b0 con 0x7f0f80073fe0 2026-02-17T05:54:54.219 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.217+0000 7f0f7cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:54.219 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.217+0000 7f0f7cff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:54.307 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.305+0000 7f0f866a7640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "32bc4c0f-3014-4c48-b6c7-f1f09db76d35", "id": 11}] 2026-02-17T05:54:54.307 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.305+0000 7f0f866a7640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "32bc4c0f-3014-4c48-b6c7-f1f09db76d35", "id": 11}] 2026-02-17T05:54:54.307 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.305+0000 7f0f866a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:54.308 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.305+0000 7f0f866a7640 1 -- 10.20.193.164:0/3356726976 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "32bc4c0f-3014-4c48-b6c7-f1f09db76d35", "id": 11} v 0) -- 0x7f0f801b8610 con 0x7f0f80073fe0 2026-02-17T05:54:54.353 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.351+0000 7f0f7cff9640 1 -- 10.20.193.164:0/3356726976 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "32bc4c0f-3014-4c48-b6c7-f1f09db76d35", "id": 11}]=0 v14) ==== 113+0+3 (secure 0 0 0) 0x7f0f700663b0 con 0x7f0f80073fe0 2026-02-17T05:54:54.353 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.351+0000 7f0f7cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "32bc4c0f-3014-4c48-b6c7-f1f09db76d35", "id": 11}] 2026-02-17T05:54:54.353 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.351+0000 7f0f7cff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:54.353 INFO:teuthology.orchestra.run.trial164.stdout:11 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 10 librados: watch_flush enter 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 10 librados: watch_flush exit 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 20 client.4196.objecter shutdown clearing up homeless session... 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 10 client.4196.objecter successfully canceled tick 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 -- 10.20.193.164:0/3356726976 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0f50078f90 msgr2=0x7f0f5007b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0f50078f90 0x7f0f5007b450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f80074510 tx=0x7f0f6c039040 comp rx=0 tx=0).stop 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 10 monclient: shutdown 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 -- 10.20.193.164:0/3356726976 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0f80073fe0 msgr2=0x7f0f801b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0f80073fe0 0x7f0f801b1ac0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000ebe0 tx=0x7f0f7000c520 comp rx=0 tx=0).stop 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 -- 10.20.193.164:0/3356726976 shutdown_connections 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0f50078f90 0x7f0f5007b450 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0f8011eec0 0x7f0f801ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0f80073fe0 0x7f0f801b1ac0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 --2- 10.20.193.164:0/3356726976 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0f800735d0 0x7f0f801b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 -- 10.20.193.164:0/3356726976 >> 10.20.193.164:0/3356726976 conn(0x7f0f8006f550 msgr2=0x7f0f80071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 -- 10.20.193.164:0/3356726976 shutdown_connections 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 -- 10.20.193.164:0/3356726976 wait complete. 2026-02-17T05:54:54.354 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.352+0000 7f0f866a7640 1 librados: shutdown 2026-02-17T05:54:54.364 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new 9e9e82f6-9b4c-44cd-a18e-7089cc27b440 12 2026-02-17T05:54:54.433 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.427+0000 7f3a7c2fa640 10 monclient: get_monmap_and_config 2026-02-17T05:54:54.433 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.427+0000 7f3a7c2fa640 10 monclient: build_initial_monmap 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.427+0000 7f3a7c2fa640 10 monclient: monmap: 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:54.429118+0000 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:54.429118+0000 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:54.434 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:54.435 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:54.435 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.435 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.427+0000 7f3a7c2fa640 1 Processor -- start 2026-02-17T05:54:54.435 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.427+0000 7f3a7c2fa640 1 -- start start 2026-02-17T05:54:54.435 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.427+0000 7f3a7c2fa640 10 monclient: init 2026-02-17T05:54:54.435 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:54.435 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 10 monclient: _add_conns ranks=[3,5,4,1,2,0] 2026-02-17T05:54:54.435 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f3a7406c870 0x7f3a7406cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.435 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 10 monclient(hunting): picked mon.noname-a con 0x7f3a7406c870 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:54.435 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f3a7406d190 0x7f3a7411ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 10 monclient(hunting): picked mon.noname-c con 0x7f3a7406d190 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f3a7411f010 0x7f3a74121400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 10 monclient(hunting): picked mon.noname-b con 0x7f3a7411f010 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3a741219d0 con 0x7f3a7406c870 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f3a74121b50 con 0x7f3a7411f010 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f3a74121cd0 con 0x7f3a7406d190 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7c2fa640 10 monclient(hunting): authenticate will time out at 955.074281s 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a06f640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f3a7406c870 0x7f3a7406cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.436 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a870640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f3a7411f010 0x7f3a74121400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7986e640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f3a7406d190 0x7f3a7411ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a870640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f3a7411f010 0x7f3a74121400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.164:39752/0 (socket says 10.20.193.164:39752) 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a870640 1 -- 10.20.193.164:0/2380308731 learned_addr learned my addr 10.20.193.164:0/2380308731 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a870640 10 monclient(hunting): get_auth_request con 0x7f3a7411f010 auth_method 0 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a870640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a870640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a06f640 10 monclient(hunting): get_auth_request con 0x7f3a7406c870 auth_method 0 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a06f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a06f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.428+0000 7f3a7a06f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7986e640 10 monclient(hunting): get_auth_request con 0x7f3a7406d190 auth_method 0 2026-02-17T05:54:54.437 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7986e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7986e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7986e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a870640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a870640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a870640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7986e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7986e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7986e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 10 monclient(hunting): handle_auth_done global_id 4170 payload 274 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 1 -- 10.20.193.164:0/2380308731 >> v2:10.20.193.148:3300/0 conn(0x7f3a7406d190 msgr2=0x7f3a7411ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:54.438 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 1 --2- 10.20.193.164:0/2380308731 >> v2:10.20.193.148:3300/0 conn(0x7f3a7406d190 0x7f3a7411ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 1 -- 10.20.193.164:0/2380308731 >> v2:10.20.193.128:3300/0 conn(0x7f3a7411f010 msgr2=0x7f3a74121400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:54.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 1 --2- 10.20.193.164:0/2380308731 >> v2:10.20.193.128:3300/0 conn(0x7f3a7411f010 0x7f3a74121400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 10 monclient: _finish_hunting 0 2026-02-17T05:54:54.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 1 monclient: found mon.noname-a 2026-02-17T05:54:54.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:54.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:54.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a06f640 1 -- 10.20.193.164:0/2380308731 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a74121e80 con 0x7f3a7406c870 2026-02-17T05:54:54.439 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.429+0000 7f3a7a870640 1 --2- 10.20.193.164:0/2380308731 >> v2:10.20.193.128:3300/0 conn(0x7f3a7411f010 0x7f3a74121400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:54:54.440 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.430+0000 7f3a7a06f640 1 --2- 10.20.193.164:0/2380308731 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406c870 0x7f3a7406cc50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3a7406d820 tx=0x7f3a6402ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9a22d438633d28 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:54.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 1 -- 10.20.193.164:0/2380308731 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6403c070 con 0x7f3a7406c870 2026-02-17T05:54:54.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:54.450 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 10 monclient: dump: 2026-02-17T05:54:54.450 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.450 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.451 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.451 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.451 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.451 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.451 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.451 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.451 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.451 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 10 monclient: _finish_auth 0 2026-02-17T05:54:54.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 10 monclient: _check_auth_tickets 2026-02-17T05:54:54.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:54.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 1 -- 10.20.193.164:0/2380308731 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6402f9f0 con 0x7f3a7406c870 2026-02-17T05:54:54.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 5 monclient: authenticate success, global_id 4170 2026-02-17T05:54:54.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:54.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:54.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 1 -- 10.20.193.164:0/2380308731 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6402fdc0 con 0x7f3a7406c870 2026-02-17T05:54:54.452 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:54.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 4 set_mon_vals no callback set 2026-02-17T05:54:54.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:54.453 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7906d640 10 monclient: dump: 2026-02-17T05:54:54.453 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.453 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.453 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.453 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.453 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.454 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.454 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.454 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.454 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.454 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 10 monclient: shutdown 2026-02-17T05:54:54.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:54.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2380308731 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406c870 msgr2=0x7f3a7406cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.454 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 1 --2- 10.20.193.164:0/2380308731 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406c870 0x7f3a7406cc50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3a7406d820 tx=0x7f3a6402ed30 comp rx=0 tx=0).stop 2026-02-17T05:54:54.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2380308731 shutdown_connections 2026-02-17T05:54:54.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 1 --2- 10.20.193.164:0/2380308731 >> v2:10.20.193.128:3300/0 conn(0x7f3a7411f010 0x7f3a74121400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 1 --2- 10.20.193.164:0/2380308731 >> v2:10.20.193.148:3300/0 conn(0x7f3a7406d190 0x7f3a7411ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 1 --2- 10.20.193.164:0/2380308731 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406c870 0x7f3a7406cc50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2380308731 >> 10.20.193.164:0/2380308731 conn(0x7f3a74071610 msgr2=0x7f3a74073a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:54.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2380308731 shutdown_connections 2026-02-17T05:54:54.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.432+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2380308731 wait complete. 2026-02-17T05:54:54.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient: build_initial_monmap 2026-02-17T05:54:54.455 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient: monmap: 2026-02-17T05:54:54.456 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:54.456 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:54.456 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:54.456 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:54.456 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:54.456 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.456 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:54.456 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:54.456 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:54.457 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.457 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 librados: starting msgr at 2026-02-17T05:54:54.457 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 librados: starting objecter 2026-02-17T05:54:54.457 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 Processor -- start 2026-02-17T05:54:54.457 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 -- start start 2026-02-17T05:54:54.457 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 librados: setting wanted keys 2026-02-17T05:54:54.457 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 librados: calling monclient init 2026-02-17T05:54:54.457 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient: init 2026-02-17T05:54:54.457 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:54.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:54:54.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3a7406c870 0x7f3a741b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient(hunting): picked mon.noname-c con 0x7f3a7406c870 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:54.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406d190 0x7f3a741b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient(hunting): picked mon.noname-a con 0x7f3a7406d190 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:54.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3a7411f010 0x7f3a741ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient(hunting): picked mon.noname-b con 0x7f3a7411f010 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:54.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.458 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a741225b0 con 0x7f3a7406d190 2026-02-17T05:54:54.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74122430 con 0x7f3a7411f010 2026-02-17T05:54:54.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74122730 con 0x7f3a7406c870 2026-02-17T05:54:54.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:54.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7c2fa640 10 monclient(hunting): authenticate will time out at 955.079586s 2026-02-17T05:54:54.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7986e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406d190 0x7f3a741b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.433+0000 7f3a7a06f640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3a7406c870 0x7f3a741b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.459 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7986e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406d190 0x7f3a741b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.164:41264/0 (socket says 10.20.193.164:41264) 2026-02-17T05:54:54.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7986e640 1 -- 10.20.193.164:0/2217129054 learned_addr learned my addr 10.20.193.164:0/2217129054 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:54.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7986e640 10 monclient(hunting): get_auth_request con 0x7f3a7406d190 auth_method 0 2026-02-17T05:54:54.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7986e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7986e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7986e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a870640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3a7411f010 0x7f3a741ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a870640 10 monclient(hunting): get_auth_request con 0x7f3a7411f010 auth_method 0 2026-02-17T05:54:54.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.460 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a870640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a870640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7986e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7986e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7986e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a06f640 10 monclient(hunting): get_auth_request con 0x7f3a7406c870 auth_method 0 2026-02-17T05:54:54.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a06f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a06f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a06f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.461 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a06f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.462 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a06f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.462 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a06f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.462 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a870640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.462 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a870640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.462 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.434+0000 7f3a7a870640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.462 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.435+0000 7f3a7986e640 10 monclient(hunting): handle_auth_done global_id 4173 payload 824 2026-02-17T05:54:54.462 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.435+0000 7f3a7986e640 1 -- 10.20.193.164:0/2217129054 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3a7406c870 msgr2=0x7f3a741b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.462 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.435+0000 7f3a7986e640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3a7406c870 0x7f3a741b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.462 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.435+0000 7f3a7986e640 1 -- 10.20.193.164:0/2217129054 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3a7411f010 msgr2=0x7f3a741ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.435+0000 7f3a7986e640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3a7411f010 0x7f3a741ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.435+0000 7f3a7986e640 10 monclient: _finish_hunting 0 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.435+0000 7f3a7986e640 1 monclient: found mon.noname-a 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.435+0000 7f3a7986e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.435+0000 7f3a7986e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.435+0000 7f3a7986e640 1 -- 10.20.193.164:0/2217129054 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a741abdb0 con 0x7f3a7406d190 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.436+0000 7f3a7986e640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406d190 0x7f3a741b1ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800ebe0 tx=0x7f3a6800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.436+0000 7f3a5affd640 10 client.?.objecter ms_handle_connect 0x7f3a7406d190 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.436+0000 7f3a5affd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a5affd640 1 -- 10.20.193.164:0/2217129054 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68017070 con 0x7f3a7406d190 2026-02-17T05:54:54.463 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a5affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a5affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a5affd640 10 monclient: dump: 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.464 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a5affd640 10 monclient: _finish_auth 0 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a5affd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a5affd640 1 -- 10.20.193.164:0/2217129054 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6800cc40 con 0x7f3a7406d190 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a7c2fa640 5 monclient: authenticate success, global_id 4173 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a7c2fa640 10 monclient: _renew_subs 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a7c2fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.437+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2217129054 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a741ac040 con 0x7f3a7406d190 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a5affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a7886c640 4 set_mon_vals no callback set 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a5affd640 1 -- 10.20.193.164:0/2217129054 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68013470 con 0x7f3a7406d190 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a5affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a7c2fa640 10 client.4173.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:54.465 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a5affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a5affd640 10 monclient: dump: 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a7c2fa640 10 monclient: _renew_subs 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a7c2fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2217129054 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a741b83d0 con 0x7f3a7406d190 2026-02-17T05:54:54.466 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a7c2fa640 1 librados: init done 2026-02-17T05:54:54.467 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a7c2fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:54.467 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a7c2fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:54.467 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a7c2fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:54.467 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.438+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2217129054 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7406cc50 con 0x7f3a7406d190 2026-02-17T05:54:54.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.449+0000 7f3a5affd640 1 -- 10.20.193.164:0/2217129054 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 3) ==== 99895+0+0 (secure 0 0 0) 0x7f3a68024060 con 0x7f3a7406d190 2026-02-17T05:54:54.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.450+0000 7f3a5affd640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3a44078f90 0x7f3a4407b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.450+0000 7f3a7a06f640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3a44078f90 0x7f3a4407b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.450+0000 7f3a7a06f640 10 monclient: get_auth_request con 0x7f3a44078f90 auth_method 0 2026-02-17T05:54:54.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.451+0000 7f3a7a06f640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3a44078f90 0x7f3a4407b450 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3a64039000 tx=0x7f3a64033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:54.468 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.451+0000 7f3a5affd640 10 client.4173.objecter ms_handle_connect 0x7f3a44078f90 2026-02-17T05:54:54.548 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.546+0000 7f3a5affd640 1 -- 10.20.193.164:0/2217129054 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2105+0+0 (secure 0 0 0) 0x7f3a6805e430 con 0x7f3a7406d190 2026-02-17T05:54:54.548 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.546+0000 7f3a5affd640 10 client.4173.objecter ms_dispatch2 0x7f3a74000b90 osd_map(14..14 src has 1..14) 2026-02-17T05:54:54.548 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.546+0000 7f3a5affd640 3 client.4173.objecter handle_osd_map got epochs [14,14] > 0 2026-02-17T05:54:54.548 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.546+0000 7f3a5affd640 3 client.4173.objecter handle_osd_map decoding full epoch 14 2026-02-17T05:54:54.548 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.546+0000 7f3a5affd640 20 client.4173.objecter dump_active .. 0 homeless 2026-02-17T05:54:54.554 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.552+0000 7f3a5affd640 1 -- 10.20.193.164:0/2217129054 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3a680623f0 con 0x7f3a7406d190 2026-02-17T05:54:54.554 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.552+0000 7f3a5affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:54.554 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.552+0000 7f3a5affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:54.642 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.640+0000 7f3a7c2fa640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "9e9e82f6-9b4c-44cd-a18e-7089cc27b440", "id": 12}] 2026-02-17T05:54:54.642 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.640+0000 7f3a7c2fa640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "9e9e82f6-9b4c-44cd-a18e-7089cc27b440", "id": 12}] 2026-02-17T05:54:54.642 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.640+0000 7f3a7c2fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:54.642 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.640+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2217129054 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9e9e82f6-9b4c-44cd-a18e-7089cc27b440", "id": 12} v 0) -- 0x7f3a741b8600 con 0x7f3a7406d190 2026-02-17T05:54:54.660 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.658+0000 7f3a5affd640 1 -- 10.20.193.164:0/2217129054 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9e9e82f6-9b4c-44cd-a18e-7089cc27b440", "id": 12}]=0 v15) ==== 113+0+3 (secure 0 0 0) 0x7f3a680666f0 con 0x7f3a7406d190 2026-02-17T05:54:54.660 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.658+0000 7f3a5affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "9e9e82f6-9b4c-44cd-a18e-7089cc27b440", "id": 12}] 2026-02-17T05:54:54.660 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.658+0000 7f3a5affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:54.660 INFO:teuthology.orchestra.run.trial164.stdout:12 2026-02-17T05:54:54.661 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 10 librados: watch_flush enter 2026-02-17T05:54:54.661 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 10 librados: watch_flush exit 2026-02-17T05:54:54.661 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 20 client.4173.objecter shutdown clearing up homeless session... 2026-02-17T05:54:54.661 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 10 client.4173.objecter successfully canceled tick 2026-02-17T05:54:54.661 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2217129054 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3a44078f90 msgr2=0x7f3a4407b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.661 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3a44078f90 0x7f3a4407b450 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3a64039000 tx=0x7f3a64033000 comp rx=0 tx=0).stop 2026-02-17T05:54:54.661 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 10 monclient: shutdown 2026-02-17T05:54:54.661 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2217129054 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406d190 msgr2=0x7f3a741b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406d190 0x7f3a741b1ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800ebe0 tx=0x7f3a6800c520 comp rx=0 tx=0).stop 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2217129054 shutdown_connections 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3a44078f90 0x7f3a4407b450 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3a7411f010 0x7f3a741ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3a7406d190 0x7f3a741b1ab0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 --2- 10.20.193.164:0/2217129054 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3a7406c870 0x7f3a741b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2217129054 >> 10.20.193.164:0/2217129054 conn(0x7f3a74071610 msgr2=0x7f3a740722c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2217129054 shutdown_connections 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 -- 10.20.193.164:0/2217129054 wait complete. 2026-02-17T05:54:54.662 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.659+0000 7f3a7c2fa640 1 librados: shutdown 2026-02-17T05:54:54.669 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new e1bec267-a188-44ef-952c-6b9a39a2265a 13 2026-02-17T05:54:54.733 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.730+0000 7fb105b79640 10 monclient: get_monmap_and_config 2026-02-17T05:54:54.733 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.730+0000 7fb105b79640 10 monclient: build_initial_monmap 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.730+0000 7fb105b79640 10 monclient: monmap: 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:54.732111+0000 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:54.732111+0000 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.730+0000 7fb105b79640 1 Processor -- start 2026-02-17T05:54:54.734 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.730+0000 7fb105b79640 1 -- start start 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.730+0000 7fb105b79640 10 monclient: init 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 10 monclient: _add_conns ranks=[2,4,1,5,0,3] 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb1000735d0 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fb1000741b0 0x7fb10011f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 10 monclient(hunting): picked mon.noname-b con 0x7fb1000741b0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb10011fff0 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb100123db0 con 0x7fb10011fff0 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb100124fd0 con 0x7fb1000735d0 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fb1001261d0 con 0x7fb1000741b0 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb105b79640 10 monclient(hunting): authenticate will time out at 955.377177s 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb0fffff640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fb10011fff0 0x7fb1001224b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.164:58492/0 (socket says 10.20.193.164:58492) 2026-02-17T05:54:54.735 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.731+0000 7fb0fffff640 1 -- 10.20.193.164:0/4239768913 learned_addr learned my addr 10.20.193.164:0/4239768913 (peer_addr_for_me v1:10.20.193.164:0/0) 2026-02-17T05:54:54.736 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.732+0000 7fb0feffd640 1 --2- 10.20.193.164:0/4239768913 >> v2:10.20.193.128:3300/0 conn(0x7fb1000741b0 0x7fb10011f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.733+0000 7fb0feffd640 10 monclient(hunting): get_auth_request con 0x7fb1000741b0 auth_method 0 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.733+0000 7fb0feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.733+0000 7fb0feffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.733+0000 7fb0feffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.733+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.733+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.733+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.734+0000 7fb0feffd640 10 monclient(hunting): handle_auth_done global_id 4174 payload 274 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.734+0000 7fb0feffd640 1 -- 10.20.193.164:0/4239768913 >> v1:10.20.193.148:6789/0 conn(0x7fb1000735d0 legacy=0x7fb1000739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.734+0000 7fb0feffd640 1 -- 10.20.193.164:0/4239768913 >> v1:10.20.193.128:6789/0 conn(0x7fb10011fff0 legacy=0x7fb1001224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.734+0000 7fb0feffd640 10 monclient: _finish_hunting 0 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.734+0000 7fb0feffd640 1 monclient: found mon.noname-b 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.734+0000 7fb0feffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.734+0000 7fb0feffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:54:54.737 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.734+0000 7fb0feffd640 1 -- 10.20.193.164:0/4239768913 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb100126350 con 0x7fb1000741b0 2026-02-17T05:54:54.738 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.735+0000 7fb0feffd640 1 --2- 10.20.193.164:0/4239768913 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 0x7fb10011f990 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00098a0 tx=0x7fb0f002edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=188713f73643c6db server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:54.741 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 1 -- 10.20.193.164:0/4239768913 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f003c070 con 0x7fb1000741b0 2026-02-17T05:54:54.741 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.741 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:54.741 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 10 monclient: dump: 2026-02-17T05:54:54.741 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.741 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 10 monclient: _finish_auth 0 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 1 -- 10.20.193.164:0/4239768913 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f002fae0 con 0x7fb1000741b0 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 5 monclient: authenticate success, global_id 4174 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 1 -- 10.20.193.164:0/4239768913 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0037440 con 0x7fb1000741b0 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 4 set_mon_vals no callback set 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.742 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb0fe7fc640 10 monclient: dump: 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 10 monclient: shutdown 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 1 -- 10.20.193.164:0/4239768913 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 msgr2=0x7fb10011f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 1 --2- 10.20.193.164:0/4239768913 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 0x7fb10011f990 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00098a0 tx=0x7fb0f002edd0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 1 -- 10.20.193.164:0/4239768913 shutdown_connections 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 1 --2- 10.20.193.164:0/4239768913 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 0x7fb10011f990 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 1 -- 10.20.193.164:0/4239768913 >> 10.20.193.164:0/4239768913 conn(0x7fb10006f550 msgr2=0x7fb100071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:54.743 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 1 -- 10.20.193.164:0/4239768913 shutdown_connections 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 1 -- 10.20.193.164:0/4239768913 wait complete. 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 10 monclient: build_initial_monmap 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 10 monclient: monmap: 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 1 librados: starting msgr at 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 1 librados: starting objecter 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.737+0000 7fb105b79640 1 Processor -- start 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 1 -- start start 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 1 librados: setting wanted keys 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 1 librados: calling monclient init 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient: init 2026-02-17T05:54:54.744 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb1000735d0 0x7fb1001b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient(hunting): picked mon.noname-a con 0x7fb1000735d0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 0x7fb1001b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient(hunting): picked mon.noname-b con 0x7fb1000741b0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb10011fff0 0x7fb10011e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient(hunting): picked mon.noname-c con 0x7fb10011fff0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb100125660 con 0x7fb1000735d0 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001254e0 con 0x7fb1000741b0 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001257e0 con 0x7fb10011fff0 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb105b79640 10 monclient(hunting): authenticate will time out at 955.384291s 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb0feffd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 0x7fb1001b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb0feffd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 0x7fb1001b1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.164:39770/0 (socket says 10.20.193.164:39770) 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb0feffd640 1 -- 10.20.193.164:0/2637522787 learned_addr learned my addr 10.20.193.164:0/2637522787 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb0feffd640 10 monclient(hunting): get_auth_request con 0x7fb1000741b0 auth_method 0 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb0feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb0feffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb0feffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.738+0000 7fb0fffff640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb10011fff0 0x7fb10011e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.739+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.739+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.739+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.739+0000 7fb0fffff640 10 monclient(hunting): get_auth_request con 0x7fb10011fff0 auth_method 0 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.739+0000 7fb0fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.739+0000 7fb0fffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.739+0000 7fb0fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.739+0000 7fb0fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.739+0000 7fb0fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:54.745 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.739+0000 7fb0fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:54.746 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.740+0000 7fb0feffd640 10 monclient(hunting): handle_auth_done global_id 4177 payload 824 2026-02-17T05:54:54.746 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.740+0000 7fb0feffd640 1 -- 10.20.193.164:0/2637522787 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb10011fff0 msgr2=0x7fb10011e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.746 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.740+0000 7fb0feffd640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb10011fff0 0x7fb10011e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.740+0000 7fb0feffd640 1 -- 10.20.193.164:0/2637522787 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb1000735d0 msgr2=0x7fb1001b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.740+0000 7fb0feffd640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb1000735d0 0x7fb1001b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.740+0000 7fb0feffd640 10 monclient: _finish_hunting 0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.740+0000 7fb0feffd640 1 monclient: found mon.noname-b 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.740+0000 7fb0feffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.740+0000 7fb0feffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.740+0000 7fb0feffd640 1 -- 10.20.193.164:0/2637522787 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10011ecf0 con 0x7fb1000741b0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.741+0000 7fb0feffd640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 0x7fb1001b1d00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009130 tx=0x7fb0f00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.741+0000 7fb0dbfff640 10 client.?.objecter ms_handle_connect 0x7fb1000741b0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.741+0000 7fb0dbfff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 1 -- 10.20.193.164:0/2637522787 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0046020 con 0x7fb1000741b0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 10 monclient: dump: 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 10 monclient: _finish_auth 0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 1 -- 10.20.193.164:0/2637522787 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f0035680 con 0x7fb1000741b0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 1 -- 10.20.193.164:0/2637522787 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f003c040 con 0x7fb1000741b0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0dbfff640 10 monclient: dump: 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:54.747 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb0fdffb640 4 set_mon_vals no callback set 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb105b79640 5 monclient: authenticate success, global_id 4177 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb105b79640 10 monclient: _renew_subs 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb105b79640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb105b79640 1 -- 10.20.193.164:0/2637522787 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10011ef20 con 0x7fb1000741b0 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb105b79640 10 client.4177.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb105b79640 10 monclient: _renew_subs 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb105b79640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb105b79640 1 -- 10.20.193.164:0/2637522787 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1001b8400 con 0x7fb1000741b0 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.742+0000 7fb105b79640 1 librados: init done 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.743+0000 7fb105b79640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.743+0000 7fb105b79640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.743+0000 7fb105b79640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:54.748 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.743+0000 7fb105b79640 1 -- 10.20.193.164:0/2637522787 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb100125cc0 con 0x7fb1000741b0 2026-02-17T05:54:54.756 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.754+0000 7fb0dbfff640 1 -- 10.20.193.164:0/2637522787 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 3) ==== 99895+0+0 (secure 0 0 0) 0x7fb0f0035a70 con 0x7fb1000741b0 2026-02-17T05:54:54.757 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.754+0000 7fb0dbfff640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb0d0078fe0 0x7fb0d007b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:54.757 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.754+0000 7fb0ff7fe640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb0d0078fe0 0x7fb0d007b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:54.757 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.754+0000 7fb0ff7fe640 10 monclient: get_auth_request con 0x7fb0d0078fe0 auth_method 0 2026-02-17T05:54:54.757 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.755+0000 7fb0ff7fe640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb0d0078fe0 0x7fb0d007b4a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec002f00 tx=0x7fb0ec007a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:54.757 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.755+0000 7fb0dbfff640 10 client.4177.objecter ms_handle_connect 0x7fb0d0078fe0 2026-02-17T05:54:54.758 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.755+0000 7fb0dbfff640 1 -- 10.20.193.164:0/2637522787 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2220+0+0 (secure 0 0 0) 0x7fb0f0035d90 con 0x7fb1000741b0 2026-02-17T05:54:54.758 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.755+0000 7fb0dbfff640 10 client.4177.objecter ms_dispatch2 0x7fb100000b90 osd_map(15..15 src has 1..15) 2026-02-17T05:54:54.758 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.755+0000 7fb0dbfff640 3 client.4177.objecter handle_osd_map got epochs [15,15] > 0 2026-02-17T05:54:54.758 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.755+0000 7fb0dbfff640 3 client.4177.objecter handle_osd_map decoding full epoch 15 2026-02-17T05:54:54.759 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.755+0000 7fb0dbfff640 20 client.4177.objecter dump_active .. 0 homeless 2026-02-17T05:54:54.861 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.859+0000 7fb0dbfff640 1 -- 10.20.193.164:0/2637522787 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb0f0050020 con 0x7fb1000741b0 2026-02-17T05:54:54.861 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.859+0000 7fb0dbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:54.861 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.859+0000 7fb0dbfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:54.919 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.917+0000 7fb0dbfff640 1 -- 10.20.193.164:0/2637522787 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7fb0f0082dd0 con 0x7fb1000741b0 2026-02-17T05:54:54.949 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.947+0000 7fb105b79640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "e1bec267-a188-44ef-952c-6b9a39a2265a", "id": 13}] 2026-02-17T05:54:54.949 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.947+0000 7fb105b79640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "e1bec267-a188-44ef-952c-6b9a39a2265a", "id": 13}] 2026-02-17T05:54:54.950 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.947+0000 7fb105b79640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:54:54.950 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.947+0000 7fb105b79640 1 -- 10.20.193.164:0/2637522787 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e1bec267-a188-44ef-952c-6b9a39a2265a", "id": 13} v 0) -- 0x7fb1000726d0 con 0x7fb1000741b0 2026-02-17T05:54:54.997 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.994+0000 7fb0dbfff640 1 -- 10.20.193.164:0/2637522787 <== mon.1 v2:10.20.193.128:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e1bec267-a188-44ef-952c-6b9a39a2265a", "id": 13}]=0 v16) ==== 113+0+3 (secure 0 0 0) 0x7fb0f0087490 con 0x7fb1000741b0 2026-02-17T05:54:54.998 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.994+0000 7fb0dbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "e1bec267-a188-44ef-952c-6b9a39a2265a", "id": 13}] 2026-02-17T05:54:54.998 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.994+0000 7fb0dbfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:54.999 INFO:teuthology.orchestra.run.trial164.stdout:13 2026-02-17T05:54:55.001 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.995+0000 7fb105b79640 10 librados: watch_flush enter 2026-02-17T05:54:55.001 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.995+0000 7fb105b79640 10 librados: watch_flush exit 2026-02-17T05:54:55.001 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.995+0000 7fb105b79640 20 client.4177.objecter shutdown clearing up homeless session... 2026-02-17T05:54:55.001 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.995+0000 7fb105b79640 10 client.4177.objecter successfully canceled tick 2026-02-17T05:54:55.001 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.995+0000 7fb105b79640 1 -- 10.20.193.164:0/2637522787 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb0d0078fe0 msgr2=0x7fb0d007b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.002 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.995+0000 7fb105b79640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb0d0078fe0 0x7fb0d007b4a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec002f00 tx=0x7fb0ec007a70 comp rx=0 tx=0).stop 2026-02-17T05:54:55.002 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.995+0000 7fb105b79640 10 monclient: shutdown 2026-02-17T05:54:55.002 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.995+0000 7fb105b79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:55.002 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.995+0000 7fb105b79640 1 -- 10.20.193.164:0/2637522787 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 msgr2=0x7fb1001b1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.002 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.995+0000 7fb105b79640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 0x7fb1001b1d00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009130 tx=0x7fb0f00047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.002 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.996+0000 7fb105b79640 1 -- 10.20.193.164:0/2637522787 shutdown_connections 2026-02-17T05:54:55.002 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.996+0000 7fb105b79640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb0d0078fe0 0x7fb0d007b4a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.002 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.996+0000 7fb105b79640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb1000741b0 0x7fb1001b1d00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.002 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.996+0000 7fb105b79640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb10011fff0 0x7fb10011e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.003 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.996+0000 7fb105b79640 1 --2- 10.20.193.164:0/2637522787 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb1000735d0 0x7fb1001b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.003 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.996+0000 7fb105b79640 1 -- 10.20.193.164:0/2637522787 >> 10.20.193.164:0/2637522787 conn(0x7fb10006f550 msgr2=0x7fb10006f930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:55.003 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.996+0000 7fb105b79640 1 -- 10.20.193.164:0/2637522787 shutdown_connections 2026-02-17T05:54:55.003 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.996+0000 7fb105b79640 1 -- 10.20.193.164:0/2637522787 wait complete. 2026-02-17T05:54:55.003 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:54.996+0000 7fb105b79640 1 librados: shutdown 2026-02-17T05:54:55.006 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new c733624a-c696-405a-a41d-17481c672857 14 2026-02-17T05:54:55.072 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.068+0000 7faf66ec9640 10 monclient: get_monmap_and_config 2026-02-17T05:54:55.072 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.068+0000 7faf66ec9640 10 monclient: build_initial_monmap 2026-02-17T05:54:55.072 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.068+0000 7faf66ec9640 10 monclient: monmap: 2026-02-17T05:54:55.073 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:55.073 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.073 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:55.070072+0000 2026-02-17T05:54:55.073 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:55.070072+0000 2026-02-17T05:54:55.073 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:55.073 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.073 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:55.073 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:55.073 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.068+0000 7faf66ec9640 1 Processor -- start 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.068+0000 7faf66ec9640 1 -- start start 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.068+0000 7faf66ec9640 10 monclient: init 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 10 monclient: _add_conns ranks=[0,5,1,3,4,2] 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faf60073870 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7faf60073c50 0x7faf60124370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.074 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 10 monclient(hunting): picked mon.noname-c con 0x7faf60073c50 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7faf601248b0 addr v1:10.20.193.128:6789/0 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf601283c0 con 0x7faf60073870 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf601295a0 con 0x7faf601248b0 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7faf6012a780 con 0x7faf60073c50 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf66ec9640 10 monclient(hunting): authenticate will time out at 955.715131s 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf64c3e640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7faf60073870 0x7faf601181f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.164:39402/0 (socket says 10.20.193.164:39402) 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf64c3e640 1 -- 10.20.193.164:0/1332529348 learned_addr learned my addr 10.20.193.164:0/1332529348 (peer_addr_for_me v1:10.20.193.164:0/0) 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf57fff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7faf60073c50 0x7faf60124370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf57fff640 10 monclient(hunting): get_auth_request con 0x7faf60073c50 auth_method 0 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf57fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf57fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.069+0000 7faf57fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.075 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.076 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 10 monclient(hunting): handle_auth_done global_id 4208 payload 274 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 1 -- 10.20.193.164:0/1332529348 >> v1:10.20.193.128:6789/0 conn(0x7faf601248b0 legacy=0x7faf60126ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 1 -- 10.20.193.164:0/1332529348 >> v1:10.20.193.16:6789/0 conn(0x7faf60073870 legacy=0x7faf601181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 1 monclient: found mon.noname-c 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.070+0000 7faf57fff640 1 -- 10.20.193.164:0/1332529348 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6012a900 con 0x7faf60073c50 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.071+0000 7faf57fff640 1 --2- 10.20.193.164:0/1332529348 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 0x7faf60124370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faf480098a0 tx=0x7faf4802edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ef6acb1adc80e5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 1 -- 10.20.193.164:0/1332529348 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4803c070 con 0x7faf60073c50 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 10 monclient: dump: 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:55.080 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 10 monclient: _finish_auth 0 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 10 monclient: _check_auth_tickets 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 1 -- 10.20.193.164:0/1332529348 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4802fae0 con 0x7faf60073c50 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 1 -- 10.20.193.164:0/1332529348 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48037440 con 0x7faf60073c50 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf577fe640 10 monclient: dump: 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.081 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 5 monclient: authenticate success, global_id 4208 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 4 set_mon_vals no callback set 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 10 monclient: shutdown 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 1 -- 10.20.193.164:0/1332529348 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 msgr2=0x7faf60124370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 1 --2- 10.20.193.164:0/1332529348 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 0x7faf60124370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faf480098a0 tx=0x7faf4802edd0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 1 -- 10.20.193.164:0/1332529348 shutdown_connections 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 1 --2- 10.20.193.164:0/1332529348 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 0x7faf60124370 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 1 -- 10.20.193.164:0/1332529348 >> 10.20.193.164:0/1332529348 conn(0x7faf6006f550 msgr2=0x7faf60071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 1 -- 10.20.193.164:0/1332529348 shutdown_connections 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.073+0000 7faf66ec9640 1 -- 10.20.193.164:0/1332529348 wait complete. 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient: build_initial_monmap 2026-02-17T05:54:55.082 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient: monmap: 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 librados: starting msgr at 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 librados: starting objecter 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 Processor -- start 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 -- start start 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 librados: setting wanted keys 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 librados: calling monclient init 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient: init 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7faf60073870 0x7faf60116fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient(hunting): picked mon.noname-a con 0x7faf60073870 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:55.083 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 0x7faf60117640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient(hunting): picked mon.noname-c con 0x7faf60073c50 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faf601248b0 0x7faf601b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient(hunting): picked mon.noname-b con 0x7faf601248b0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf60129bc0 con 0x7faf60073870 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7faf60129a40 con 0x7faf601248b0 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7faf60129d40 con 0x7faf60073c50 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf66ec9640 10 monclient(hunting): authenticate will time out at 955.720712s 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.074+0000 7faf57fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 0x7faf60117640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf64c3e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7faf60073870 0x7faf60116fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf6543f640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faf601248b0 0x7faf601b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf57fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 0x7faf60117640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.164:34088/0 (socket says 10.20.193.164:34088) 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf57fff640 1 -- 10.20.193.164:0/661415593 learned_addr learned my addr 10.20.193.164:0/661415593 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf57fff640 10 monclient(hunting): get_auth_request con 0x7faf60073c50 auth_method 0 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf57fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf57fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf57fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.084 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf6543f640 10 monclient(hunting): get_auth_request con 0x7faf601248b0 auth_method 0 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf6543f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf6543f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf6543f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf64c3e640 10 monclient(hunting): get_auth_request con 0x7faf60073870 auth_method 0 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf64c3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf64c3e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf64c3e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf57fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf57fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf57fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf6543f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf6543f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf6543f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf64c3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf64c3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.075+0000 7faf64c3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.076+0000 7faf57fff640 10 monclient(hunting): handle_auth_done global_id 4211 payload 824 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.076+0000 7faf57fff640 1 -- 10.20.193.164:0/661415593 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faf601248b0 msgr2=0x7faf601b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.076+0000 7faf57fff640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faf601248b0 0x7faf601b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.076+0000 7faf57fff640 1 -- 10.20.193.164:0/661415593 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7faf60073870 msgr2=0x7faf60116fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.076+0000 7faf57fff640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7faf60073870 0x7faf60116fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.076+0000 7faf57fff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:55.085 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.076+0000 7faf57fff640 1 monclient: found mon.noname-c 2026-02-17T05:54:55.086 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.076+0000 7faf57fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:55.086 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.076+0000 7faf57fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.086 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.076+0000 7faf57fff640 1 -- 10.20.193.164:0/661415593 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf601b60d0 con 0x7faf60073c50 2026-02-17T05:54:55.086 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.077+0000 7faf57fff640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 0x7faf60117640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faf48009130 tx=0x7faf480047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:55.086 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.077+0000 7faf557fa640 10 client.?.objecter ms_handle_connect 0x7faf60073c50 2026-02-17T05:54:55.086 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.077+0000 7faf557fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf557fa640 1 -- 10.20.193.164:0/661415593 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48046020 con 0x7faf60073c50 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf557fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf557fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf557fa640 10 monclient: dump: 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.087 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.089 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.089 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.089 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf557fa640 10 monclient: _finish_auth 0 2026-02-17T05:54:55.089 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf557fa640 10 monclient: _check_auth_tickets 2026-02-17T05:54:55.089 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:55.089 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf66ec9640 5 monclient: authenticate success, global_id 4211 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf66ec9640 10 monclient: _renew_subs 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf66ec9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf66ec9640 1 -- 10.20.193.164:0/661415593 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf601b6300 con 0x7faf60073c50 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf66ec9640 10 client.4211.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf66ec9640 10 monclient: _renew_subs 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf66ec9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf66ec9640 1 -- 10.20.193.164:0/661415593 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf601bc950 con 0x7faf60073c50 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.078+0000 7faf66ec9640 1 librados: init done 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf557fa640 1 -- 10.20.193.164:0/661415593 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4802fae0 con 0x7faf60073c50 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf557fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf557fa640 1 -- 10.20.193.164:0/661415593 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4803c040 con 0x7faf60073c50 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf557fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf557fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf56ffd640 4 set_mon_vals no callback set 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf557fa640 10 monclient: dump: 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.090 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.091 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.091 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.091 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.091 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.091 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.091 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf66ec9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:55.091 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf66ec9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:55.091 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf66ec9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.091 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.079+0000 7faf66ec9640 1 -- 10.20.193.164:0/661415593 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6012a220 con 0x7faf60073c50 2026-02-17T05:54:55.185 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.183+0000 7faf557fa640 1 -- 10.20.193.164:0/661415593 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7faf48035880 con 0x7faf60073c50 2026-02-17T05:54:55.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.184+0000 7faf557fa640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faf300790b0 0x7faf3007b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.184+0000 7faf64c3e640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faf300790b0 0x7faf3007b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.184+0000 7faf64c3e640 10 monclient: get_auth_request con 0x7faf300790b0 auth_method 0 2026-02-17T05:54:55.186 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.185+0000 7faf64c3e640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faf300790b0 0x7faf3007b570 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf50002770 tx=0x7faf50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:55.187 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.185+0000 7faf557fa640 10 client.4211.objecter ms_handle_connect 0x7faf300790b0 2026-02-17T05:54:55.192 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.190+0000 7faf557fa640 1 -- 10.20.193.164:0/661415593 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2335+0+0 (secure 0 0 0) 0x7faf480831c0 con 0x7faf60073c50 2026-02-17T05:54:55.192 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.190+0000 7faf557fa640 10 client.4211.objecter ms_dispatch2 0x7faf60000b90 osd_map(16..16 src has 1..16) 2026-02-17T05:54:55.192 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.190+0000 7faf557fa640 3 client.4211.objecter handle_osd_map got epochs [16,16] > 0 2026-02-17T05:54:55.192 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.190+0000 7faf557fa640 3 client.4211.objecter handle_osd_map decoding full epoch 16 2026-02-17T05:54:55.192 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.191+0000 7faf557fa640 20 client.4211.objecter dump_active .. 0 homeless 2026-02-17T05:54:55.193 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.191+0000 7faf557fa640 1 -- 10.20.193.164:0/661415593 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7faf48050070 con 0x7faf60073c50 2026-02-17T05:54:55.193 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.192+0000 7faf557fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:55.193 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.192+0000 7faf557fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:55.282 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.280+0000 7faf66ec9640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c733624a-c696-405a-a41d-17481c672857", "id": 14}] 2026-02-17T05:54:55.282 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.280+0000 7faf66ec9640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c733624a-c696-405a-a41d-17481c672857", "id": 14}] 2026-02-17T05:54:55.282 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.280+0000 7faf66ec9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.282 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.280+0000 7faf66ec9640 1 -- 10.20.193.164:0/661415593 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c733624a-c696-405a-a41d-17481c672857", "id": 14} v 0) -- 0x7faf601bcb80 con 0x7faf60073c50 2026-02-17T05:54:55.331 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.328+0000 7faf557fa640 1 -- 10.20.193.164:0/661415593 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c733624a-c696-405a-a41d-17481c672857", "id": 14}]=0 v17) ==== 113+0+3 (secure 0 0 0) 0x7faf48087180 con 0x7faf60073c50 2026-02-17T05:54:55.331 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.328+0000 7faf557fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c733624a-c696-405a-a41d-17481c672857", "id": 14}] 2026-02-17T05:54:55.331 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.328+0000 7faf557fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:55.331 INFO:teuthology.orchestra.run.trial164.stdout:14 2026-02-17T05:54:55.333 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.329+0000 7faf66ec9640 10 librados: watch_flush enter 2026-02-17T05:54:55.333 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.329+0000 7faf66ec9640 10 librados: watch_flush exit 2026-02-17T05:54:55.333 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.329+0000 7faf66ec9640 20 client.4211.objecter shutdown clearing up homeless session... 2026-02-17T05:54:55.333 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.329+0000 7faf66ec9640 10 client.4211.objecter successfully canceled tick 2026-02-17T05:54:55.333 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.329+0000 7faf66ec9640 1 -- 10.20.193.164:0/661415593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faf300790b0 msgr2=0x7faf3007b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.333 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.329+0000 7faf66ec9640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faf300790b0 0x7faf3007b570 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf50002770 tx=0x7faf50009040 comp rx=0 tx=0).stop 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.329+0000 7faf66ec9640 10 monclient: shutdown 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.329+0000 7faf66ec9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.329+0000 7faf66ec9640 1 -- 10.20.193.164:0/661415593 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 msgr2=0x7faf60117640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.329+0000 7faf66ec9640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 0x7faf60117640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faf48009130 tx=0x7faf480047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.330+0000 7faf66ec9640 1 -- 10.20.193.164:0/661415593 shutdown_connections 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.330+0000 7faf66ec9640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faf300790b0 0x7faf3007b570 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.330+0000 7faf66ec9640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faf601248b0 0x7faf601b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.330+0000 7faf66ec9640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faf60073c50 0x7faf60117640 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.330+0000 7faf66ec9640 1 --2- 10.20.193.164:0/661415593 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7faf60073870 0x7faf60116fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.330+0000 7faf66ec9640 1 -- 10.20.193.164:0/661415593 >> 10.20.193.164:0/661415593 conn(0x7faf6006f550 msgr2=0x7faf60126e20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:55.334 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.330+0000 7faf66ec9640 1 -- 10.20.193.164:0/661415593 shutdown_connections 2026-02-17T05:54:55.335 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.330+0000 7faf66ec9640 1 -- 10.20.193.164:0/661415593 wait complete. 2026-02-17T05:54:55.335 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.330+0000 7faf66ec9640 1 librados: shutdown 2026-02-17T05:54:55.342 DEBUG:teuthology.orchestra.run.trial164:> sudo ceph --cluster ceph osd new 8fa42467-586f-4e64-b428-9afd5dd14b77 15 2026-02-17T05:54:55.409 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient: get_monmap_and_config 2026-02-17T05:54:55.409 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient: build_initial_monmap 2026-02-17T05:54:55.409 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient: monmap: 2026-02-17T05:54:55.409 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:55.409 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.409 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:54:55.404797+0000 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:54:55.404797+0000 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 1 Processor -- start 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 1 -- start start 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient: init 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient: _add_conns ranks=[5,3,0,4,1,2] 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6d8c0735d0 0x7f6d8c0739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.410 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient(hunting): picked mon.noname-c con 0x7f6d8c0735d0 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f6d8c073fe0 0x7f6d8c11e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient(hunting): picked mon.noname-a con 0x7f6d8c073fe0 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6d8c11eec0 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d8c122bd0 con 0x7f6d8c11eec0 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6d8c123df0 con 0x7f6d8c073fe0 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6d8c123f70 con 0x7f6d8c0735d0 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.403+0000 7f6d92114640 10 monclient(hunting): authenticate will time out at 956.049871s 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8b7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6d8c0735d0 0x7f6d8c0739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8affd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f6d8c073fe0 0x7f6d8c11e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8b7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6d8c0735d0 0x7f6d8c0739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.164:34096/0 (socket says 10.20.193.164:34096) 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8b7fe640 1 -- 10.20.193.164:0/1749788004 learned_addr learned my addr 10.20.193.164:0/1749788004 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:55.411 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8b7fe640 10 monclient(hunting): get_auth_request con 0x7f6d8c0735d0 auth_method 0 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8affd640 10 monclient(hunting): get_auth_request con 0x7f6d8c073fe0 auth_method 0 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.404+0000 7f6d8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.405+0000 7f6d8b7fe640 10 monclient(hunting): handle_auth_done global_id 4214 payload 274 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.405+0000 7f6d8b7fe640 1 -- 10.20.193.164:0/1749788004 >> v2:10.20.193.16:3300/0 conn(0x7f6d8c073fe0 msgr2=0x7f6d8c11e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:55.412 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.405+0000 7f6d8b7fe640 1 --2- 10.20.193.164:0/1749788004 >> v2:10.20.193.16:3300/0 conn(0x7f6d8c073fe0 0x7f6d8c11e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.413 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.405+0000 7f6d8b7fe640 1 -- 10.20.193.164:0/1749788004 >> v1:10.20.193.16:6789/0 conn(0x7f6d8c11eec0 legacy=0x7f6d8c1212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:55.413 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.405+0000 7f6d8b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:54:55.413 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.405+0000 7f6d8b7fe640 1 monclient: found mon.noname-c 2026-02-17T05:54:55.413 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.405+0000 7f6d8b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:55.413 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.405+0000 7f6d8b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.413 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.405+0000 7f6d8b7fe640 1 -- 10.20.193.164:0/1749788004 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c124120 con 0x7f6d8c0735d0 2026-02-17T05:54:55.413 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.406+0000 7f6d8b7fe640 1 --2- 10.20.193.164:0/1749788004 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d8c0735d0 0x7f6d8c0739b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0766a0 tx=0x7f6d7802ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee8152b11eb705f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:55.418 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 1 -- 10.20.193.164:0/1749788004 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7803c070 con 0x7f6d8c0735d0 2026-02-17T05:54:55.418 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.418 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:55.418 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 10 monclient: dump: 2026-02-17T05:54:55.418 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:55.418 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.418 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.418 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.418 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 1 -- 10.20.193.164:0/1749788004 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d7802fa90 con 0x7f6d8c0735d0 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 5 monclient: authenticate success, global_id 4214 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 1 -- 10.20.193.164:0/1749788004 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7802fe60 con 0x7f6d8c0735d0 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d8a7fc640 10 monclient: dump: 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:55.419 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 4 set_mon_vals no callback set 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 10 monclient: shutdown 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 1 -- 10.20.193.164:0/1749788004 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d8c0735d0 msgr2=0x7f6d8c0739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 1 --2- 10.20.193.164:0/1749788004 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d8c0735d0 0x7f6d8c0739b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0766a0 tx=0x7f6d7802ee60 comp rx=0 tx=0).stop 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 1 -- 10.20.193.164:0/1749788004 shutdown_connections 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 1 --2- 10.20.193.164:0/1749788004 >> v2:10.20.193.16:3300/0 conn(0x7f6d8c073fe0 0x7f6d8c11e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.420 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 1 --2- 10.20.193.164:0/1749788004 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d8c0735d0 0x7f6d8c0739b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 1 -- 10.20.193.164:0/1749788004 >> 10.20.193.164:0/1749788004 conn(0x7f6d8c06f550 msgr2=0x7f6d8c071970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 1 -- 10.20.193.164:0/1749788004 shutdown_connections 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.408+0000 7f6d92114640 1 -- 10.20.193.164:0/1749788004 wait complete. 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient: build_initial_monmap 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient: monmap: 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:epoch 0 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 0.000000 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:created 0.000000 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:55.421 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 librados: starting msgr at 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 librados: starting objecter 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 Processor -- start 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 -- start start 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 librados: setting wanted keys 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 librados: calling monclient init 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient: init 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d8c0735d0 0x7f6d8c1ab970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient(hunting): picked mon.noname-b con 0x7f6d8c0735d0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d8c073fe0 0x7f6d8c1abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient(hunting): picked mon.noname-a con 0x7f6d8c073fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d8c11eec0 0x7f6d8c1b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient(hunting): picked mon.noname-c con 0x7f6d8c11eec0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.422 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c123260 con 0x7f6d8c073fe0 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1230e0 con 0x7f6d8c0735d0 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1233e0 con 0x7f6d8c11eec0 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d92114640 10 monclient(hunting): authenticate will time out at 956.055451s 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d8affd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d8c073fe0 0x7f6d8c1abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d8b7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d8c0735d0 0x7f6d8c1ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d8bfff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d8c11eec0 0x7f6d8c1b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.409+0000 7f6d8b7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d8c0735d0 0x7f6d8c1ab970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.164:39788/0 (socket says 10.20.193.164:39788) 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8b7fe640 1 -- 10.20.193.164:0/3479699470 learned_addr learned my addr 10.20.193.164:0/3479699470 (peer_addr_for_me v2:10.20.193.164:0/0) 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8b7fe640 10 monclient(hunting): get_auth_request con 0x7f6d8c0735d0 auth_method 0 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8affd640 10 monclient(hunting): get_auth_request con 0x7f6d8c073fe0 auth_method 0 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8bfff640 10 monclient(hunting): get_auth_request con 0x7f6d8c11eec0 auth_method 0 2026-02-17T05:54:55.423 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.410+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.411+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_done global_id 4217 payload 824 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.411+0000 7f6d8bfff640 1 -- 10.20.193.164:0/3479699470 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d8c0735d0 msgr2=0x7f6d8c1ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.411+0000 7f6d8bfff640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d8c0735d0 0x7f6d8c1ab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.411+0000 7f6d8bfff640 1 -- 10.20.193.164:0/3479699470 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d8c073fe0 msgr2=0x7f6d8c1abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.411+0000 7f6d8bfff640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d8c073fe0 0x7f6d8c1abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.411+0000 7f6d8bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.411+0000 7f6d8bfff640 1 monclient: found mon.noname-c 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.411+0000 7f6d8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.411+0000 7f6d8bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.424 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.411+0000 7f6d8bfff640 1 -- 10.20.193.164:0/3479699470 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c1b0920 con 0x7f6d8c11eec0 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.412+0000 7f6d8bfff640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d8c11eec0 0x7f6d8c1b0060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000cae0 tx=0x7f6d8000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.412+0000 7f6d67fff640 10 client.?.objecter ms_handle_connect 0x7f6d8c11eec0 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.412+0000 7f6d67fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 1 -- 10.20.193.164:0/3479699470 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80016020 con 0x7f6d8c11eec0 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 10 monclient: dump: 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 10 monclient: _finish_auth 0 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 10 monclient: _check_auth_tickets 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 1 -- 10.20.193.164:0/3479699470 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8000acb0 con 0x7f6d8c11eec0 2026-02-17T05:54:55.425 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 1 -- 10.20.193.164:0/3479699470 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80012660 con 0x7f6d8c11eec0 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d67fff640 10 monclient: dump: 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:epoch 1 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:election_strategy: 1 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d89ffb640 4 set_mon_vals no callback set 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d92114640 5 monclient: authenticate success, global_id 4217 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d92114640 10 monclient: _renew_subs 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d92114640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d92114640 1 -- 10.20.193.164:0/3479699470 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c1b0c10 con 0x7f6d8c11eec0 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d92114640 10 client.4217.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d92114640 10 monclient: _renew_subs 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d92114640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d92114640 1 -- 10.20.193.164:0/3479699470 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c1b8390 con 0x7f6d8c11eec0 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.414+0000 7f6d92114640 1 librados: init done 2026-02-17T05:54:55.426 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.415+0000 7f6d92114640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:55.427 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.415+0000 7f6d92114640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:55.427 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.415+0000 7f6d92114640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.427 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.415+0000 7f6d92114640 1 -- 10.20.193.164:0/3479699470 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c0739b0 con 0x7f6d8c11eec0 2026-02-17T05:54:55.427 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.426+0000 7f6d67fff640 1 -- 10.20.193.164:0/3479699470 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f6d80012800 con 0x7f6d8c11eec0 2026-02-17T05:54:55.428 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.426+0000 7f6d67fff640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d5c079100 0x7f6d5c07b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.428 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.426+0000 7f6d8b7fe640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d5c079100 0x7f6d5c07b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.428 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.426+0000 7f6d8b7fe640 10 monclient: get_auth_request con 0x7f6d5c079100 auth_method 0 2026-02-17T05:54:55.429 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.427+0000 7f6d8b7fe640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d5c079100 0x7f6d5c07b5c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009730 tx=0x7f6d78033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:55.429 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.427+0000 7f6d67fff640 10 client.4217.objecter ms_handle_connect 0x7f6d5c079100 2026-02-17T05:54:55.529 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.526+0000 7f6d67fff640 1 -- 10.20.193.164:0/3479699470 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2450+0+0 (secure 0 0 0) 0x7f6d8005ea10 con 0x7f6d8c11eec0 2026-02-17T05:54:55.529 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.526+0000 7f6d67fff640 10 client.4217.objecter ms_dispatch2 0x7f6d8c000b90 osd_map(17..17 src has 1..17) 2026-02-17T05:54:55.529 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.526+0000 7f6d67fff640 3 client.4217.objecter handle_osd_map got epochs [17,17] > 0 2026-02-17T05:54:55.529 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.526+0000 7f6d67fff640 3 client.4217.objecter handle_osd_map decoding full epoch 17 2026-02-17T05:54:55.529 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.526+0000 7f6d67fff640 20 client.4217.objecter dump_active .. 0 homeless 2026-02-17T05:54:55.530 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.526+0000 7f6d67fff640 1 -- 10.20.193.164:0/3479699470 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6d80020020 con 0x7f6d8c11eec0 2026-02-17T05:54:55.530 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.526+0000 7f6d67fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:55.530 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.526+0000 7f6d67fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:55.616 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.614+0000 7f6d92114640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8fa42467-586f-4e64-b428-9afd5dd14b77", "id": 15}] 2026-02-17T05:54:55.617 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.614+0000 7f6d92114640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8fa42467-586f-4e64-b428-9afd5dd14b77", "id": 15}] 2026-02-17T05:54:55.617 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.614+0000 7f6d92114640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.617 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.614+0000 7f6d92114640 1 -- 10.20.193.164:0/3479699470 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8fa42467-586f-4e64-b428-9afd5dd14b77", "id": 15} v 0) -- 0x7f6d8c1b8560 con 0x7f6d8c11eec0 2026-02-17T05:54:55.708 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.706+0000 7f6d67fff640 1 -- 10.20.193.164:0/3479699470 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8fa42467-586f-4e64-b428-9afd5dd14b77", "id": 15}]=0 v18) ==== 113+0+3 (secure 0 0 0) 0x7f6d800629d0 con 0x7f6d8c11eec0 2026-02-17T05:54:55.709 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.706+0000 7f6d67fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8fa42467-586f-4e64-b428-9afd5dd14b77", "id": 15}] 2026-02-17T05:54:55.709 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.706+0000 7f6d67fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:55.709 INFO:teuthology.orchestra.run.trial164.stdout:15 2026-02-17T05:54:55.711 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 10 librados: watch_flush enter 2026-02-17T05:54:55.712 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 10 librados: watch_flush exit 2026-02-17T05:54:55.712 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 20 client.4217.objecter shutdown clearing up homeless session... 2026-02-17T05:54:55.712 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 10 client.4217.objecter successfully canceled tick 2026-02-17T05:54:55.712 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 -- 10.20.193.164:0/3479699470 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d5c079100 msgr2=0x7f6d5c07b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.712 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d5c079100 0x7f6d5c07b5c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009730 tx=0x7f6d78033000 comp rx=0 tx=0).stop 2026-02-17T05:54:55.712 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 10 monclient: shutdown 2026-02-17T05:54:55.712 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:55.712 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 -- 10.20.193.164:0/3479699470 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d8c11eec0 msgr2=0x7f6d8c1b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.712 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d8c11eec0 0x7f6d8c1b0060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000cae0 tx=0x7f6d8000a060 comp rx=0 tx=0).stop 2026-02-17T05:54:55.713 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 -- 10.20.193.164:0/3479699470 shutdown_connections 2026-02-17T05:54:55.713 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d5c079100 0x7f6d5c07b5c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.713 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d8c11eec0 0x7f6d8c1b0060 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.713 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d8c073fe0 0x7f6d8c1abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.713 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 --2- 10.20.193.164:0/3479699470 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d8c0735d0 0x7f6d8c1ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.713 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 -- 10.20.193.164:0/3479699470 >> 10.20.193.164:0/3479699470 conn(0x7f6d8c06f550 msgr2=0x7f6d8c11f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:55.713 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 -- 10.20.193.164:0/3479699470 shutdown_connections 2026-02-17T05:54:55.713 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 -- 10.20.193.164:0/3479699470 wait complete. 2026-02-17T05:54:55.713 INFO:teuthology.orchestra.run.trial164.stderr:2026-02-17T05:54:55.707+0000 7f6d92114640 1 librados: shutdown 2026-02-17T05:54:55.719 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.719 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-17T05:54:55.720 DEBUG:teuthology.orchestra.run.trial016:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2026-02-17T05:54:55.722 INFO:tasks.ceph.osd.0:Started 2026-02-17T05:54:55.722 DEBUG:tasks.ceph_manager:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.723 INFO:tasks.ceph.osd.4:Restarting daemon 2026-02-17T05:54:55.723 DEBUG:teuthology.orchestra.run.trial016:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 2026-02-17T05:54:55.811 INFO:tasks.ceph.osd.4:Started 2026-02-17T05:54:55.811 DEBUG:tasks.ceph_manager:running osd.8 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.8.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.811 INFO:tasks.ceph.osd.8:Restarting daemon 2026-02-17T05:54:55.811 DEBUG:teuthology.orchestra.run.trial016:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.8.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 8 2026-02-17T05:54:55.813 INFO:tasks.ceph.osd.8:Started 2026-02-17T05:54:55.813 DEBUG:tasks.ceph_manager:running osd.12 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.12.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.813 INFO:tasks.ceph.osd.12:Restarting daemon 2026-02-17T05:54:55.813 DEBUG:teuthology.orchestra.run.trial016:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.12.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 12 2026-02-17T05:54:55.815 INFO:tasks.ceph.osd.12:Started 2026-02-17T05:54:55.815 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.815 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-17T05:54:55.815 DEBUG:teuthology.orchestra.run.trial128:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2026-02-17T05:54:55.816 INFO:tasks.ceph.osd.1:Started 2026-02-17T05:54:55.816 DEBUG:tasks.ceph_manager:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.817 INFO:tasks.ceph.osd.5:Restarting daemon 2026-02-17T05:54:55.817 DEBUG:teuthology.orchestra.run.trial128:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 2026-02-17T05:54:55.818 INFO:tasks.ceph.osd.5:Started 2026-02-17T05:54:55.818 DEBUG:tasks.ceph_manager:running osd.9 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.9.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.818 INFO:tasks.ceph.osd.9:Restarting daemon 2026-02-17T05:54:55.818 DEBUG:teuthology.orchestra.run.trial128:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.9.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 9 2026-02-17T05:54:55.819 INFO:tasks.ceph.osd.9:Started 2026-02-17T05:54:55.819 DEBUG:tasks.ceph_manager:running osd.13 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.13.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.819 INFO:tasks.ceph.osd.13:Restarting daemon 2026-02-17T05:54:55.820 DEBUG:teuthology.orchestra.run.trial128:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.13.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 13 2026-02-17T05:54:55.821 INFO:tasks.ceph.osd.13:Started 2026-02-17T05:54:55.821 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.821 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-17T05:54:55.821 DEBUG:teuthology.orchestra.run.trial148:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2026-02-17T05:54:55.823 INFO:tasks.ceph.osd.2:Started 2026-02-17T05:54:55.823 DEBUG:tasks.ceph_manager:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.823 INFO:tasks.ceph.osd.6:Restarting daemon 2026-02-17T05:54:55.823 DEBUG:teuthology.orchestra.run.trial148:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 2026-02-17T05:54:55.824 INFO:tasks.ceph.osd.6:Started 2026-02-17T05:54:55.824 DEBUG:tasks.ceph_manager:running osd.10 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.10.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.824 INFO:tasks.ceph.osd.10:Restarting daemon 2026-02-17T05:54:55.824 DEBUG:teuthology.orchestra.run.trial148:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.10.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 10 2026-02-17T05:54:55.826 INFO:tasks.ceph.osd.10:Started 2026-02-17T05:54:55.826 DEBUG:tasks.ceph_manager:running osd.14 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.14.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.826 INFO:tasks.ceph.osd.14:Restarting daemon 2026-02-17T05:54:55.826 DEBUG:teuthology.orchestra.run.trial148:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.14.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 14 2026-02-17T05:54:55.827 INFO:tasks.ceph.osd.14:Started 2026-02-17T05:54:55.827 DEBUG:tasks.ceph_manager:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.827 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-17T05:54:55.827 DEBUG:teuthology.orchestra.run.trial164:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 2026-02-17T05:54:55.829 INFO:tasks.ceph.osd.3:Started 2026-02-17T05:54:55.829 DEBUG:tasks.ceph_manager:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.829 INFO:tasks.ceph.osd.7:Restarting daemon 2026-02-17T05:54:55.829 DEBUG:teuthology.orchestra.run.trial164:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2026-02-17T05:54:55.831 INFO:tasks.ceph.osd.7:Started 2026-02-17T05:54:55.831 DEBUG:tasks.ceph_manager:running osd.11 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.11.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.831 INFO:tasks.ceph.osd.11:Restarting daemon 2026-02-17T05:54:55.831 DEBUG:teuthology.orchestra.run.trial164:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.11.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 11 2026-02-17T05:54:55.833 INFO:tasks.ceph.osd.11:Started 2026-02-17T05:54:55.833 DEBUG:tasks.ceph_manager:running osd.15 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.15.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-17T05:54:55.833 INFO:tasks.ceph.osd.15:Restarting daemon 2026-02-17T05:54:55.833 DEBUG:teuthology.orchestra.run.trial164:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.15.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 15 2026-02-17T05:54:55.835 INFO:tasks.ceph.osd.15:Started 2026-02-17T05:54:55.835 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.906+0000 7f206ef3e640 10 monclient: get_monmap_and_config 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.906+0000 7f206ef3e640 10 monclient: build_initial_monmap 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.906+0000 7f206ef3e640 10 monclient: monmap: 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:54:55.907112+0000 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:54:55.907112+0000 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:55.906 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.906+0000 7f206ef3e640 1 Processor -- start 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.906+0000 7f206ef3e640 1 -- start start 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.906+0000 7f206ef3e640 10 monclient: init 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 10 monclient: _add_conns ranks=[5,2,0,4,1,3] 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f20680700f0 0x7f20680704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 10 monclient(hunting): picked mon.noname-c con 0x7f20680700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f20680c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f20681536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20680c4820 con 0x7f20681536f0 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20680c4b00 con 0x7f20680c7fe0 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f20680c4de0 con 0x7f20680700f0 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ef3e640 10 monclient(hunting): authenticate will time out at 948.476244s 2026-02-17T05:54:55.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206d4b4640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f20681536f0 0x7f2068153b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:59262/0 (socket says 10.20.193.16:59262) 2026-02-17T05:54:55.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206d4b4640 1 -- 10.20.193.16:0/3785114471 learned_addr learned my addr 10.20.193.16:0/3785114471 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:54:55.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.907+0000 7f206ccb3640 1 --2- 10.20.193.16:0/3785114471 >> v2:10.20.193.148:3300/0 conn(0x7f20680700f0 0x7f20680704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.908+0000 7f206ccb3640 10 monclient(hunting): get_auth_request con 0x7f20680700f0 auth_method 0 2026-02-17T05:54:55.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.908+0000 7f206ccb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.908+0000 7f206ccb3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.908+0000 7f206ccb3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.908+0000 7f206ccb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.908+0000 7f206ccb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.908+0000 7f206ccb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f205f7fe640 1 -- 10.20.193.16:0/3785114471 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4104521755 0 0) 0x7f20680c4820 con 0x7f20681536f0 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f205f7fe640 10 monclient(hunting): _negotiate 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f205f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f205f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f205f7fe640 10 monclient(hunting): my global_id is 4188 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f205f7fe640 1 -- 10.20.193.16:0/3785114471 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2054003620 con 0x7f20681536f0 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f206ccb3640 10 monclient(hunting): handle_auth_done global_id 4220 payload 274 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f206ccb3640 1 -- 10.20.193.16:0/3785114471 >> v1:10.20.193.148:6789/0 conn(0x7f20680c7fe0 legacy=0x7f20680c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f206ccb3640 1 -- 10.20.193.16:0/3785114471 >> v1:10.20.193.16:6789/0 conn(0x7f20681536f0 legacy=0x7f2068153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f206ccb3640 10 monclient: _finish_hunting 0 2026-02-17T05:54:55.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f206ccb3640 1 monclient: found mon.noname-c 2026-02-17T05:54:55.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f206ccb3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:55.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f206ccb3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.909+0000 7f206ccb3640 1 -- 10.20.193.16:0/3785114471 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20680c50c0 con 0x7f20680700f0 2026-02-17T05:54:55.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.912+0000 7f206ccb3640 1 --2- 10.20.193.16:0/3785114471 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20680700f0 0x7f20680704d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f20580098a0 tx=0x7f205802edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31e7d1ea0d6e2221 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 1 -- 10.20.193.16:0/3785114471 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205803c070 con 0x7f20680700f0 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 10 monclient: dump: 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:55.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 1 -- 10.20.193.16:0/3785114471 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f205802fae0 con 0x7f20680700f0 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 1 -- 10.20.193.16:0/3785114471 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058037440 con 0x7f20680700f0 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f206ef3e640 5 monclient: authenticate success, global_id 4220 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f206ef3e640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f206ef3e640 4 set_mon_vals no callback set 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.914+0000 7f205f7fe640 10 monclient: dump: 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.916 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.917 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:55.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.915+0000 7f206ef3e640 10 monclient: shutdown 2026-02-17T05:54:55.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.915+0000 7f206ef3e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:55.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.915+0000 7f206ef3e640 1 -- 10.20.193.16:0/3785114471 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20680700f0 msgr2=0x7f20680704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.915+0000 7f206ef3e640 1 --2- 10.20.193.16:0/3785114471 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20680700f0 0x7f20680704d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f20580098a0 tx=0x7f205802edd0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.915+0000 7f206ef3e640 1 -- 10.20.193.16:0/3785114471 shutdown_connections 2026-02-17T05:54:55.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.915+0000 7f206ef3e640 1 --2- 10.20.193.16:0/3785114471 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20680700f0 0x7f20680704d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.915+0000 7f206ef3e640 1 -- 10.20.193.16:0/3785114471 >> 10.20.193.16:0/3785114471 conn(0x7f20680cd770 msgr2=0x7f20680cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:55.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.915+0000 7f206ef3e640 1 -- 10.20.193.16:0/3785114471 shutdown_connections 2026-02-17T05:54:55.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.915+0000 7f206ef3e640 1 -- 10.20.193.16:0/3785114471 wait complete. 2026-02-17T05:54:55.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient: build_initial_monmap 2026-02-17T05:54:55.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient: monmap: 2026-02-17T05:54:55.921 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:54:55.921 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:55.921 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:54:55.921 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:54:55.921 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:55.921 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:55.922 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:55.922 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:55.922 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:55.922 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:55.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 librados: starting msgr at 2026-02-17T05:54:55.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 librados: starting objecter 2026-02-17T05:54:55.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 Processor -- start 2026-02-17T05:54:55.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 -- start start 2026-02-17T05:54:55.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 librados: setting wanted keys 2026-02-17T05:54:55.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 librados: calling monclient init 2026-02-17T05:54:55.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient: init 2026-02-17T05:54:55.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:55.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:54:55.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20680700f0 0x7f2068148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient(hunting): picked mon.noname-c con 0x7f20680700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:55.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20680c7fe0 0x7f20681f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient(hunting): picked mon.noname-a con 0x7f20680c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:55.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20681536f0 0x7f206814e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient(hunting): picked mon.noname-b con 0x7f20681536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:55.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2068074c00 con 0x7f20680c7fe0 2026-02-17T05:54:55.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f20680c5680 con 0x7f20681536f0 2026-02-17T05:54:55.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:55.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2068074d80 con 0x7f20680700f0 2026-02-17T05:54:55.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:55.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.916+0000 7f206ef3e640 10 monclient(hunting): authenticate will time out at 948.485927s 2026-02-17T05:54:55.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206ccb3640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20680700f0 0x7f2068148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f205ffff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20680c7fe0 0x7f20681f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f205ffff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20680c7fe0 0x7f20681f5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51044/0 (socket says 10.20.193.16:51044) 2026-02-17T05:54:55.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f205ffff640 1 -- 10.20.193.16:0/4102603095 learned_addr learned my addr 10.20.193.16:0/4102603095 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:54:55.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f205ffff640 10 monclient(hunting): get_auth_request con 0x7f20680c7fe0 auth_method 0 2026-02-17T05:54:55.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f205ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f205ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f205ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206ccb3640 10 monclient(hunting): get_auth_request con 0x7f20680700f0 auth_method 0 2026-02-17T05:54:55.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206ccb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206ccb3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206ccb3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206d4b4640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20681536f0 0x7f206814e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206d4b4640 10 monclient(hunting): get_auth_request con 0x7f20681536f0 auth_method 0 2026-02-17T05:54:55.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206d4b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:55.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206d4b4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:55.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206d4b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:55.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206ccb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206ccb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f206ccb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f205ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f205ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.917+0000 7f205ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.918+0000 7f206d4b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:55.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.918+0000 7f206d4b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:55.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.918+0000 7f206d4b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:55.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.919+0000 7f206ccb3640 10 monclient(hunting): handle_auth_done global_id 4223 payload 824 2026-02-17T05:54:55.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.919+0000 7f206ccb3640 1 -- 10.20.193.16:0/4102603095 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20681536f0 msgr2=0x7f206814e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.919+0000 7f206ccb3640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20681536f0 0x7f206814e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.919+0000 7f206ccb3640 1 -- 10.20.193.16:0/4102603095 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20680c7fe0 msgr2=0x7f20681f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:55.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.919+0000 7f206ccb3640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20680c7fe0 0x7f20681f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:55.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.919+0000 7f206ccb3640 10 monclient: _finish_hunting 0 2026-02-17T05:54:55.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.919+0000 7f206ccb3640 1 monclient: found mon.noname-c 2026-02-17T05:54:55.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.919+0000 7f206ccb3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:55.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.919+0000 7f206ccb3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.919+0000 7f206ccb3640 1 -- 10.20.193.16:0/4102603095 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20681f61f0 con 0x7f20680700f0 2026-02-17T05:54:55.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.920+0000 7f206ccb3640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20680700f0 0x7f2068148490 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f20680c9260 tx=0x7f20580047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:55.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.920+0000 7f205d7fa640 10 client.?.objecter ms_handle_connect 0x7f20680700f0 2026-02-17T05:54:55.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.920+0000 7f205d7fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:55.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f205d7fa640 1 -- 10.20.193.16:0/4102603095 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205803c040 con 0x7f20680700f0 2026-02-17T05:54:55.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f205d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f205d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:55.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f205d7fa640 10 monclient: dump: 2026-02-17T05:54:55.934 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:54:55.934 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.934 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.934 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.935 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.935 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:55.935 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.935 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.935 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.935 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:55.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f205d7fa640 10 monclient: _finish_auth 0 2026-02-17T05:54:55.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f205d7fa640 10 monclient: _check_auth_tickets 2026-02-17T05:54:55.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f205d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:55.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f206ef3e640 5 monclient: authenticate success, global_id 4223 2026-02-17T05:54:55.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f206ef3e640 10 monclient: _renew_subs 2026-02-17T05:54:55.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f206ef3e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f206ef3e640 1 -- 10.20.193.16:0/4102603095 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20681f64d0 con 0x7f20680700f0 2026-02-17T05:54:55.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f206ef3e640 10 client.4223.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:55.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f206ef3e640 10 monclient: _renew_subs 2026-02-17T05:54:55.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f206ef3e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f206ef3e640 1 -- 10.20.193.16:0/4102603095 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20681f6a90 con 0x7f20680700f0 2026-02-17T05:54:55.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.921+0000 7f206ef3e640 1 librados: init done 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f205d7fa640 1 -- 10.20.193.16:0/4102603095 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f205802fa20 con 0x7f20680700f0 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f205d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f205d7fa640 1 -- 10.20.193.16:0/4102603095 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058042810 con 0x7f20680700f0 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f205d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f205d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f205effd640 4 set_mon_vals no callback set 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f205d7fa640 10 monclient: dump: 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.937 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:55.938 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:55.938 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:54:55.938 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:55.938 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:55.938 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:55.938 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:54:55.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f206ef3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:55.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f206ef3e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:55.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f206ef3e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:55.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.922+0000 7f206ef3e640 1 -- 10.20.193.16:0/4102603095 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2068050ab0 con 0x7f20680700f0 2026-02-17T05:54:55.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.933+0000 7f205d7fa640 1 -- 10.20.193.16:0/4102603095 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f20580429b0 con 0x7f20680700f0 2026-02-17T05:54:55.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.933+0000 7f205d7fa640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f20380790b0 0x7f203807b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:55.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.933+0000 7f205ffff640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f20380790b0 0x7f203807b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:55.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.933+0000 7f205ffff640 10 monclient: get_auth_request con 0x7f20380790b0 auth_method 0 2026-02-17T05:54:55.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.934+0000 7f205d7fa640 10 client.4223.objecter ms_handle_connect 0x7f20380790b0 2026-02-17T05:54:55.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.934+0000 7f205ffff640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f20380790b0 0x7f203807b570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2068149800 tx=0x7f2050002990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:55.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.936+0000 7f205d7fa640 1 -- 10.20.193.16:0/4102603095 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f2058042cd0 con 0x7f20680700f0 2026-02-17T05:54:55.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.936+0000 7f205d7fa640 10 client.4223.objecter ms_dispatch2 0x7f2068000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:54:55.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.936+0000 7f205d7fa640 3 client.4223.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:54:55.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.936+0000 7f205d7fa640 3 client.4223.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:54:55.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:55.936+0000 7f205d7fa640 20 client.4223.objecter dump_active .. 0 homeless 2026-02-17T05:54:56.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.034+0000 7f205d7fa640 1 -- 10.20.193.16:0/4102603095 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2068050ab0 con 0x7f20680700f0 2026-02-17T05:54:56.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.034+0000 7f205d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:56.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.034+0000 7f205d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:56.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.124+0000 7f206ef3e640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:54:56.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.124+0000 7f206ef3e640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:54:56.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.124+0000 7f206ef3e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:54:56.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.124+0000 7f206ef3e640 1 -- 10.20.193.16:0/4102603095 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20680704d0 con 0x7f20680700f0 2026-02-17T05:54:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.150+0000 7f205d7fa640 1 -- 10.20.193.16:0/4102603095 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f205808f020 con 0x7f20680700f0 2026-02-17T05:54:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.150+0000 7f205d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:54:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.150+0000 7f205d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:56.150 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:54:56.151 INFO:teuthology.orchestra.run.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:54:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 10 librados: watch_flush enter 2026-02-17T05:54:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 10 librados: watch_flush exit 2026-02-17T05:54:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 20 client.4223.objecter shutdown clearing up homeless session... 2026-02-17T05:54:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 10 client.4223.objecter successfully canceled tick 2026-02-17T05:54:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 -- 10.20.193.16:0/4102603095 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f20380790b0 msgr2=0x7f203807b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f20380790b0 0x7f203807b570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2068149800 tx=0x7f2050002990 comp rx=0 tx=0).stop 2026-02-17T05:54:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 10 monclient: shutdown 2026-02-17T05:54:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 -- 10.20.193.16:0/4102603095 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20680700f0 msgr2=0x7f2068148490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20680700f0 0x7f2068148490 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f20680c9260 tx=0x7f20580047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 -- 10.20.193.16:0/4102603095 shutdown_connections 2026-02-17T05:54:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f20380790b0 0x7f203807b570 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20681536f0 0x7f206814e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20680c7fe0 0x7f20681f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 --2- 10.20.193.16:0/4102603095 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20680700f0 0x7f2068148490 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:56.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 -- 10.20.193.16:0/4102603095 >> 10.20.193.16:0/4102603095 conn(0x7f20680cd770 msgr2=0x7f2068148cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:56.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 -- 10.20.193.16:0/4102603095 shutdown_connections 2026-02-17T05:54:56.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 -- 10.20.193.16:0/4102603095 wait complete. 2026-02-17T05:54:56.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:54:56.152+0000 7f206ef3e640 1 librados: shutdown 2026-02-17T05:54:56.162 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-17T05:54:56.162 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-17T05:54:57.775 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.837+0000 7f49e977e640 10 monclient: get_monmap_and_config 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.837+0000 7f49e977e640 10 monclient: build_initial_monmap 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.837+0000 7f49e977e640 10 monclient: monmap: 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:54:57.838109+0000 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:54:57.838109+0000 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:54:57.837 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.837+0000 7f49e977e640 1 Processor -- start 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.837+0000 7f49e977e640 1 -- start start 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.837+0000 7f49e977e640 10 monclient: init 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 10 monclient: _add_conns ranks=[4,3,0,1,2,5] 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f49e415d3e0 0x7f49e415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 10 monclient(hunting): picked mon.noname-b con 0x7f49e415d3e0 addr v2:10.20.193.128:3300/0 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f49e40c9260 0x7f49e40c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 10 monclient(hunting): picked mon.noname-a con 0x7f49e40c9260 addr v2:10.20.193.16:3300/0 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f49e40c52d0 addr v1:10.20.193.16:6789/0 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e40c7ee0 con 0x7f49e40c52d0 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f49e40c8dd0 con 0x7f49e40c9260 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f49e40c81c0 con 0x7f49e415d3e0 2026-02-17T05:54:57.839 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e977e640 10 monclient(hunting): authenticate will time out at 950.407157s 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e27fc640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f49e40c9260 0x7f49e40c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.838+0000 7f49e2ffd640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f49e415d3e0 0x7f49e415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e2ffd640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f49e415d3e0 0x7f49e415d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:46840/0 (socket says 10.20.193.16:46840) 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e2ffd640 1 -- 10.20.193.16:0/3170037459 learned_addr learned my addr 10.20.193.16:0/3170037459 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e2ffd640 10 monclient(hunting): get_auth_request con 0x7f49e415d3e0 auth_method 0 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e27fc640 10 monclient(hunting): get_auth_request con 0x7f49e40c9260 auth_method 0 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:57.840 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.839+0000 7f49e27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.840+0000 7f49e2ffd640 10 monclient(hunting): handle_auth_done global_id 4192 payload 274 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.840+0000 7f49e2ffd640 1 -- 10.20.193.16:0/3170037459 >> v2:10.20.193.16:3300/0 conn(0x7f49e40c9260 msgr2=0x7f49e40c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.840+0000 7f49e2ffd640 1 --2- 10.20.193.16:0/3170037459 >> v2:10.20.193.16:3300/0 conn(0x7f49e40c9260 0x7f49e40c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.840+0000 7f49e2ffd640 1 -- 10.20.193.16:0/3170037459 >> v1:10.20.193.16:6789/0 conn(0x7f49e40c52d0 legacy=0x7f49e40c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.840+0000 7f49e2ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.840+0000 7f49e2ffd640 1 monclient: found mon.noname-b 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.840+0000 7f49e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.840+0000 7f49e2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:54:57.841 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.840+0000 7f49e2ffd640 1 -- 10.20.193.16:0/3170037459 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e40c84a0 con 0x7f49e415d3e0 2026-02-17T05:54:57.842 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.842+0000 7f49e2ffd640 1 --2- 10.20.193.16:0/3170037459 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f49e415d3e0 0x7f49e415d7c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f49e4069bb0 tx=0x7f49d802eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4053b1ceec2f59c5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 1 -- 10.20.193.16:0/3170037459 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d803c070 con 0x7f49e415d3e0 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 10 monclient: dump: 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:57.844 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:54:57.845 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:54:57.845 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:54:57.845 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:57.845 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 5 monclient: authenticate success, global_id 4192 2026-02-17T05:54:57.845 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:57.848 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 1 -- 10.20.193.16:0/3170037459 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d802fbe0 con 0x7f49e415d3e0 2026-02-17T05:54:57.848 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:57.848 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 1 -- 10.20.193.16:0/3170037459 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8037440 con 0x7f49e415d3e0 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e1ffb640 10 monclient: dump: 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 4 set_mon_vals no callback set 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 10 monclient: shutdown 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 1 -- 10.20.193.16:0/3170037459 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f49e415d3e0 msgr2=0x7f49e415d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 1 --2- 10.20.193.16:0/3170037459 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f49e415d3e0 0x7f49e415d7c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f49e4069bb0 tx=0x7f49d802eed0 comp rx=0 tx=0).stop 2026-02-17T05:54:57.849 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 1 -- 10.20.193.16:0/3170037459 shutdown_connections 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 1 --2- 10.20.193.16:0/3170037459 >> v2:10.20.193.16:3300/0 conn(0x7f49e40c9260 0x7f49e40c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.844+0000 7f49e977e640 1 --2- 10.20.193.16:0/3170037459 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f49e415d3e0 0x7f49e415d7c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 1 -- 10.20.193.16:0/3170037459 >> 10.20.193.16:0/3170037459 conn(0x7f49e40c9f40 msgr2=0x7f49e40ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 1 -- 10.20.193.16:0/3170037459 shutdown_connections 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 1 -- 10.20.193.16:0/3170037459 wait complete. 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 10 monclient: build_initial_monmap 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 10 monclient: monmap: 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 1 librados: starting msgr at 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 1 librados: starting objecter 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 1 Processor -- start 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 1 -- start start 2026-02-17T05:54:57.850 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 1 librados: setting wanted keys 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 1 librados: calling monclient init 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.845+0000 7f49e977e640 10 monclient: init 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f49e40c52d0 0x7f49e414f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 10 monclient(hunting): picked mon.noname-b con 0x7f49e40c52d0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f49e40c9260 0x7f49e41f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 10 monclient(hunting): picked mon.noname-a con 0x7f49e40c9260 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f49e415d3e0 0x7f49e41f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 10 monclient(hunting): picked mon.noname-c con 0x7f49e415d3e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49e40c9660 con 0x7f49e40c9260 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f49e40c8a60 con 0x7f49e40c52d0 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f49e40c97e0 con 0x7f49e415d3e0 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e977e640 10 monclient(hunting): authenticate will time out at 950.415096s 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e27fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f49e40c9260 0x7f49e41f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e27fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f49e40c9260 0x7f49e41f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51064/0 (socket says 10.20.193.16:51064) 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e27fc640 1 -- 10.20.193.16:0/704201184 learned_addr learned my addr 10.20.193.16:0/704201184 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e27fc640 10 monclient(hunting): get_auth_request con 0x7f49e40c9260 auth_method 0 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:57.851 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e2ffd640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f49e40c52d0 0x7f49e414f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e37fe640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f49e415d3e0 0x7f49e41f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e2ffd640 10 monclient(hunting): get_auth_request con 0x7f49e40c52d0 auth_method 0 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e37fe640 10 monclient(hunting): get_auth_request con 0x7f49e415d3e0 auth_method 0 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.846+0000 7f49e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.847+0000 7f49e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.847+0000 7f49e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.847+0000 7f49e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.847+0000 7f49e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.847+0000 7f49e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.847+0000 7f49e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.848+0000 7f49e27fc640 10 monclient(hunting): handle_auth_done global_id 4200 payload 824 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.848+0000 7f49e27fc640 1 -- 10.20.193.16:0/704201184 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f49e415d3e0 msgr2=0x7f49e41f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.848+0000 7f49e27fc640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f49e415d3e0 0x7f49e41f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:57.852 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.848+0000 7f49e27fc640 1 -- 10.20.193.16:0/704201184 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f49e40c52d0 msgr2=0x7f49e414f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:57.853 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.848+0000 7f49e27fc640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f49e40c52d0 0x7f49e414f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:57.853 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.848+0000 7f49e27fc640 10 monclient: _finish_hunting 0 2026-02-17T05:54:57.853 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.848+0000 7f49e27fc640 1 monclient: found mon.noname-a 2026-02-17T05:54:57.853 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.848+0000 7f49e27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:57.853 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.848+0000 7f49e27fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:54:57.853 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.848+0000 7f49e27fc640 1 -- 10.20.193.16:0/704201184 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e4149910 con 0x7f49e40c9260 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.849+0000 7f49e27fc640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f49e40c9260 0x7f49e41f4460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00ec10 tx=0x7f49cc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.849+0000 7f49c37fe640 10 client.?.objecter ms_handle_connect 0x7f49e40c9260 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.849+0000 7f49c37fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 1 -- 10.20.193.16:0/704201184 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc017070 con 0x7f49e40c9260 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 10 monclient: dump: 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:54:57.854 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 10 monclient: _finish_auth 0 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 1 -- 10.20.193.16:0/704201184 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49cc00cc40 con 0x7f49e40c9260 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 1 -- 10.20.193.16:0/704201184 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc013470 con 0x7f49e40c9260 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49e17fa640 4 set_mon_vals no callback set 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49c37fe640 10 monclient: dump: 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49e977e640 5 monclient: authenticate success, global_id 4200 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49e977e640 10 monclient: _renew_subs 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49e977e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49e977e640 1 -- 10.20.193.16:0/704201184 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e41482b0 con 0x7f49e40c9260 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49e977e640 10 client.4200.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49e977e640 10 monclient: _renew_subs 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49e977e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49e977e640 1 -- 10.20.193.16:0/704201184 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e4148870 con 0x7f49e40c9260 2026-02-17T05:54:57.855 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.851+0000 7f49e977e640 1 librados: init done 2026-02-17T05:54:57.856 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.852+0000 7f49e977e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:57.856 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.852+0000 7f49e977e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:57.856 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.852+0000 7f49e977e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:57.856 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.852+0000 7f49e977e640 1 -- 10.20.193.16:0/704201184 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e415b070 con 0x7f49e40c9260 2026-02-17T05:54:57.863 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.863+0000 7f49c37fe640 1 -- 10.20.193.16:0/704201184 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f49cc024060 con 0x7f49e40c9260 2026-02-17T05:54:57.863 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.863+0000 7f49c37fe640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f49b00790b0 0x7f49b007b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:57.863 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.863+0000 7f49e2ffd640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f49b00790b0 0x7f49b007b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:57.863 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.863+0000 7f49e2ffd640 10 monclient: get_auth_request con 0x7f49b00790b0 auth_method 0 2026-02-17T05:54:57.863 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.863+0000 7f49e2ffd640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f49b00790b0 0x7f49b007b570 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49e40c4790 tx=0x7f49d8039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:57.864 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.863+0000 7f49c37fe640 10 client.4200.objecter ms_handle_connect 0x7f49b00790b0 2026-02-17T05:54:57.865 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.865+0000 7f49c37fe640 1 -- 10.20.193.16:0/704201184 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f49cc05e3e0 con 0x7f49e40c9260 2026-02-17T05:54:57.865 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.865+0000 7f49c37fe640 10 client.4200.objecter ms_dispatch2 0x7f49e4000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:54:57.865 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.865+0000 7f49c37fe640 3 client.4200.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:54:57.865 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.865+0000 7f49c37fe640 3 client.4200.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:54:57.865 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.865+0000 7f49c37fe640 20 client.4200.objecter dump_active .. 0 homeless 2026-02-17T05:54:57.969 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.969+0000 7f49c37fe640 1 -- 10.20.193.16:0/704201184 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f49cc0623a0 con 0x7f49e40c9260 2026-02-17T05:54:57.969 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.969+0000 7f49c37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:57.970 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:57.969+0000 7f49c37fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:58.058 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.058+0000 7f49e977e640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:54:58.058 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.058+0000 7f49e977e640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:54:58.058 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.058+0000 7f49e977e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:54:58.059 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.058+0000 7f49e977e640 1 -- 10.20.193.16:0/704201184 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49e40c99b0 con 0x7f49e40c9260 2026-02-17T05:54:58.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.083+0000 7f49c37fe640 1 -- 10.20.193.16:0/704201184 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f49cc0666a0 con 0x7f49e40c9260 2026-02-17T05:54:58.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.083+0000 7f49c37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:54:58.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.083+0000 7f49c37fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:58.084 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:54:58.085 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:54:58.087 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 10 librados: watch_flush enter 2026-02-17T05:54:58.087 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 10 librados: watch_flush exit 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 20 client.4200.objecter shutdown clearing up homeless session... 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 10 client.4200.objecter successfully canceled tick 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 -- 10.20.193.16:0/704201184 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f49b00790b0 msgr2=0x7f49b007b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f49b00790b0 0x7f49b007b570 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49e40c4790 tx=0x7f49d8039040 comp rx=0 tx=0).stop 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 10 monclient: shutdown 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 -- 10.20.193.16:0/704201184 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f49e40c9260 msgr2=0x7f49e41f4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f49e40c9260 0x7f49e41f4460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00ec10 tx=0x7f49cc00c520 comp rx=0 tx=0).stop 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 -- 10.20.193.16:0/704201184 shutdown_connections 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f49b00790b0 0x7f49b007b570 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:58.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f49e415d3e0 0x7f49e41f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:58.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f49e40c9260 0x7f49e41f4460 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:58.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 --2- 10.20.193.16:0/704201184 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f49e40c52d0 0x7f49e414f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:58.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 -- 10.20.193.16:0/704201184 >> 10.20.193.16:0/704201184 conn(0x7f49e40c9f40 msgr2=0x7f49e40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:58.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 -- 10.20.193.16:0/704201184 shutdown_connections 2026-02-17T05:54:58.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 -- 10.20.193.16:0/704201184 wait complete. 2026-02-17T05:54:58.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:54:58.086+0000 7f49e977e640 1 librados: shutdown 2026-02-17T05:54:58.095 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:55:05.708 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:55:05.710 INFO:tasks.ceph.osd.14.trial148.stderr:--00:00:00:09.838 44710-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.711 INFO:tasks.ceph.osd.2.trial148.stderr:--00:00:00:09.838 44708-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.723 INFO:tasks.ceph.osd.6.trial148.stderr:--00:00:00:09.850 44707-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.725 INFO:tasks.ceph.osd.10.trial148.stderr:--00:00:00:09.853 44709-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.766 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.765+0000 7f98a80f3640 10 monclient: get_monmap_and_config 2026-02-17T05:55:05.766 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.765+0000 7f98a80f3640 10 monclient: build_initial_monmap 2026-02-17T05:55:05.766 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.765+0000 7f98a80f3640 10 monclient: monmap: 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:55:05.766879+0000 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:55:05.766879+0000 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:55:05.767 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:55:05.768 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:55:05.768 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:05.768 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.766+0000 7f98a80f3640 1 Processor -- start 2026-02-17T05:55:05.768 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.766+0000 7f98a80f3640 1 -- start start 2026-02-17T05:55:05.768 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.766+0000 7f98a80f3640 10 monclient: init 2026-02-17T05:55:05.770 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:05.770 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 10 monclient: _add_conns ranks=[0,2,5,4,1,3] 2026-02-17T05:55:05.770 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f98a015c3b0 addr v1:10.20.193.16:6789/0 2026-02-17T05:55:05.770 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f98a015c790 addr v1:10.20.193.148:6789/0 2026-02-17T05:55:05.770 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f98a006e6c0 0x7f98a0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:05.770 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 10 monclient(hunting): picked mon.noname-c con 0x7f98a006e6c0 addr v2:10.20.193.148:3300/0 2026-02-17T05:55:05.770 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a00c7270 con 0x7f98a015c3b0 2026-02-17T05:55:05.770 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a00c7550 con 0x7f98a015c790 2026-02-17T05:55:05.770 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:05.770 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f98a00c7830 con 0x7f98a006e6c0 2026-02-17T05:55:05.771 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:05.771 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a80f3640 10 monclient(hunting): authenticate will time out at 958.336126s 2026-02-17T05:55:05.771 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a6669640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f98a006e6c0 0x7f98a0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:05.771 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a5667640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f98a015c790 0x7f98a006dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:43402/0 (socket says 10.20.193.16:43402) 2026-02-17T05:55:05.771 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.767+0000 7f98a5667640 1 -- 10.20.193.16:0/3327599699 learned_addr learned my addr 10.20.193.16:0/3327599699 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:55:05.771 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.768+0000 7f98a6669640 10 monclient(hunting): get_auth_request con 0x7f98a006e6c0 auth_method 0 2026-02-17T05:55:05.771 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.768+0000 7f98a6669640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:05.771 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.768+0000 7f98a6669640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:05.771 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.768+0000 7f98a6669640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:05.772 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.768+0000 7f98a6669640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:05.772 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.768+0000 7f98a6669640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:05.772 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.768+0000 7f98a6669640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:05.773 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.770+0000 7f98a6669640 10 monclient(hunting): handle_auth_done global_id 4229 payload 274 2026-02-17T05:55:05.773 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.770+0000 7f98a6669640 1 -- 10.20.193.16:0/3327599699 >> v1:10.20.193.148:6789/0 conn(0x7f98a015c790 legacy=0x7f98a006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:05.773 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.770+0000 7f98a6669640 1 -- 10.20.193.16:0/3327599699 >> v1:10.20.193.16:6789/0 conn(0x7f98a015c3b0 legacy=0x7f98a00c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:05.773 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.770+0000 7f98a6669640 10 monclient: _finish_hunting 0 2026-02-17T05:55:05.773 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.770+0000 7f98a6669640 1 monclient: found mon.noname-c 2026-02-17T05:55:05.773 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.770+0000 7f98a6669640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:05.773 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.770+0000 7f98a6669640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:55:05.774 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.770+0000 7f98a6669640 1 -- 10.20.193.16:0/3327599699 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a006d4d0 con 0x7f98a006e6c0 2026-02-17T05:55:05.774 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.771+0000 7f98a6669640 1 --2- 10.20.193.16:0/3327599699 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 0x7f98a0070c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f989c009d90 tx=0x7f989c031470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b918d70d374032f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:05.778 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 1 -- 10.20.193.16:0/3327599699 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c007740 con 0x7f98a006e6c0 2026-02-17T05:55:05.778 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:05.778 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:05.778 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 10 monclient: dump: 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:05.779 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 10 monclient: _finish_auth 0 2026-02-17T05:55:05.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 10 monclient: _check_auth_tickets 2026-02-17T05:55:05.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:05.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 1 -- 10.20.193.16:0/3327599699 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989c0078e0 con 0x7f98a006e6c0 2026-02-17T05:55:05.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 5 monclient: authenticate success, global_id 4229 2026-02-17T05:55:05.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:05.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:05.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:05.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 1 -- 10.20.193.16:0/3327599699 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c007cb0 con 0x7f98a006e6c0 2026-02-17T05:55:05.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:05.781 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 4 set_mon_vals no callback set 2026-02-17T05:55:05.781 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:05.781 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a4e66640 10 monclient: dump: 2026-02-17T05:55:05.781 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:05.781 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:05.781 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:05.781 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:05.781 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:05.781 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:05.781 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 10 monclient: shutdown 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 1 -- 10.20.193.16:0/3327599699 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 msgr2=0x7f98a0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 1 --2- 10.20.193.16:0/3327599699 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 0x7f98a0070c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f989c009d90 tx=0x7f989c031470 comp rx=0 tx=0).stop 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 1 -- 10.20.193.16:0/3327599699 shutdown_connections 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 1 --2- 10.20.193.16:0/3327599699 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 0x7f98a0070c60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 1 -- 10.20.193.16:0/3327599699 >> 10.20.193.16:0/3327599699 conn(0x7f98a00c9f40 msgr2=0x7f98a00ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:05.782 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.773+0000 7f98a80f3640 1 -- 10.20.193.16:0/3327599699 shutdown_connections 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 1 -- 10.20.193.16:0/3327599699 wait complete. 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 10 monclient: build_initial_monmap 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 10 monclient: monmap: 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:55:05.783 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 1 librados: starting msgr at 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 1 librados: starting objecter 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 1 Processor -- start 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 1 -- start start 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 1 librados: setting wanted keys 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 1 librados: calling monclient init 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 10 monclient: init 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 0x7f98a014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.774+0000 7f98a80f3640 10 monclient(hunting): picked mon.noname-c con 0x7f98a006e6c0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:55:05.784 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f98a015c3b0 0x7f98a0073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 10 monclient(hunting): picked mon.noname-b con 0x7f98a015c3b0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f98a015c790 0x7f98a0073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 10 monclient(hunting): picked mon.noname-a con 0x7f98a015c790 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00c6cb0 con 0x7f98a015c790 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00c6b30 con 0x7f98a015c3b0 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00c6e30 con 0x7f98a006e6c0 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a80f3640 10 monclient(hunting): authenticate will time out at 958.344029s 2026-02-17T05:55:05.785 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5e68640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 0x7f98a014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5667640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f98a015c3b0 0x7f98a0073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5e68640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 0x7f98a014d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57678/0 (socket says 10.20.193.16:57678) 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5e68640 1 -- 10.20.193.16:0/3504106459 learned_addr learned my addr 10.20.193.16:0/3504106459 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5e68640 10 monclient(hunting): get_auth_request con 0x7f98a006e6c0 auth_method 0 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5e68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5e68640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5e68640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5667640 10 monclient(hunting): get_auth_request con 0x7f98a015c3b0 auth_method 0 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5667640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5667640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5667640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:05.786 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5e68640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5e68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.775+0000 7f98a5e68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.776+0000 7f98a5667640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.776+0000 7f98a5667640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.776+0000 7f98a5667640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.777+0000 7f98a5e68640 10 monclient(hunting): handle_auth_done global_id 4235 payload 824 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.777+0000 7f98a5e68640 1 -- 10.20.193.16:0/3504106459 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f98a015c3b0 msgr2=0x7f98a0073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.777+0000 7f98a5e68640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f98a015c3b0 0x7f98a0073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.777+0000 7f98a5e68640 1 -- 10.20.193.16:0/3504106459 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f98a015c790 msgr2=0x7f98a0073b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.777+0000 7f98a5e68640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f98a015c790 0x7f98a0073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.777+0000 7f98a5e68640 10 monclient: _finish_hunting 0 2026-02-17T05:55:05.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.777+0000 7f98a5e68640 1 monclient: found mon.noname-c 2026-02-17T05:55:05.788 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.777+0000 7f98a5e68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:05.788 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.777+0000 7f98a5e68640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:55:05.788 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.777+0000 7f98a5e68640 1 -- 10.20.193.16:0/3504106459 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a0146950 con 0x7f98a006e6c0 2026-02-17T05:55:05.788 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.778+0000 7f98a5e68640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 0x7f98a014d230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f989000b4d0 tx=0x7f989000b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:05.788 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.778+0000 7f988e7fc640 10 client.?.objecter ms_handle_connect 0x7f98a006e6c0 2026-02-17T05:55:05.788 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.778+0000 7f988e7fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:55:05.789 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 1 -- 10.20.193.16:0/3504106459 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890004480 con 0x7f98a006e6c0 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 10 monclient: dump: 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:05.790 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 5 monclient: authenticate success, global_id 4235 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 1 -- 10.20.193.16:0/3504106459 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890015070 con 0x7f98a006e6c0 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 1 -- 10.20.193.16:0/3504106459 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890002920 con 0x7f98a006e6c0 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988ffff640 4 set_mon_vals no callback set 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:05.791 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f988e7fc640 10 monclient: dump: 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 10 monclient: _renew_subs 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:05.792 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 1 -- 10.20.193.16:0/3504106459 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a0072840 con 0x7f98a006e6c0 2026-02-17T05:55:05.793 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 10 client.4235.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:55:05.793 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 10 monclient: _renew_subs 2026-02-17T05:55:05.793 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:05.793 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 1 -- 10.20.193.16:0/3504106459 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a0072e00 con 0x7f98a006e6c0 2026-02-17T05:55:05.793 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 1 librados: init done 2026-02-17T05:55:05.793 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:05.793 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:05.793 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:05.793 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.780+0000 7f98a80f3640 1 -- 10.20.193.16:0/3504106459 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a0070c60 con 0x7f98a006e6c0 2026-02-17T05:55:05.797 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.797+0000 7f988e7fc640 1 -- 10.20.193.16:0/3504106459 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f9890002ac0 con 0x7f98a006e6c0 2026-02-17T05:55:05.798 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.797+0000 7f988e7fc640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f98700790e0 0x7f987007b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:05.798 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.797+0000 7f98a5667640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f98700790e0 0x7f987007b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:05.798 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.797+0000 7f98a5667640 10 monclient: get_auth_request con 0x7f98700790e0 auth_method 0 2026-02-17T05:55:05.798 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.798+0000 7f988e7fc640 10 client.4235.objecter ms_handle_connect 0x7f98700790e0 2026-02-17T05:55:05.798 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.798+0000 7f98a5667640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f98700790e0 0x7f987007b5a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98a01f0d30 tx=0x7f9894009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:05.799 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.798+0000 7f988e7fc640 1 -- 10.20.193.16:0/3504106459 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f989005e3a0 con 0x7f98a006e6c0 2026-02-17T05:55:05.799 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.798+0000 7f988e7fc640 10 client.4235.objecter ms_dispatch2 0x7f98a0000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:55:05.799 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.798+0000 7f988e7fc640 3 client.4235.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:55:05.799 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.798+0000 7f988e7fc640 3 client.4235.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:55:05.799 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.798+0000 7f988e7fc640 20 client.4235.objecter dump_active .. 0 homeless 2026-02-17T05:55:05.805 INFO:tasks.ceph.osd.0.trial016.stderr:--00:00:00:10.038 44804-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.808 INFO:tasks.ceph.osd.11.trial164.stderr:--00:00:00:09.926 44920-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.809 INFO:tasks.ceph.osd.1.trial128.stderr:--00:00:00:09.946 44603-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.809 INFO:tasks.ceph.osd.7.trial164.stderr:--00:00:00:09.927 44919-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.809 INFO:tasks.ceph.osd.13.trial128.stderr:--00:00:00:09.941 44606-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.810 INFO:tasks.ceph.osd.3.trial164.stderr:--00:00:00:09.927 44918-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.810 INFO:tasks.ceph.osd.15.trial164.stderr:--00:00:00:09.927 44921-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.810 INFO:tasks.ceph.osd.5.trial128.stderr:--00:00:00:09.947 44604-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.811 INFO:tasks.ceph.osd.9.trial128.stderr:--00:00:00:09.944 44605-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.861 INFO:tasks.ceph.osd.8.trial016.stderr:--00:00:00:10.002 44881-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.880 INFO:tasks.ceph.osd.12.trial016.stderr:--00:00:00:10.020 44882-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.887 INFO:tasks.ceph.osd.4.trial016.stderr:--00:00:00:10.118 44805-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/ef/ee4a9c333705c9516f2b13939f82e08af4fbfc.debug 2026-02-17T05:55:05.903 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.902+0000 7f988e7fc640 1 -- 10.20.193.16:0/3504106459 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9890062a60 con 0x7f98a006e6c0 2026-02-17T05:55:05.903 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.902+0000 7f988e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:05.903 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.902+0000 7f988e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:05.996 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.996+0000 7f98a80f3640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:05.997 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.996+0000 7f98a80f3640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:05.997 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.996+0000 7f98a80f3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:05.997 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:05.996+0000 7f98a80f3640 1 -- 10.20.193.16:0/3504106459 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98a0150f70 con 0x7f98a006e6c0 2026-02-17T05:55:06.008 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.008+0000 7f988e7fc640 1 -- 10.20.193.16:0/3504106459 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f9890062360 con 0x7f98a006e6c0 2026-02-17T05:55:06.008 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.008+0000 7f988e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:06.008 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.008+0000 7f988e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:06.009 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:55:06.009 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:55:06.012 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 10 librados: watch_flush enter 2026-02-17T05:55:06.012 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 10 librados: watch_flush exit 2026-02-17T05:55:06.012 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 20 client.4235.objecter shutdown clearing up homeless session... 2026-02-17T05:55:06.012 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 10 client.4235.objecter successfully canceled tick 2026-02-17T05:55:06.012 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 1 -- 10.20.193.16:0/3504106459 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f98700790e0 msgr2=0x7f987007b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:06.012 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f98700790e0 0x7f987007b5a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98a01f0d30 tx=0x7f9894009040 comp rx=0 tx=0).stop 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 10 monclient: shutdown 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 1 -- 10.20.193.16:0/3504106459 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 msgr2=0x7f98a014d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 0x7f98a014d230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f989000b4d0 tx=0x7f989000b9a0 comp rx=0 tx=0).stop 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 1 -- 10.20.193.16:0/3504106459 shutdown_connections 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f98700790e0 0x7f987007b5a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f98a015c3b0 0x7f98a0073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f98a015c790 0x7f98a0073b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 1 --2- 10.20.193.16:0/3504106459 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98a006e6c0 0x7f98a014d230 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.010+0000 7f98a80f3640 1 -- 10.20.193.16:0/3504106459 >> 10.20.193.16:0/3504106459 conn(0x7f98a00c9f40 msgr2=0x7f98a00ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:06.013 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.011+0000 7f98a80f3640 1 -- 10.20.193.16:0/3504106459 shutdown_connections 2026-02-17T05:55:06.014 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.011+0000 7f98a80f3640 1 -- 10.20.193.16:0/3504106459 wait complete. 2026-02-17T05:55:06.014 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:06.011+0000 7f98a80f3640 1 librados: shutdown 2026-02-17T05:55:06.021 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:55:10.237 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:14.364 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.266 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:14.394 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.269 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:14.397 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.278 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:14.406 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.342 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:14.460 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.364 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:14.482 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.365 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:14.481 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.365 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:14.482 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.366 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:14.597 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.397 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:14.529 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.409 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:14.547 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.414 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:14.555 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.422 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:14.554 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.445 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:14.582 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.492 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:14.631 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.492 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:14.723 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.524 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:14.651 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.549 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:14.676 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.560 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:14.687 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.575 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:14.704 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.579 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:14.707 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.583 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:14.711 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.591 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:14.718 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.603 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:14.731 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.607 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:14.735 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.627 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:14.746 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.631 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:14.759 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.650 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:14.768 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.651 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:14.884 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.659 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:14.777 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.659 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:14.777 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.673 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:14.906 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.686 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:14.804 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.687 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:14.804 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.688 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:14.820 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.704 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:14.845 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.707 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:14.845 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.710 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:14.828 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.713 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:14.845 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.716 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:14.848 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.724 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:14.865 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.732 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:14.869 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.746 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:14.887 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.775 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:14.915 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.797 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:15.028 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.806 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:14.946 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.822 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:15.053 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.831 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:14.970 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.847 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:15.077 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.847 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:14.975 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.863 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:14.990 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.879 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:15.007 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.891 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:15.010 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.901 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:15.029 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.911 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:15.030 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.927 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:15.045 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.937 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:15.170 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.952 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:15.070 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.971 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:15.108 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.979 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:15.097 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.983 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:15.115 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.987 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:15.119 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:10.995 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:15.132 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.002 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:15.140 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.010 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:15.151 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.012 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:15.144 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.095 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:15.234 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.113 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:15.241 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.120 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:15.351 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.129 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:15.257 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.133 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:15.260 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.154 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:15.282 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.165 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:15.292 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.175 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:15.294 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.182 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:15.309 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.183 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:15.311 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.197 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:15.316 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.198 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:15.326 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.205 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:15.438 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.218 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:15.336 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.247 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:15.365 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.261 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:15.393 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.262 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:15.380 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.267 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:15.404 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.276 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:15.417 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.279 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:15.416 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.287 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:15.419 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.291 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:15.432 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.361 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:15.500 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.376 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:15.515 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.392 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:15.623 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.408 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:15.638 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.423 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:15.551 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.439 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:15.567 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.442 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:15.560 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.457 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:15.576 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.471 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:15.704 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.486 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:15.719 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.488 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:15.607 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.501 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:15.619 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.516 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:15.634 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.527 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:15.659 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.533 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:15.671 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.541 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:15.674 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.549 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:15.686 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.564 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:15.702 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.575 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:15.708 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.591 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:15.723 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.654 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:55:11.650+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:11.717 INFO:tasks.ceph.osd.10.trial148.stderr:2026-02-17T05:55:11.713+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:11.742 INFO:tasks.ceph.osd.2.trial148.stderr:2026-02-17T05:55:11.737+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:11.758 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:15.876 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.773 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:15.891 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.800 INFO:tasks.ceph.osd.15.trial164.stderr:2026-02-17T05:55:11.794+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:11.832 INFO:tasks.ceph.osd.8.trial016.stderr:2026-02-17T05:55:11.829+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:11.839 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:15.976 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.854 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:15.991 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:11.903 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:55:11.899+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:11.944 INFO:tasks.ceph.osd.4.trial016.stderr:2026-02-17T05:55:11.940+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:11.980 INFO:tasks.ceph.osd.6.trial148.stderr:2026-02-17T05:55:11.975+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:11.993 INFO:tasks.ceph.osd.11.trial164.stderr:2026-02-17T05:55:11.988+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:12.018 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:55:12.014+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:12.067 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:55:12.062+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:12.069 INFO:tasks.ceph.osd.9.trial128.stderr:2026-02-17T05:55:12.066+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:12.075 INFO:tasks.ceph.osd.1.trial128.stderr:2026-02-17T05:55:12.072+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:12.122 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:55:12.118+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:12.308 INFO:tasks.ceph.osd.3.trial164.stderr:2026-02-17T05:55:12.303+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:12.387 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:55:12.383+0000 77b6940 -1 Falling back to public interface 2026-02-17T05:55:13.634 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.697+0000 7ff835fa1640 10 monclient: get_monmap_and_config 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.697+0000 7ff835fa1640 10 monclient: build_initial_monmap 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.697+0000 7ff835fa1640 10 monclient: monmap: 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:55:13.698246+0000 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:55:13.698246+0000 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:55:13.697 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.697+0000 7ff835fa1640 1 Processor -- start 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.697+0000 7ff835fa1640 1 -- start start 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.697+0000 7ff835fa1640 10 monclient: init 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 10 monclient: _add_conns ranks=[2,5,0,3,1,4] 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff83015f4e0 addr v1:10.20.193.148:6789/0 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7ff83015f8c0 0x7ff8300cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 10 monclient(hunting): picked mon.noname-c con 0x7ff83015f8c0 addr v2:10.20.193.148:3300/0 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff83015d170 addr v1:10.20.193.16:6789/0 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8300c8f30 con 0x7ff83015d170 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8300c9210 con 0x7ff83015f4e0 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7ff8300c8ce0 con 0x7ff83015f8c0 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:13.698 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff835fa1640 10 monclient(hunting): authenticate will time out at 966.267428s 2026-02-17T05:55:13.699 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff82ffff640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7ff83015d170 0x7ff83015d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:38972/0 (socket says 10.20.193.16:38972) 2026-02-17T05:55:13.699 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff82ffff640 1 -- 10.20.193.16:0/1189279162 learned_addr learned my addr 10.20.193.16:0/1189279162 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:55:13.699 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.698+0000 7ff82effd640 1 --2- 10.20.193.16:0/1189279162 >> v2:10.20.193.148:3300/0 conn(0x7ff83015f8c0 0x7ff8300cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:13.699 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.699+0000 7ff82effd640 10 monclient(hunting): get_auth_request con 0x7ff83015f8c0 auth_method 0 2026-02-17T05:55:13.699 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.699+0000 7ff82effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:13.699 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.699+0000 7ff82effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:13.699 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.699+0000 7ff82effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:13.699 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.699+0000 7ff82effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:13.699 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.699+0000 7ff82effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:13.699 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.699+0000 7ff82effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:13.700 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.700+0000 7ff82e7fc640 1 -- 10.20.193.16:0/1189279162 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2090448761 0 0) 0x7ff8300c8f30 con 0x7ff83015d170 2026-02-17T05:55:13.700 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.700+0000 7ff82e7fc640 10 monclient(hunting): _negotiate 2026-02-17T05:55:13.700 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.700+0000 7ff82e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:13.700 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.700+0000 7ff82e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:13.700 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.700+0000 7ff82e7fc640 10 monclient(hunting): my global_id is 4251 2026-02-17T05:55:13.700 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.700+0000 7ff82e7fc640 1 -- 10.20.193.16:0/1189279162 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff814003620 con 0x7ff83015d170 2026-02-17T05:55:13.701 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.701+0000 7ff82effd640 10 monclient(hunting): handle_auth_done global_id 4277 payload 274 2026-02-17T05:55:13.701 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.701+0000 7ff82effd640 1 -- 10.20.193.16:0/1189279162 >> v1:10.20.193.148:6789/0 conn(0x7ff83015f4e0 legacy=0x7ff8300d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:13.701 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.701+0000 7ff82effd640 1 -- 10.20.193.16:0/1189279162 >> v1:10.20.193.16:6789/0 conn(0x7ff83015d170 legacy=0x7ff83015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:13.701 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.701+0000 7ff82effd640 10 monclient: _finish_hunting 0 2026-02-17T05:55:13.701 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.701+0000 7ff82effd640 1 monclient: found mon.noname-c 2026-02-17T05:55:13.701 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.701+0000 7ff82effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:13.701 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.701+0000 7ff82effd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:55:13.701 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.701+0000 7ff82effd640 1 -- 10.20.193.16:0/1189279162 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8300c94f0 con 0x7ff83015f8c0 2026-02-17T05:55:13.703 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.703+0000 7ff82effd640 1 --2- 10.20.193.16:0/1189279162 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff83015f8c0 0x7ff8300cb6d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8300c8e60 tx=0x7ff82002ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f212e8862677932d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:13.705 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 1 -- 10.20.193.16:0/1189279162 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82003c070 con 0x7ff83015f8c0 2026-02-17T05:55:13.705 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:13.705 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:13.705 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 10 monclient: dump: 2026-02-17T05:55:13.705 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:13.705 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:13.705 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:13.705 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 1 -- 10.20.193.16:0/1189279162 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82002fb70 con 0x7ff83015f8c0 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 1 -- 10.20.193.16:0/1189279162 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820037440 con 0x7ff83015f8c0 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff82e7fc640 10 monclient: dump: 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff835fa1640 5 monclient: authenticate success, global_id 4277 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff835fa1640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:13.706 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.705+0000 7ff835fa1640 4 set_mon_vals no callback set 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 10 monclient: shutdown 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 1 -- 10.20.193.16:0/1189279162 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff83015f8c0 msgr2=0x7ff8300cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 1 --2- 10.20.193.16:0/1189279162 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff83015f8c0 0x7ff8300cb6d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8300c8e60 tx=0x7ff82002ee60 comp rx=0 tx=0).stop 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 1 -- 10.20.193.16:0/1189279162 shutdown_connections 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 1 --2- 10.20.193.16:0/1189279162 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff83015f8c0 0x7ff8300cb6d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 1 -- 10.20.193.16:0/1189279162 >> 10.20.193.16:0/1189279162 conn(0x7ff830074130 msgr2=0x7ff830074540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 1 -- 10.20.193.16:0/1189279162 shutdown_connections 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 1 -- 10.20.193.16:0/1189279162 wait complete. 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 10 monclient: build_initial_monmap 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.706+0000 7ff835fa1640 10 monclient: monmap: 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 librados: starting msgr at 2026-02-17T05:55:13.707 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 librados: starting objecter 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 Processor -- start 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 -- start start 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 librados: setting wanted keys 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 librados: calling monclient init 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient: init 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff83015d170 0x7ff830146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient(hunting): picked mon.noname-b con 0x7ff83015d170 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff83015f4e0 0x7ff8301f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient(hunting): picked mon.noname-c con 0x7ff83015f4e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff83015f8c0 0x7ff8301f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient(hunting): picked mon.noname-a con 0x7ff83015f8c0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8300c9d90 con 0x7ff83015f8c0 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7ff8300c9ab0 con 0x7ff83015d170 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8300cbc10 con 0x7ff83015f4e0 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.707+0000 7ff835fa1640 10 monclient(hunting): authenticate will time out at 966.276878s 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82ffff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff83015f8c0 0x7ff8301f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82f7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff83015d170 0x7ff830146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:13.708 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82effd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff83015f4e0 0x7ff8301f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82f7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff83015d170 0x7ff830146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:34614/0 (socket says 10.20.193.16:34614) 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82f7fe640 1 -- 10.20.193.16:0/3874864964 learned_addr learned my addr 10.20.193.16:0/3874864964 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82effd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff83015f4e0 0x7ff8301f5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:53412/0 (socket says 10.20.193.16:53412) 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82f7fe640 10 monclient(hunting): get_auth_request con 0x7ff83015d170 auth_method 0 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82effd640 10 monclient(hunting): get_auth_request con 0x7ff83015f4e0 auth_method 0 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82ffff640 10 monclient(hunting): get_auth_request con 0x7ff83015f8c0 auth_method 0 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:13.709 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.708+0000 7ff82ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:13.710 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.710+0000 7ff82f7fe640 10 monclient(hunting): handle_auth_done global_id 4258 payload 824 2026-02-17T05:55:13.710 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.710+0000 7ff82f7fe640 1 -- 10.20.193.16:0/3874864964 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff83015f4e0 msgr2=0x7ff8301f5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:13.710 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.710+0000 7ff82f7fe640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff83015f4e0 0x7ff8301f5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:13.710 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.710+0000 7ff82f7fe640 1 -- 10.20.193.16:0/3874864964 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff83015f8c0 msgr2=0x7ff8301f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:13.710 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.710+0000 7ff82f7fe640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff83015f8c0 0x7ff8301f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:13.710 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.710+0000 7ff82f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:13.710 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.710+0000 7ff82f7fe640 1 monclient: found mon.noname-b 2026-02-17T05:55:13.710 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.710+0000 7ff82f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:13.710 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.710+0000 7ff82f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:55:13.710 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.710+0000 7ff82f7fe640 1 -- 10.20.193.16:0/3874864964 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8301432d0 con 0x7ff83015d170 2026-02-17T05:55:13.711 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.711+0000 7ff82f7fe640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff83015d170 0x7ff830146550 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00c700 tx=0x7ff81c00cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:13.711 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.711+0000 7ff80bfff640 10 client.?.objecter ms_handle_connect 0x7ff83015d170 2026-02-17T05:55:13.711 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.711+0000 7ff80bfff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 1 -- 10.20.193.16:0/3874864964 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0052d0 con 0x7ff83015d170 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 10 monclient: dump: 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 10 monclient: _finish_auth 0 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 1 -- 10.20.193.16:0/3874864964 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81c005470 con 0x7ff83015d170 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 1 -- 10.20.193.16:0/3874864964 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0038d0 con 0x7ff83015d170 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 5 monclient: authenticate success, global_id 4258 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff82dffb640 4 set_mon_vals no callback set 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 10 monclient: _renew_subs 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 1 -- 10.20.193.16:0/3874864964 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8301435b0 con 0x7ff83015d170 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff80bfff640 10 monclient: dump: 2026-02-17T05:55:13.713 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 10 client.4258.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 10 monclient: _renew_subs 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 1 -- 10.20.193.16:0/3874864964 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8301f4a90 con 0x7ff83015d170 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 1 librados: init done 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:13.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.713+0000 7ff835fa1640 1 -- 10.20.193.16:0/3874864964 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8300d1550 con 0x7ff83015d170 2026-02-17T05:55:13.725 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.725+0000 7ff80bfff640 1 -- 10.20.193.16:0/3874864964 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7ff81c003a70 con 0x7ff83015d170 2026-02-17T05:55:13.725 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.725+0000 7ff80bfff640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff8000790e0 0x7ff80007b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:13.725 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.725+0000 7ff82effd640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff8000790e0 0x7ff80007b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:13.726 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.726+0000 7ff82effd640 10 monclient: get_auth_request con 0x7ff8000790e0 auth_method 0 2026-02-17T05:55:13.726 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.726+0000 7ff82effd640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff8000790e0 0x7ff80007b5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff820039000 tx=0x7ff820033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:13.726 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.726+0000 7ff80bfff640 10 client.4258.objecter ms_handle_connect 0x7ff8000790e0 2026-02-17T05:55:13.728 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.728+0000 7ff80bfff640 1 -- 10.20.193.16:0/3874864964 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7ff81c05e270 con 0x7ff83015d170 2026-02-17T05:55:13.728 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.728+0000 7ff80bfff640 10 client.4258.objecter ms_dispatch2 0x7ff830000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:55:13.728 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.728+0000 7ff80bfff640 3 client.4258.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:55:13.728 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.728+0000 7ff80bfff640 3 client.4258.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:55:13.728 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.728+0000 7ff80bfff640 20 client.4258.objecter dump_active .. 0 homeless 2026-02-17T05:55:13.833 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.833+0000 7ff80bfff640 1 -- 10.20.193.16:0/3874864964 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff81c062230 con 0x7ff83015d170 2026-02-17T05:55:13.833 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.833+0000 7ff80bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:13.833 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.833+0000 7ff80bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:13.927 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.927+0000 7ff835fa1640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:13.928 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.927+0000 7ff835fa1640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:13.928 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.927+0000 7ff835fa1640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:13.928 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.927+0000 7ff835fa1640 1 -- 10.20.193.16:0/3874864964 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8301501a0 con 0x7ff83015d170 2026-02-17T05:55:13.975 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.975+0000 7ff80bfff640 1 -- 10.20.193.16:0/3874864964 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7ff81c066530 con 0x7ff83015d170 2026-02-17T05:55:13.975 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.975+0000 7ff80bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:13.975 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.975+0000 7ff80bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:13.976 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:55:13.976 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 10 librados: watch_flush enter 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 10 librados: watch_flush exit 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 20 client.4258.objecter shutdown clearing up homeless session... 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 10 client.4258.objecter successfully canceled tick 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 -- 10.20.193.16:0/3874864964 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff8000790e0 msgr2=0x7ff80007b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff8000790e0 0x7ff80007b5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff820039000 tx=0x7ff820033000 comp rx=0 tx=0).stop 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 10 monclient: shutdown 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 -- 10.20.193.16:0/3874864964 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff83015d170 msgr2=0x7ff830146550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff83015d170 0x7ff830146550 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00c700 tx=0x7ff81c00cbd0 comp rx=0 tx=0).stop 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 -- 10.20.193.16:0/3874864964 shutdown_connections 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff8000790e0 0x7ff80007b5a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:13.979 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff83015f8c0 0x7ff8301f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:13.980 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff83015f4e0 0x7ff8301f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:13.980 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 --2- 10.20.193.16:0/3874864964 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff83015d170 0x7ff830146550 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:13.980 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 -- 10.20.193.16:0/3874864964 >> 10.20.193.16:0/3874864964 conn(0x7ff830074130 msgr2=0x7ff8301506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:13.980 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 -- 10.20.193.16:0/3874864964 shutdown_connections 2026-02-17T05:55:13.980 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 -- 10.20.193.16:0/3874864964 wait complete. 2026-02-17T05:55:13.980 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:13.979+0000 7ff835fa1640 1 librados: shutdown 2026-02-17T05:55:13.988 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:55:16.459 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:20.586 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.516 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:20.643 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.536 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:20.663 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.656 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:20.774 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.769 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:20.896 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.770 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:20.898 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.829 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:20.969 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.833 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:20.960 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.850 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:20.977 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.892 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:21.010 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.940 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:21.058 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.953 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:21.091 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.953 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:21.085 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.954 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:21.185 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.954 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:21.094 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:16.978 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:21.096 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.023 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:21.155 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.043 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:21.275 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.080 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:21.208 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.091 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:21.219 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.121 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:21.248 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.137 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:21.278 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.147 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:21.275 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.188 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:21.307 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.210 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:21.329 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.251 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:21.369 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.264 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:21.401 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.264 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:21.396 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.265 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:21.496 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.269 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:21.387 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.269 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:21.407 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.271 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:21.411 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.336 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:21.468 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.353 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:21.586 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.368 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:21.496 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.371 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:21.499 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.395 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:21.522 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.418 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:21.546 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.422 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:21.563 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.498 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:21.616 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.503 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:21.621 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.546 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:21.664 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.550 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:21.781 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.556 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:21.694 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.557 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:21.689 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.566 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:21.684 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.581 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:21.720 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.590 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:21.727 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.632 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:21.759 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.646 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:21.778 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.658 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:21.786 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.662 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:21.895 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.675 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:21.802 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.690 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:21.831 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.691 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:21.819 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.770 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:21.889 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.786 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:21.904 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.810 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:21.928 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.814 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:22.044 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.827 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:21.964 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.835 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:21.967 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.844 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:21.963 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.855 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:21.995 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.895 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:22.023 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.896 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:22.033 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.924 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:22.056 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.932 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:22.060 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.936 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:22.169 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.938 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:22.065 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.964 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:22.104 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:17.965 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:22.093 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.037 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:22.155 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.052 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:22.170 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.077 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:22.308 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.085 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:22.203 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.097 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:22.234 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.105 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:22.237 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.122 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:22.240 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.129 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:22.269 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.163 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:22.291 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.170 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:22.307 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.199 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:22.331 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.205 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:22.332 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.210 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:22.442 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.213 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:22.341 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.232 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:22.373 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.241 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:22.370 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.300 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:22.419 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.322 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:22.440 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.345 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:22.575 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.353 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:22.471 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.361 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:22.498 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.383 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:22.515 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.395 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:22.514 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.403 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:22.543 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.427 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:22.555 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.444 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:22.581 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.469 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:22.596 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.475 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:22.607 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.484 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:22.716 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.496 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:22.624 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.507 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:22.636 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.510 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:22.651 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.564 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:22.682 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.591 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:22.709 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.608 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:22.839 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.619 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:22.737 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.624 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:22.762 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.650 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:22.782 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.671 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:22.789 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.681 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:22.820 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.691 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:22.818 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.718 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:22.855 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.732 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:22.859 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.747 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:22.880 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.758 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:22.990 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.772 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:22.900 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.778 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:22.906 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.785 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:22.925 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.827 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:22.945 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.855 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:22.973 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.871 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:23.102 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.883 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:23.001 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.888 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:23.025 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.916 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:23.048 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.946 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:23.064 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.950 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:23.089 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.954 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:23.082 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:18.996 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:23.133 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.032 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:23.264 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.033 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:23.164 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.055 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:23.195 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.085 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:23.213 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.090 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:23.208 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.121 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:23.239 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.134 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:23.365 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.140 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:23.268 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.140 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:23.268 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.151 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:23.288 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.159 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:23.286 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.169 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:23.297 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.180 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:23.308 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.183 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:23.316 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.194 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:23.322 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.199 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:23.326 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.207 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:23.334 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.224 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:23.363 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.226 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:23.344 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.240 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:23.358 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.272 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:23.409 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.309 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:23.441 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.310 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:23.542 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.328 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:23.456 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.374 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:23.502 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.395 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:23.513 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.408 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:23.535 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.416 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:23.557 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.445 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:23.564 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.451 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:23.579 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.483 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:23.601 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.494 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:23.622 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.495 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:23.726 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.499 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:23.627 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.504 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:23.642 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.532 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:23.650 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.547 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:23.679 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.550 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:23.687 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.552 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:23.690 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.570 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:23.688 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.587 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:23.724 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.604 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:23.722 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.605 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:23.745 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.610 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:23.728 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.615 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:23.733 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.656 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:23.774 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.686 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:23.919 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.688 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:23.806 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.693 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:23.821 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.694 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:23.826 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.699 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:23.827 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.706 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:23.847 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.728 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:23.846 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.732 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:23.965 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.733 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:23.851 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.739 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:23.867 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.755 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:23.873 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.755 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:23.873 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.756 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:23.988 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.771 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:23.899 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.776 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:23.904 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.791 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:24.022 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.812 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:24.043 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.817 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:23.935 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.844 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:23.976 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.865 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:24.002 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.870 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:24.008 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.878 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:24.010 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.891 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:24.031 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.913 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:24.053 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.918 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:24.050 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.923 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:24.055 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.926 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:24.063 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.939 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:24.079 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.944 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:24.084 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.984 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:24.121 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:19.987 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:24.128 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.001 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:24.133 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.023 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:24.160 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.026 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:24.144 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.026 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:24.259 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.031 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:24.149 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.032 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:24.264 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.055 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:24.191 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.059 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:24.196 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.087 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:24.318 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.092 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:24.323 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.102 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:24.220 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.138 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:24.256 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.143 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:24.261 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.259 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:24.400 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.264 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:24.405 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.282 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:24.415 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.317 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:24.449 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:20.321 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:24.454 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:21.598 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:55:21.658 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.657+0000 7f33a98a0640 10 monclient: get_monmap_and_config 2026-02-17T05:55:21.659 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.657+0000 7f33a98a0640 10 monclient: build_initial_monmap 2026-02-17T05:55:21.659 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.658+0000 7f33a98a0640 10 monclient: monmap: 2026-02-17T05:55:21.659 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:21.659 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:21.659 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:55:21.659013+0000 2026-02-17T05:55:21.659 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:55:21.659013+0000 2026-02-17T05:55:21.659 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:21.659 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:21.659 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:55:21.660 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:55:21.660 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:55:21.660 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:55:21.660 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:55:21.660 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:55:21.660 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:21.660 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.658+0000 7f33a98a0640 1 Processor -- start 2026-02-17T05:55:21.660 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.658+0000 7f33a98a0640 1 -- start start 2026-02-17T05:55:21.660 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.658+0000 7f33a98a0640 10 monclient: init 2026-02-17T05:55:21.663 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.658+0000 7f33a98a0640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:21.663 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 10 monclient: _add_conns ranks=[2,5,1,4,0,3] 2026-02-17T05:55:21.663 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f33a415c080 addr v1:10.20.193.148:6789/0 2026-02-17T05:55:21.663 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f33a40cba00 0x7f33a415c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:21.663 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 10 monclient(hunting): picked mon.noname-c con 0x7f33a40cba00 addr v2:10.20.193.148:3300/0 2026-02-17T05:55:21.663 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f33a40c8ce0 addr v1:10.20.193.128:6789/0 2026-02-17T05:55:21.663 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a40c9580 con 0x7f33a40c8ce0 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a40c9860 con 0x7f33a415c080 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f33a40ca3e0 con 0x7f33a40cba00 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a98a0640 10 monclient(hunting): authenticate will time out at 974.228076s 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a37fe640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f33a40c8ce0 0x7f33a415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:52212/0 (socket says 10.20.193.16:52212) 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a37fe640 1 -- 10.20.193.16:0/427202917 learned_addr learned my addr 10.20.193.16:0/427202917 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.659+0000 7f33a27fc640 1 --2- 10.20.193.16:0/427202917 >> v2:10.20.193.148:3300/0 conn(0x7f33a40cba00 0x7f33a415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.660+0000 7f33a27fc640 10 monclient(hunting): get_auth_request con 0x7f33a40cba00 auth_method 0 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.660+0000 7f33a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.660+0000 7f33a27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:21.664 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.660+0000 7f33a27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:21.665 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.660+0000 7f33a27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:21.665 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.660+0000 7f33a27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:21.665 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.660+0000 7f33a27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:21.665 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a1ffb640 1 -- 10.20.193.16:0/427202917 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1819565559 0 0) 0x7f33a40c9580 con 0x7f33a40c8ce0 2026-02-17T05:55:21.665 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a1ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:55:21.665 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:21.665 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:21.665 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a1ffb640 10 monclient(hunting): my global_id is 4261 2026-02-17T05:55:21.665 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a1ffb640 1 -- 10.20.193.16:0/427202917 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3390003620 con 0x7f33a40c8ce0 2026-02-17T05:55:21.665 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a27fc640 10 monclient(hunting): handle_auth_done global_id 4286 payload 274 2026-02-17T05:55:21.666 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a27fc640 1 -- 10.20.193.16:0/427202917 >> v1:10.20.193.148:6789/0 conn(0x7f33a415c080 legacy=0x7f33a40d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:21.666 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a27fc640 1 -- 10.20.193.16:0/427202917 >> v1:10.20.193.128:6789/0 conn(0x7f33a40c8ce0 legacy=0x7f33a415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:21.666 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a27fc640 10 monclient: _finish_hunting 0 2026-02-17T05:55:21.666 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a27fc640 1 monclient: found mon.noname-c 2026-02-17T05:55:21.666 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:21.666 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a27fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:55:21.666 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.661+0000 7f33a27fc640 1 -- 10.20.193.16:0/427202917 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a40c9b40 con 0x7f33a40cba00 2026-02-17T05:55:21.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.663+0000 7f33a27fc640 1 --2- 10.20.193.16:0/427202917 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a40cba00 0x7f33a415c460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f338c0098d0 tx=0x7f338c02eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b4b23a6572e436 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:21.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 1 -- 10.20.193.16:0/427202917 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c03c070 con 0x7f33a40cba00 2026-02-17T05:55:21.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 10 monclient: dump: 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:21.672 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:21.673 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:21.673 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:55:21.673 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:55:21.673 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:21.673 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 1 -- 10.20.193.16:0/427202917 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f338c02fba0 con 0x7f33a40cba00 2026-02-17T05:55:21.673 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:21.673 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 1 -- 10.20.193.16:0/427202917 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c037440 con 0x7f33a40cba00 2026-02-17T05:55:21.673 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 5 monclient: authenticate success, global_id 4286 2026-02-17T05:55:21.673 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 4 set_mon_vals no callback set 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a1ffb640 10 monclient: dump: 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:21.674 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 10 monclient: shutdown 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 1 -- 10.20.193.16:0/427202917 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a40cba00 msgr2=0x7f33a415c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 1 --2- 10.20.193.16:0/427202917 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a40cba00 0x7f33a415c460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f338c0098d0 tx=0x7f338c02eed0 comp rx=0 tx=0).stop 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 1 -- 10.20.193.16:0/427202917 shutdown_connections 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 1 --2- 10.20.193.16:0/427202917 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a40cba00 0x7f33a415c460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 1 -- 10.20.193.16:0/427202917 >> 10.20.193.16:0/427202917 conn(0x7f33a40740d0 msgr2=0x7f33a40744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:21.675 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 1 -- 10.20.193.16:0/427202917 shutdown_connections 2026-02-17T05:55:21.676 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.665+0000 7f33a98a0640 1 -- 10.20.193.16:0/427202917 wait complete. 2026-02-17T05:55:21.676 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.666+0000 7f33a98a0640 10 monclient: build_initial_monmap 2026-02-17T05:55:21.676 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.666+0000 7f33a98a0640 10 monclient: monmap: 2026-02-17T05:55:21.676 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:21.676 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:21.676 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:55:21.676 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:55:21.676 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:21.676 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:21.676 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.666+0000 7f33a98a0640 1 librados: starting msgr at 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.666+0000 7f33a98a0640 1 librados: starting objecter 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.666+0000 7f33a98a0640 1 Processor -- start 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.666+0000 7f33a98a0640 1 -- start start 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.666+0000 7f33a98a0640 1 librados: setting wanted keys 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.666+0000 7f33a98a0640 1 librados: calling monclient init 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.666+0000 7f33a98a0640 10 monclient: init 2026-02-17T05:55:21.677 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f33a40c8ce0 0x7f33a4147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 10 monclient(hunting): picked mon.noname-b con 0x7f33a40c8ce0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33a40cba00 0x7f33a41474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 10 monclient(hunting): picked mon.noname-a con 0x7f33a40cba00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a415c080 0x7f33a40d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 10 monclient(hunting): picked mon.noname-c con 0x7f33a415c080 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40c9150 con 0x7f33a40cba00 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40ca100 con 0x7f33a40c8ce0 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:21.678 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40c92d0 con 0x7f33a415c080 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a98a0640 10 monclient(hunting): authenticate will time out at 974.236269s 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a2ffd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f33a40c8ce0 0x7f33a4147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a37fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a415c080 0x7f33a40d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a27fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33a40cba00 0x7f33a41474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a37fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a415c080 0x7f33a40d1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:38038/0 (socket says 10.20.193.16:38038) 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a37fe640 1 -- 10.20.193.16:0/3104765082 learned_addr learned my addr 10.20.193.16:0/3104765082 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a37fe640 10 monclient(hunting): get_auth_request con 0x7f33a415c080 auth_method 0 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:21.679 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a2ffd640 10 monclient(hunting): get_auth_request con 0x7f33a40c8ce0 auth_method 0 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.667+0000 7f33a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a27fc640 10 monclient(hunting): get_auth_request con 0x7f33a40cba00 auth_method 0 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:21.680 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.668+0000 7f33a27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.669+0000 7f33a37fe640 10 monclient(hunting): handle_auth_done global_id 4289 payload 824 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.669+0000 7f33a37fe640 1 -- 10.20.193.16:0/3104765082 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f33a40c8ce0 msgr2=0x7f33a4147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.669+0000 7f33a37fe640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f33a40c8ce0 0x7f33a4147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.669+0000 7f33a37fe640 1 -- 10.20.193.16:0/3104765082 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33a40cba00 msgr2=0x7f33a41474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.669+0000 7f33a37fe640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33a40cba00 0x7f33a41474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.669+0000 7f33a37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.669+0000 7f33a37fe640 1 monclient: found mon.noname-c 2026-02-17T05:55:21.681 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.669+0000 7f33a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:21.682 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.669+0000 7f33a37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:55:21.682 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.669+0000 7f33a37fe640 1 -- 10.20.193.16:0/3104765082 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a4158170 con 0x7f33a415c080 2026-02-17T05:55:21.682 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.670+0000 7f33a37fe640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a415c080 0x7f33a40d1f90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f339400caa0 tx=0x7f339400cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:21.682 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.670+0000 7f337bfff640 10 client.?.objecter ms_handle_connect 0x7f33a415c080 2026-02-17T05:55:21.682 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.670+0000 7f337bfff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:55:21.684 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.671+0000 7f337bfff640 1 -- 10.20.193.16:0/3104765082 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339400abe0 con 0x7f33a415c080 2026-02-17T05:55:21.684 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.671+0000 7f337bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:21.684 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.671+0000 7f337bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:21.684 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.671+0000 7f337bfff640 10 monclient: dump: 2026-02-17T05:55:21.684 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:21.684 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:21.684 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:21.684 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:21.684 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.671+0000 7f337bfff640 10 monclient: _finish_auth 0 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.671+0000 7f337bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.671+0000 7f337bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 5 monclient: authenticate success, global_id 4289 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 10 monclient: _renew_subs 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 1 -- 10.20.193.16:0/3104765082 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a4158450 con 0x7f33a415c080 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 10 client.4289.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:55:21.685 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 10 monclient: _renew_subs 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 1 -- 10.20.193.16:0/3104765082 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a4158a10 con 0x7f33a415c080 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 1 librados: init done 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a98a0640 1 -- 10.20.193.16:0/3104765082 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a415c460 con 0x7f33a415c080 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f337bfff640 1 -- 10.20.193.16:0/3104765082 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339400ad80 con 0x7f33a415c080 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f337bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f337bfff640 1 -- 10.20.193.16:0/3104765082 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394003370 con 0x7f33a415c080 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f337bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f337bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f33a17fa640 4 set_mon_vals no callback set 2026-02-17T05:55:21.686 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.672+0000 7f337bfff640 10 monclient: dump: 2026-02-17T05:55:21.687 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:21.687 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:21.687 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:21.687 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:21.687 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:21.687 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:21.687 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:21.687 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:21.687 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:21.687 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:21.779 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.779+0000 7f337bfff640 1 -- 10.20.193.16:0/3104765082 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f3394025630 con 0x7f33a415c080 2026-02-17T05:55:21.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.779+0000 7f337bfff640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f33740790e0 0x7f337407b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:21.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.780+0000 7f33a2ffd640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f33740790e0 0x7f337407b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:21.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.780+0000 7f33a2ffd640 10 monclient: get_auth_request con 0x7f33740790e0 auth_method 0 2026-02-17T05:55:21.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.780+0000 7f337bfff640 10 client.4289.objecter ms_handle_connect 0x7f33740790e0 2026-02-17T05:55:21.780 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.780+0000 7f33a2ffd640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f33740790e0 0x7f337407b5a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3398002410 tx=0x7f3398007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:21.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.787+0000 7f337bfff640 1 -- 10.20.193.16:0/3104765082 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f339405ea00 con 0x7f33a415c080 2026-02-17T05:55:21.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.787+0000 7f337bfff640 10 client.4289.objecter ms_dispatch2 0x7f33a4000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:55:21.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.787+0000 7f337bfff640 3 client.4289.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:55:21.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.787+0000 7f337bfff640 3 client.4289.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:55:21.787 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.787+0000 7f337bfff640 20 client.4289.objecter dump_active .. 0 homeless 2026-02-17T05:55:21.794 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.794+0000 7f337bfff640 1 -- 10.20.193.16:0/3104765082 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f33940629c0 con 0x7f33a415c080 2026-02-17T05:55:21.794 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.794+0000 7f337bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:21.794 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.794+0000 7f337bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:21.881 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.881+0000 7f33a98a0640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:21.882 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.881+0000 7f33a98a0640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:21.882 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.881+0000 7f33a98a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:21.882 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.881+0000 7f33a98a0640 1 -- 10.20.193.16:0/3104765082 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33a415ccf0 con 0x7f33a415c080 2026-02-17T05:55:21.894 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.893+0000 7f337bfff640 1 -- 10.20.193.16:0/3104765082 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f3394066cc0 con 0x7f33a415c080 2026-02-17T05:55:21.894 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.893+0000 7f337bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:21.894 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.893+0000 7f337bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:21.894 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:55:21.895 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:55:21.896 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 10 librados: watch_flush enter 2026-02-17T05:55:21.896 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 10 librados: watch_flush exit 2026-02-17T05:55:21.896 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 20 client.4289.objecter shutdown clearing up homeless session... 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 10 client.4289.objecter successfully canceled tick 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 -- 10.20.193.16:0/3104765082 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f33740790e0 msgr2=0x7f337407b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f33740790e0 0x7f337407b5a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3398002410 tx=0x7f3398007ef0 comp rx=0 tx=0).stop 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 10 monclient: shutdown 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 -- 10.20.193.16:0/3104765082 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a415c080 msgr2=0x7f33a40d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a415c080 0x7f33a40d1f90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f339400caa0 tx=0x7f339400cf70 comp rx=0 tx=0).stop 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 -- 10.20.193.16:0/3104765082 shutdown_connections 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f33740790e0 0x7f337407b5a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33a415c080 0x7f33a40d1f90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:21.897 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33a40cba00 0x7f33a41474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:21.898 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 --2- 10.20.193.16:0/3104765082 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f33a40c8ce0 0x7f33a4147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:21.898 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 -- 10.20.193.16:0/3104765082 >> 10.20.193.16:0/3104765082 conn(0x7f33a40740d0 msgr2=0x7f33a4152360 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:21.898 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 -- 10.20.193.16:0/3104765082 shutdown_connections 2026-02-17T05:55:21.898 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 -- 10.20.193.16:0/3104765082 wait complete. 2026-02-17T05:55:21.898 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:21.895+0000 7f33a98a0640 1 librados: shutdown 2026-02-17T05:55:21.904 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:55:29.517 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:55:29.585 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.584+0000 7fb35d185640 10 monclient: get_monmap_and_config 2026-02-17T05:55:29.585 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.584+0000 7fb35d185640 10 monclient: build_initial_monmap 2026-02-17T05:55:29.586 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.584+0000 7fb35d185640 10 monclient: monmap: 2026-02-17T05:55:29.586 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:29.586 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:29.586 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:55:29.585840+0000 2026-02-17T05:55:29.586 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:55:29.585840+0000 2026-02-17T05:55:29.586 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:29.586 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:29.586 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:55:29.586 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:55:29.586 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:55:29.587 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:55:29.587 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:55:29.587 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:55:29.587 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:29.587 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 1 Processor -- start 2026-02-17T05:55:29.587 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 1 -- start start 2026-02-17T05:55:29.587 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 10 monclient: init 2026-02-17T05:55:29.589 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:29.589 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 10 monclient: _add_conns ranks=[5,4,2,0,3,1] 2026-02-17T05:55:29.589 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fb3581561a0 0x7fb35815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:29.589 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 10 monclient(hunting): picked mon.noname-c con 0x7fb3581561a0 addr v2:10.20.193.148:3300/0 2026-02-17T05:55:29.589 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fb3580ccc80 0x7fb35815d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:29.589 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 10 monclient(hunting): picked mon.noname-b con 0x7fb3580ccc80 addr v2:10.20.193.128:3300/0 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb35815d970 addr v1:10.20.193.148:6789/0 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35815de00 con 0x7fb35815d970 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fb35815ce00 con 0x7fb3580ccc80 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb358069300 con 0x7fb3581561a0 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.585+0000 7fb35d185640 10 monclient(hunting): authenticate will time out at 982.154926s 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb356d76640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fb3581561a0 0x7fb35815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb356575640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fb3580ccc80 0x7fb35815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:29.590 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb357577640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fb35815d970 0x7fb3580cb070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:59378/0 (socket says 10.20.193.16:59378) 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb357577640 1 -- 10.20.193.16:0/2869493327 learned_addr learned my addr 10.20.193.16:0/2869493327 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb356d76640 10 monclient(hunting): get_auth_request con 0x7fb3581561a0 auth_method 0 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb356d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb356d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb356d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb356575640 10 monclient(hunting): get_auth_request con 0x7fb3580ccc80 auth_method 0 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb356575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb356575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.586+0000 7fb356575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.587+0000 7fb356d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.587+0000 7fb356d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.587+0000 7fb356d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:29.591 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.587+0000 7fb356575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.587+0000 7fb356575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.587+0000 7fb356575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.588+0000 7fb356575640 10 monclient(hunting): handle_auth_done global_id 4267 payload 274 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.588+0000 7fb356575640 1 -- 10.20.193.16:0/2869493327 >> v2:10.20.193.148:3300/0 conn(0x7fb3581561a0 msgr2=0x7fb35815c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.588+0000 7fb356575640 1 --2- 10.20.193.16:0/2869493327 >> v2:10.20.193.148:3300/0 conn(0x7fb3581561a0 0x7fb35815c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.588+0000 7fb356575640 1 -- 10.20.193.16:0/2869493327 >> v1:10.20.193.148:6789/0 conn(0x7fb35815d970 legacy=0x7fb3580cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.588+0000 7fb356575640 10 monclient: _finish_hunting 0 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.588+0000 7fb356575640 1 monclient: found mon.noname-b 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.588+0000 7fb356575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.588+0000 7fb356575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:55:29.592 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.588+0000 7fb356575640 1 -- 10.20.193.16:0/2869493327 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb358156580 con 0x7fb3580ccc80 2026-02-17T05:55:29.598 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.589+0000 7fb356575640 1 --2- 10.20.193.16:0/2869493327 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb3580ccc80 0x7fb35815d430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb358071fa0 tx=0x7fb34002ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3e46a75428460b7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:29.598 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 1 -- 10.20.193.16:0/2869493327 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34003c070 con 0x7fb3580ccc80 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 10 monclient: dump: 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:29.599 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 10 monclient: _finish_auth 0 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 10 monclient: _check_auth_tickets 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 1 -- 10.20.193.16:0/2869493327 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34002faf0 con 0x7fb3580ccc80 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb35d185640 5 monclient: authenticate success, global_id 4267 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 1 -- 10.20.193.16:0/2869493327 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340037440 con 0x7fb3580ccc80 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb355d74640 10 monclient: dump: 2026-02-17T05:55:29.600 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb35d185640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:29.601 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.591+0000 7fb35d185640 4 set_mon_vals no callback set 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.592+0000 7fb35d185640 10 monclient: shutdown 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.592+0000 7fb35d185640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.592+0000 7fb35d185640 1 -- 10.20.193.16:0/2869493327 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb3580ccc80 msgr2=0x7fb35815d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.592+0000 7fb35d185640 1 --2- 10.20.193.16:0/2869493327 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb3580ccc80 0x7fb35815d430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb358071fa0 tx=0x7fb34002ede0 comp rx=0 tx=0).stop 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.592+0000 7fb35d185640 1 -- 10.20.193.16:0/2869493327 shutdown_connections 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.592+0000 7fb35d185640 1 --2- 10.20.193.16:0/2869493327 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb3580ccc80 0x7fb35815d430 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.592+0000 7fb35d185640 1 --2- 10.20.193.16:0/2869493327 >> v2:10.20.193.148:3300/0 conn(0x7fb3581561a0 0x7fb35815c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.592+0000 7fb35d185640 1 -- 10.20.193.16:0/2869493327 >> 10.20.193.16:0/2869493327 conn(0x7fb3580cc160 msgr2=0x7fb3580cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.592+0000 7fb35d185640 1 -- 10.20.193.16:0/2869493327 shutdown_connections 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.592+0000 7fb35d185640 1 -- 10.20.193.16:0/2869493327 wait complete. 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient: build_initial_monmap 2026-02-17T05:55:29.602 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient: monmap: 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 librados: starting msgr at 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 librados: starting objecter 2026-02-17T05:55:29.603 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 Processor -- start 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 -- start start 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 librados: setting wanted keys 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 librados: calling monclient init 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient: init 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb3580ccc80 0x7fb3581f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient(hunting): picked mon.noname-b con 0x7fb3580ccc80 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb3581561a0 0x7fb3580ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient(hunting): picked mon.noname-a con 0x7fb3581561a0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb35815d970 0x7fb3581447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:29.604 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient(hunting): picked mon.noname-c con 0x7fb35815d970 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb35806cef0 con 0x7fb3581561a0 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fb35806cc10 con 0x7fb3580ccc80 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb35806d1d0 con 0x7fb35815d970 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.593+0000 7fb35d185640 10 monclient(hunting): authenticate will time out at 982.162836s 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb3581561a0 0x7fb3580ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356d76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb3580ccc80 0x7fb3581f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:29.605 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb3581561a0 0x7fb3580ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44924/0 (socket says 10.20.193.16:44924) 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356575640 1 -- 10.20.193.16:0/1277958047 learned_addr learned my addr 10.20.193.16:0/1277958047 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356575640 10 monclient(hunting): get_auth_request con 0x7fb3581561a0 auth_method 0 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356d76640 10 monclient(hunting): get_auth_request con 0x7fb3580ccc80 auth_method 0 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb357577640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb35815d970 0x7fb3581447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:29.606 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb357577640 10 monclient(hunting): get_auth_request con 0x7fb35815d970 auth_method 0 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb357577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb357577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb357577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.594+0000 7fb356d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.595+0000 7fb357577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.595+0000 7fb357577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.595+0000 7fb357577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:29.607 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.596+0000 7fb357577640 10 monclient(hunting): handle_auth_done global_id 4298 payload 824 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.596+0000 7fb357577640 1 -- 10.20.193.16:0/1277958047 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb3580ccc80 msgr2=0x7fb3581f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.596+0000 7fb357577640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb3580ccc80 0x7fb3581f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.596+0000 7fb357577640 1 -- 10.20.193.16:0/1277958047 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb3581561a0 msgr2=0x7fb3580ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.596+0000 7fb357577640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb3581561a0 0x7fb3580ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.596+0000 7fb357577640 10 monclient: _finish_hunting 0 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.596+0000 7fb357577640 1 monclient: found mon.noname-c 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.596+0000 7fb357577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.596+0000 7fb357577640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.596+0000 7fb357577640 1 -- 10.20.193.16:0/1277958047 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3581f5290 con 0x7fb35815d970 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.597+0000 7fb357577640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb35815d970 0x7fb3581447b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb34800cb10 tx=0x7fb34800a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.598+0000 7fb3377fe640 10 client.?.objecter ms_handle_connect 0x7fb35815d970 2026-02-17T05:55:29.608 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.598+0000 7fb3377fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:55:29.610 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 1 -- 10.20.193.16:0/1277958047 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348017070 con 0x7fb35815d970 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 10 monclient: dump: 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:29.611 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 10 monclient: _finish_auth 0 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 10 monclient: _check_auth_tickets 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 1 -- 10.20.193.16:0/1277958047 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34800acb0 con 0x7fb35815d970 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 1 -- 10.20.193.16:0/1277958047 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348012620 con 0x7fb35815d970 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb35d185640 5 monclient: authenticate success, global_id 4298 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:29.612 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb355573640 4 set_mon_vals no callback set 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb3377fe640 10 monclient: dump: 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:29.613 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb35d185640 10 monclient: _renew_subs 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb35d185640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb35d185640 1 -- 10.20.193.16:0/1277958047 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb358150df0 con 0x7fb35815d970 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb35d185640 10 client.4298.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb35d185640 10 monclient: _renew_subs 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb35d185640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb35d185640 1 -- 10.20.193.16:0/1277958047 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3581513b0 con 0x7fb35815d970 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.600+0000 7fb35d185640 1 librados: init done 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.601+0000 7fb35d185640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.601+0000 7fb35d185640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.601+0000 7fb35d185640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:29.614 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.601+0000 7fb35d185640 1 -- 10.20.193.16:0/1277958047 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35806d4b0 con 0x7fb35815d970 2026-02-17T05:55:29.712 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.712+0000 7fb3377fe640 1 -- 10.20.193.16:0/1277958047 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7fb34801b3b0 con 0x7fb35815d970 2026-02-17T05:55:29.713 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.712+0000 7fb3377fe640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb3240790b0 0x7fb32407b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:29.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.713+0000 7fb356d76640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb3240790b0 0x7fb32407b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:29.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.713+0000 7fb356d76640 10 monclient: get_auth_request con 0x7fb3240790b0 auth_method 0 2026-02-17T05:55:29.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.713+0000 7fb3377fe640 10 client.4298.objecter ms_handle_connect 0x7fb3240790b0 2026-02-17T05:55:29.714 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.713+0000 7fb356d76640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb3240790b0 0x7fb32407b570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009c40 tx=0x7fb34c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:29.721 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.720+0000 7fb3377fe640 1 -- 10.20.193.16:0/1277958047 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7fb34805de30 con 0x7fb35815d970 2026-02-17T05:55:29.721 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.720+0000 7fb3377fe640 10 client.4298.objecter ms_dispatch2 0x7fb358000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:55:29.721 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.720+0000 7fb3377fe640 3 client.4298.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:55:29.721 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.720+0000 7fb3377fe640 3 client.4298.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:55:29.721 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.720+0000 7fb3377fe640 20 client.4298.objecter dump_active .. 0 homeless 2026-02-17T05:55:29.721 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.721+0000 7fb3377fe640 1 -- 10.20.193.16:0/1277958047 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb348021020 con 0x7fb35815d970 2026-02-17T05:55:29.721 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.721+0000 7fb3377fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:29.721 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.721+0000 7fb3377fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:29.811 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.810+0000 7fb35d185640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:29.811 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.810+0000 7fb35d185640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:29.811 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.810+0000 7fb35d185640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:29.811 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.810+0000 7fb35d185640 1 -- 10.20.193.16:0/1277958047 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb35815ea80 con 0x7fb35815d970 2026-02-17T05:55:29.821 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.821+0000 7fb3377fe640 1 -- 10.20.193.16:0/1277958047 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7fb34807b260 con 0x7fb35815d970 2026-02-17T05:55:29.821 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.821+0000 7fb3377fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:29.821 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.821+0000 7fb3377fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:29.821 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:55:29.821 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:55:29.823 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 10 librados: watch_flush enter 2026-02-17T05:55:29.823 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 10 librados: watch_flush exit 2026-02-17T05:55:29.823 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 20 client.4298.objecter shutdown clearing up homeless session... 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 10 client.4298.objecter successfully canceled tick 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 -- 10.20.193.16:0/1277958047 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb3240790b0 msgr2=0x7fb32407b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb3240790b0 0x7fb32407b570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009c40 tx=0x7fb34c009290 comp rx=0 tx=0).stop 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 10 monclient: shutdown 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 -- 10.20.193.16:0/1277958047 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb35815d970 msgr2=0x7fb3581447b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb35815d970 0x7fb3581447b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb34800cb10 tx=0x7fb34800a060 comp rx=0 tx=0).stop 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 -- 10.20.193.16:0/1277958047 shutdown_connections 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb3240790b0 0x7fb32407b570 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb35815d970 0x7fb3581447b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb3581561a0 0x7fb3580ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 --2- 10.20.193.16:0/1277958047 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb3580ccc80 0x7fb3581f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 -- 10.20.193.16:0/1277958047 >> 10.20.193.16:0/1277958047 conn(0x7fb3580cc160 msgr2=0x7fb358154300 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 -- 10.20.193.16:0/1277958047 shutdown_connections 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 -- 10.20.193.16:0/1277958047 wait complete. 2026-02-17T05:55:29.824 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:29.823+0000 7fb35d185640 1 librados: shutdown 2026-02-17T05:55:29.831 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:55:30.767 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:34.894 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.073 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:35.201 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.234 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:35.362 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.334 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:35.462 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.335 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:35.453 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.468 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:35.586 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.724 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:35.856 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.753 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:35.890 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.796 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:35.915 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.898 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:36.038 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.931 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:36.068 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.936 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:36.054 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:31.998 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:36.230 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:32.123 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:36.353 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:32.183 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:36.315 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:32.240 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:36.381 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.428 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:37.556 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.458 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:37.585 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.460 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:37.588 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.756 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:37.883 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.768 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:37.887 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.796 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:37.925 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.799 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:37.927 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.812 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:37.930 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.815 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:37.933 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.816 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:37.944 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:33.914 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:38.042 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.103 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:38.230 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.106 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:38.233 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.149 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:38.267 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.190 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:38.308 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.191 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:38.310 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.193 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:38.311 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.202 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:38.329 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.204 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:38.332 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.230 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:38.348 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.233 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:38.352 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.398 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:38.530 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.428 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:38.566 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.440 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:38.580 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.465 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:38.605 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.468 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:38.608 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.487 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:38.605 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.523 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:38.641 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.526 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:38.644 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.612 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:38.749 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.634 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:38.771 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.636 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:38.774 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.662 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:38.794 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.664 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:38.796 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.675 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:38.908 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.703 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:38.840 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.705 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:38.843 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.706 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:38.939 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.709 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:38.942 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.819 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:38.952 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.838 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:39.069 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.858 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:38.990 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.861 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:38.993 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.900 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:39.041 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.925 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:39.066 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:34.928 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:39.069 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:35.037 INFO:tasks.ceph.osd.10.trial148.stderr:==00:00:00:39.165 44709== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:35.105 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:39.336 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:35.108 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:39.339 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:35.497 INFO:tasks.ceph.osd.2.trial148.stderr:==00:00:00:39.625 44708== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:35.623 INFO:tasks.ceph.osd.15.trial164.stderr:==00:00:00:39.741 44921== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:35.915 INFO:tasks.ceph.osd.14.trial148.stderr:==00:00:00:40.043 44710== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:35.934 INFO:tasks.ceph.osd.11.trial164.stderr:==00:00:00:40.053 44920== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:35.935 INFO:tasks.ceph.osd.7.trial164.stderr:==00:00:00:40.053 44919== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:35.939 INFO:tasks.ceph.osd.6.trial148.stderr:==00:00:00:40.067 44707== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:36.295 INFO:tasks.ceph.osd.12.trial016.stderr:==00:00:00:40.435 44882== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:36.298 INFO:tasks.ceph.osd.5.trial128.stderr:==00:00:00:40.435 44604== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:36.310 INFO:tasks.ceph.osd.3.trial164.stderr:==00:00:00:40.429 44918== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:36.402 INFO:tasks.ceph.osd.9.trial128.stderr:==00:00:00:40.534 44605== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:36.402 INFO:tasks.ceph.osd.1.trial128.stderr:==00:00:00:40.539 44603== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:36.432 INFO:tasks.ceph.osd.0.trial016.stderr:==00:00:00:40.665 44804== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:36.693 INFO:tasks.ceph.osd.13.trial128.stderr:==00:00:00:40.824 44606== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:36.746 INFO:tasks.ceph.osd.8.trial016.stderr:==00:00:00:40.886 44881== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:36.918 INFO:tasks.ceph.osd.4.trial016.stderr:==00:00:00:41.148 44805== Warning: unimplemented fcntl command: 1038 2026-02-17T05:55:37.443 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.503+0000 7f9c13161640 10 monclient: get_monmap_and_config 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.503+0000 7f9c13161640 10 monclient: build_initial_monmap 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.503+0000 7f9c13161640 10 monclient: monmap: 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:55:37.504147+0000 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:55:37.504147+0000 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:55:37.503 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.503+0000 7f9c13161640 1 Processor -- start 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.503+0000 7f9c13161640 1 -- start start 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.503+0000 7f9c13161640 10 monclient: init 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 10 monclient: _add_conns ranks=[2,4,1,5,0,3] 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9c0c15c3b0 addr v1:10.20.193.148:6789/0 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f9c0c15c790 0x7f9c0c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 10 monclient(hunting): picked mon.noname-b con 0x7f9c0c15c790 addr v2:10.20.193.128:3300/0 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f9c0c0c6b30 addr v1:10.20.193.128:6789/0 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c0c0c7270 con 0x7f9c0c0c6b30 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c0c0c7550 con 0x7f9c0c15c3b0 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f9c0c0c7830 con 0x7f9c0c15c790 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:37.504 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c13161640 10 monclient(hunting): authenticate will time out at 990.073228s 2026-02-17T05:55:37.505 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c10ed6640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f9c0c15c3b0 0x7f9c0c0c7e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:39806/0 (socket says 10.20.193.16:39806) 2026-02-17T05:55:37.505 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c10ed6640 1 -- 10.20.193.16:0/222366566 learned_addr learned my addr 10.20.193.16:0/222366566 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:55:37.505 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.504+0000 7f9c03fff640 1 --2- 10.20.193.16:0/222366566 >> v2:10.20.193.128:3300/0 conn(0x7f9c0c15c790 0x7f9c0c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.505 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.505+0000 7f9c03fff640 10 monclient(hunting): get_auth_request con 0x7f9c0c15c790 auth_method 0 2026-02-17T05:55:37.505 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.505+0000 7f9c03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.505 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.505+0000 7f9c03fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.505 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.505+0000 7f9c03fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.505 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.505+0000 7f9c03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.505 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.505+0000 7f9c03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.506 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.505+0000 7f9c03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.506 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.506+0000 7f9c037fe640 1 -- 10.20.193.16:0/222366566 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2475504882 0 0) 0x7f9c0c0c7550 con 0x7f9c0c15c3b0 2026-02-17T05:55:37.506 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.506+0000 7f9c037fe640 10 monclient(hunting): _negotiate 2026-02-17T05:55:37.506 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.506+0000 7f9c037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.506 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.506+0000 7f9c037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.506 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.506+0000 7f9c037fe640 10 monclient(hunting): my global_id is 4301 2026-02-17T05:55:37.506 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.506+0000 7f9c037fe640 1 -- 10.20.193.16:0/222366566 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bf80058b0 con 0x7f9c0c15c3b0 2026-02-17T05:55:37.508 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.508+0000 7f9c03fff640 10 monclient(hunting): handle_auth_done global_id 4273 payload 274 2026-02-17T05:55:37.508 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.508+0000 7f9c03fff640 1 -- 10.20.193.16:0/222366566 >> v1:10.20.193.148:6789/0 conn(0x7f9c0c15c3b0 legacy=0x7f9c0c0c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.508 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.508+0000 7f9c03fff640 1 -- 10.20.193.16:0/222366566 >> v1:10.20.193.128:6789/0 conn(0x7f9c0c0c6b30 legacy=0x7f9c0c06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.508 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.508+0000 7f9c03fff640 10 monclient: _finish_hunting 0 2026-02-17T05:55:37.508 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.508+0000 7f9c03fff640 1 monclient: found mon.noname-b 2026-02-17T05:55:37.508 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.508+0000 7f9c03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:37.508 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.508+0000 7f9c03fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:55:37.508 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.508+0000 7f9c03fff640 1 -- 10.20.193.16:0/222366566 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c06d4d0 con 0x7f9c0c15c790 2026-02-17T05:55:37.509 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.509+0000 7f9c03fff640 1 --2- 10.20.193.16:0/222366566 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 0x7f9c0c070c60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40098a0 tx=0x7f9bf402ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa74315ebadc5cd9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 1 -- 10.20.193.16:0/222366566 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf403c070 con 0x7f9c0c15c790 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 10 monclient: dump: 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 10 monclient: _finish_auth 0 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 10 monclient: _check_auth_tickets 2026-02-17T05:55:37.512 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 1 -- 10.20.193.16:0/222366566 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf402fb70 con 0x7f9c0c15c790 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 1 -- 10.20.193.16:0/222366566 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4037440 con 0x7f9c0c15c790 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c13161640 5 monclient: authenticate success, global_id 4273 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c13161640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c13161640 4 set_mon_vals no callback set 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.511+0000 7f9c037fe640 10 monclient: dump: 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:37.513 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:37.514 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.514 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:37.514 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:37.514 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:37.514 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:37.514 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:37.520 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 10 monclient: shutdown 2026-02-17T05:55:37.520 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:37.520 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 1 -- 10.20.193.16:0/222366566 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 msgr2=0x7f9c0c070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.520 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 1 --2- 10.20.193.16:0/222366566 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 0x7f9c0c070c60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40098a0 tx=0x7f9bf402ee60 comp rx=0 tx=0).stop 2026-02-17T05:55:37.520 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 1 -- 10.20.193.16:0/222366566 shutdown_connections 2026-02-17T05:55:37.520 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 1 --2- 10.20.193.16:0/222366566 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 0x7f9c0c070c60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.520 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 1 -- 10.20.193.16:0/222366566 >> 10.20.193.16:0/222366566 conn(0x7f9c0c0c9f40 msgr2=0x7f9c0c0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:37.520 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 1 -- 10.20.193.16:0/222366566 shutdown_connections 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 1 -- 10.20.193.16:0/222366566 wait complete. 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 10 monclient: build_initial_monmap 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 10 monclient: monmap: 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 1 librados: starting msgr at 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.512+0000 7f9c13161640 1 librados: starting objecter 2026-02-17T05:55:37.521 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 1 Processor -- start 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 1 -- start start 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 1 librados: setting wanted keys 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 1 librados: calling monclient init 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient: init 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c0c0c6b30 0x7f9c0c0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient(hunting): picked mon.noname-a con 0x7f9c0c0c6b30 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c0c15c3b0 0x7f9c0c1f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient(hunting): picked mon.noname-c con 0x7f9c0c15c3b0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 0x7f9c0c1f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient(hunting): picked mon.noname-b con 0x7f9c0c15c790 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c06e6c0 con 0x7f9c0c0c6b30 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c0c6f80 con 0x7f9c0c15c790 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c06e840 con 0x7f9c0c15c3b0 2026-02-17T05:55:37.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c13161640 10 monclient(hunting): authenticate will time out at 990.082645s 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.513+0000 7f9c116d7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 0x7f9c0c1f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c116d7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 0x7f9c0c1f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50998/0 (socket says 10.20.193.16:50998) 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c116d7640 1 -- 10.20.193.16:0/508142905 learned_addr learned my addr 10.20.193.16:0/508142905 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c116d7640 10 monclient(hunting): get_auth_request con 0x7f9c0c15c790 auth_method 0 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c116d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c116d7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c116d7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c10ed6640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c0c0c6b30 0x7f9c0c0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c03fff640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c0c15c3b0 0x7f9c0c1f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c116d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c116d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c116d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c03fff640 10 monclient(hunting): get_auth_request con 0x7f9c0c15c3b0 auth_method 0 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c03fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c03fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c10ed6640 10 monclient(hunting): get_auth_request con 0x7f9c0c0c6b30 auth_method 0 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c10ed6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.523 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c10ed6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c10ed6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c10ed6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c10ed6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.514+0000 7f9c10ed6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.515+0000 7f9c116d7640 10 monclient(hunting): handle_auth_done global_id 4279 payload 824 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.515+0000 7f9c116d7640 1 -- 10.20.193.16:0/508142905 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c0c15c3b0 msgr2=0x7f9c0c1f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.515+0000 7f9c116d7640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c0c15c3b0 0x7f9c0c1f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.515+0000 7f9c116d7640 1 -- 10.20.193.16:0/508142905 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c0c0c6b30 msgr2=0x7f9c0c0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.515+0000 7f9c116d7640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c0c0c6b30 0x7f9c0c0cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.515+0000 7f9c116d7640 10 monclient: _finish_hunting 0 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.515+0000 7f9c116d7640 1 monclient: found mon.noname-b 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.515+0000 7f9c116d7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.515+0000 7f9c116d7640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.515+0000 7f9c116d7640 1 -- 10.20.193.16:0/508142905 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c1f6e90 con 0x7f9c0c15c790 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.516+0000 7f9c116d7640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 0x7f9c0c1f62e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800cba0 tx=0x7f9c0800a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.516+0000 7f9c017fa640 10 client.?.objecter ms_handle_connect 0x7f9c0c15c790 2026-02-17T05:55:37.524 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.516+0000 7f9c017fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 1 -- 10.20.193.16:0/508142905 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800ad50 con 0x7f9c0c15c790 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 10 monclient: dump: 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 10 monclient: _finish_auth 0 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 10 monclient: _check_auth_tickets 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 5 monclient: authenticate success, global_id 4279 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 10 monclient: _renew_subs 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:37.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 1 -- 10.20.193.16:0/508142905 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c1f7170 con 0x7f9c0c15c790 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 10 client.4279.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 10 monclient: _renew_subs 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 1 -- 10.20.193.16:0/508142905 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c1f7730 con 0x7f9c0c15c790 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 1 librados: init done 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 1 -- 10.20.193.16:0/508142905 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c08003a70 con 0x7f9c0c15c790 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c02ffd640 4 set_mon_vals no callback set 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 1 -- 10.20.193.16:0/508142905 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080050d0 con 0x7f9c0c15c790 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c017fa640 10 monclient: dump: 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:37.526 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:37.527 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:37.527 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:37.527 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:37.527 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.518+0000 7f9c13161640 1 -- 10.20.193.16:0/508142905 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c070c60 con 0x7f9c0c15c790 2026-02-17T05:55:37.630 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.630+0000 7f9c017fa640 1 -- 10.20.193.16:0/508142905 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f9c08005270 con 0x7f9c0c15c790 2026-02-17T05:55:37.631 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.631+0000 7f9c017fa640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9bdc0790b0 0x7f9bdc07b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.631 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.631+0000 7f9c10ed6640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9bdc0790b0 0x7f9bdc07b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.631 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.631+0000 7f9c10ed6640 10 monclient: get_auth_request con 0x7f9bdc0790b0 auth_method 0 2026-02-17T05:55:37.631 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.631+0000 7f9c10ed6640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9bdc0790b0 0x7f9bdc07b570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc005980 tx=0x7f9bfc007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.631 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.631+0000 7f9c017fa640 10 client.4279.objecter ms_handle_connect 0x7f9bdc0790b0 2026-02-17T05:55:37.639 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.639+0000 7f9c017fa640 1 -- 10.20.193.16:0/508142905 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f9c080982c0 con 0x7f9c0c15c790 2026-02-17T05:55:37.639 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.639+0000 7f9c017fa640 10 client.4279.objecter ms_dispatch2 0x7f9c0c000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:55:37.639 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.639+0000 7f9c017fa640 3 client.4279.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:55:37.639 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.639+0000 7f9c017fa640 3 client.4279.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:55:37.639 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.639+0000 7f9c017fa640 20 client.4279.objecter dump_active .. 0 homeless 2026-02-17T05:55:37.639 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.639+0000 7f9c017fa640 1 -- 10.20.193.16:0/508142905 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9c080628c0 con 0x7f9c0c15c790 2026-02-17T05:55:37.639 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.639+0000 7f9c017fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:37.639 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.639+0000 7f9c017fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:37.725 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.725+0000 7f9c13161640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:37.726 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.725+0000 7f9c13161640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:37.726 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.725+0000 7f9c13161640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:37.726 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.725+0000 7f9c13161640 1 -- 10.20.193.16:0/508142905 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c0c1516c0 con 0x7f9c0c15c790 2026-02-17T05:55:37.736 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.736+0000 7f9c017fa640 1 -- 10.20.193.16:0/508142905 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f9c08066bc0 con 0x7f9c0c15c790 2026-02-17T05:55:37.736 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.736+0000 7f9c017fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:37.736 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.736+0000 7f9c017fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:37.736 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:55:37.737 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 10 librados: watch_flush enter 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 10 librados: watch_flush exit 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 20 client.4279.objecter shutdown clearing up homeless session... 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 10 client.4279.objecter successfully canceled tick 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 -- 10.20.193.16:0/508142905 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9bdc0790b0 msgr2=0x7f9bdc07b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9bdc0790b0 0x7f9bdc07b570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc005980 tx=0x7f9bfc007dd0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 10 monclient: shutdown 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 -- 10.20.193.16:0/508142905 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 msgr2=0x7f9c0c1f62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 0x7f9c0c1f62e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800cba0 tx=0x7f9c0800a430 comp rx=0 tx=0).stop 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 -- 10.20.193.16:0/508142905 shutdown_connections 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9bdc0790b0 0x7f9bdc07b570 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c0c15c790 0x7f9c0c1f62e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c0c15c3b0 0x7f9c0c1f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 --2- 10.20.193.16:0/508142905 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c0c0c6b30 0x7f9c0c0cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 -- 10.20.193.16:0/508142905 >> 10.20.193.16:0/508142905 conn(0x7f9c0c0c9f40 msgr2=0x7f9c0c14a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 -- 10.20.193.16:0/508142905 shutdown_connections 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 -- 10.20.193.16:0/508142905 wait complete. 2026-02-17T05:55:37.739 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:37.738+0000 7f9c13161640 1 librados: shutdown 2026-02-17T05:55:37.746 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:55:43.157 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:01:21.984 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:55:43.214 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:01:22.041 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:55:43.233 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:01:22.059 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:55:43.234 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:01:22.023 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:55:43.263 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:01:22.053 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:55:43.265 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:01:22.091 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:55:43.342 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:01:22.169 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:55:43.382 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:01:22.171 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:55:43.400 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:01:22.227 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:55:45.358 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:55:45.424 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.422+0000 7ffa55b0a640 10 monclient: get_monmap_and_config 2026-02-17T05:55:45.424 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.422+0000 7ffa55b0a640 10 monclient: build_initial_monmap 2026-02-17T05:55:45.424 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.422+0000 7ffa55b0a640 10 monclient: monmap: 2026-02-17T05:55:45.424 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:45.424 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:55:45.423144+0000 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:55:45.423144+0000 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:55:45.425 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.422+0000 7ffa55b0a640 1 Processor -- start 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.422+0000 7ffa55b0a640 1 -- start start 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.422+0000 7ffa55b0a640 10 monclient: init 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 10 monclient: _add_conns ranks=[5,1,2,3,4,0] 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7ffa5015c170 0x7ffa5015c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 10 monclient(hunting): picked mon.noname-c con 0x7ffa5015c170 addr v2:10.20.193.148:3300/0 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ffa500c9500 addr v1:10.20.193.128:6789/0 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ffa501562d0 addr v1:10.20.193.148:6789/0 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa500c8180 con 0x7ffa500c9500 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa500c8460 con 0x7ffa501562d0 2026-02-17T05:55:45.426 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:45.427 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7ffa500c9070 con 0x7ffa5015c170 2026-02-17T05:55:45.427 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:45.427 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa55b0a640 10 monclient(hunting): authenticate will time out at 997.992117s 2026-02-17T05:55:45.427 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa4effd640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7ffa500c9500 0x7ffa50155bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:49574/0 (socket says 10.20.193.16:49574) 2026-02-17T05:55:45.427 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa4effd640 1 -- 10.20.193.16:0/143109587 learned_addr learned my addr 10.20.193.16:0/143109587 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:55:45.427 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.423+0000 7ffa4f7fe640 1 --2- 10.20.193.16:0/143109587 >> v2:10.20.193.148:3300/0 conn(0x7ffa5015c170 0x7ffa5015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:45.427 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.424+0000 7ffa4f7fe640 10 monclient(hunting): get_auth_request con 0x7ffa5015c170 auth_method 0 2026-02-17T05:55:45.427 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.424+0000 7ffa4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:45.427 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.424+0000 7ffa4f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:45.427 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.424+0000 7ffa4f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:45.429 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.424+0000 7ffa4f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:45.429 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.424+0000 7ffa4f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:45.429 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.424+0000 7ffa4f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:45.429 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.425+0000 7ffa4e7fc640 1 -- 10.20.193.16:0/143109587 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 813844493 0 0) 0x7ffa500c8180 con 0x7ffa500c9500 2026-02-17T05:55:45.429 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.425+0000 7ffa4e7fc640 10 monclient(hunting): _negotiate 2026-02-17T05:55:45.429 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.425+0000 7ffa4e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:45.429 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.425+0000 7ffa4e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.425+0000 7ffa4e7fc640 10 monclient(hunting): my global_id is 4282 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.425+0000 7ffa4e7fc640 1 -- 10.20.193.16:0/143109587 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa34003620 con 0x7ffa500c9500 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.426+0000 7ffa4f7fe640 10 monclient(hunting): handle_auth_done global_id 4307 payload 274 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.426+0000 7ffa4f7fe640 1 -- 10.20.193.16:0/143109587 >> v1:10.20.193.148:6789/0 conn(0x7ffa501562d0 legacy=0x7ffa5006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.426+0000 7ffa4f7fe640 1 -- 10.20.193.16:0/143109587 >> v1:10.20.193.128:6789/0 conn(0x7ffa500c9500 legacy=0x7ffa50155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.426+0000 7ffa4f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.426+0000 7ffa4f7fe640 1 monclient: found mon.noname-c 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.426+0000 7ffa4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.426+0000 7ffa4f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.426+0000 7ffa4f7fe640 1 -- 10.20.193.16:0/143109587 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa500c8740 con 0x7ffa5015c170 2026-02-17T05:55:45.430 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.428+0000 7ffa4f7fe640 1 --2- 10.20.193.16:0/143109587 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa5015c170 0x7ffa5015c550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0098a0 tx=0x7ffa3c02edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb36bc992058e4b1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:45.436 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.429+0000 7ffa4e7fc640 1 -- 10.20.193.16:0/143109587 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c03c070 con 0x7ffa5015c170 2026-02-17T05:55:45.436 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.429+0000 7ffa4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:45.436 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.429+0000 7ffa4e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:45.436 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.429+0000 7ffa4e7fc640 10 monclient: dump: 2026-02-17T05:55:45.436 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:45.436 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.429+0000 7ffa4e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.429+0000 7ffa4e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.429+0000 7ffa4e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:45.437 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.429+0000 7ffa55b0a640 5 monclient: authenticate success, global_id 4307 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.429+0000 7ffa55b0a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa4e7fc640 1 -- 10.20.193.16:0/143109587 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3c02fae0 con 0x7ffa5015c170 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa4e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa4e7fc640 1 -- 10.20.193.16:0/143109587 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c037440 con 0x7ffa5015c170 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 4 set_mon_vals no callback set 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa4e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa4e7fc640 10 monclient: dump: 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:45.438 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 10 monclient: shutdown 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 1 -- 10.20.193.16:0/143109587 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa5015c170 msgr2=0x7ffa5015c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:45.439 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 1 --2- 10.20.193.16:0/143109587 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa5015c170 0x7ffa5015c550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0098a0 tx=0x7ffa3c02edd0 comp rx=0 tx=0).stop 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 1 -- 10.20.193.16:0/143109587 shutdown_connections 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 1 --2- 10.20.193.16:0/143109587 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa5015c170 0x7ffa5015c550 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 1 -- 10.20.193.16:0/143109587 >> 10.20.193.16:0/143109587 conn(0x7ffa500c9f40 msgr2=0x7ffa500ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 1 -- 10.20.193.16:0/143109587 shutdown_connections 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.430+0000 7ffa55b0a640 1 -- 10.20.193.16:0/143109587 wait complete. 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient: build_initial_monmap 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient: monmap: 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:55:45.440 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 librados: starting msgr at 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 librados: starting objecter 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 Processor -- start 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 -- start start 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 librados: setting wanted keys 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 librados: calling monclient init 2026-02-17T05:55:45.441 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient: init 2026-02-17T05:55:45.442 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:45.442 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:55:45.442 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa500c9500 0x7ffa5006e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:45.442 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient(hunting): picked mon.noname-c con 0x7ffa500c9500 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:55:45.442 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffa501562d0 0x7ffa501f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:45.442 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient(hunting): picked mon.noname-a con 0x7ffa501562d0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:55:45.442 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffa5015c170 0x7ffa501f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:45.442 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient(hunting): picked mon.noname-b con 0x7ffa5015c170 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa500c7e40 con 0x7ffa501562d0 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7ffa500c8d00 con 0x7ffa5015c170 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50072660 con 0x7ffa500c9500 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.431+0000 7ffa55b0a640 10 monclient(hunting): authenticate will time out at 998.000948s 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4ffff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffa5015c170 0x7ffa501f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4f7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa500c9500 0x7ffa5006e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:45.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4effd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffa501562d0 0x7ffa501f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4ffff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffa5015c170 0x7ffa501f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:33832/0 (socket says 10.20.193.16:33832) 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4ffff640 1 -- 10.20.193.16:0/82205504 learned_addr learned my addr 10.20.193.16:0/82205504 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4ffff640 10 monclient(hunting): get_auth_request con 0x7ffa5015c170 auth_method 0 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4f7fe640 10 monclient(hunting): get_auth_request con 0x7ffa500c9500 auth_method 0 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4effd640 10 monclient(hunting): get_auth_request con 0x7ffa501562d0 auth_method 0 2026-02-17T05:55:45.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.432+0000 7ffa4effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.433+0000 7ffa4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.433+0000 7ffa4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.433+0000 7ffa4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.433+0000 7ffa4f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.433+0000 7ffa4f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.433+0000 7ffa4f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.433+0000 7ffa4effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.433+0000 7ffa4effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.433+0000 7ffa4effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:45.445 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.434+0000 7ffa4f7fe640 10 monclient(hunting): handle_auth_done global_id 4310 payload 824 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.434+0000 7ffa4f7fe640 1 -- 10.20.193.16:0/82205504 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffa5015c170 msgr2=0x7ffa501f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.434+0000 7ffa4f7fe640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffa5015c170 0x7ffa501f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.434+0000 7ffa4f7fe640 1 -- 10.20.193.16:0/82205504 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffa501562d0 msgr2=0x7ffa501f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.434+0000 7ffa4f7fe640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffa501562d0 0x7ffa501f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.434+0000 7ffa4f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.434+0000 7ffa4f7fe640 1 monclient: found mon.noname-c 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.434+0000 7ffa4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.434+0000 7ffa4f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.434+0000 7ffa4f7fe640 1 -- 10.20.193.16:0/82205504 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5015f960 con 0x7ffa500c9500 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.435+0000 7ffa4f7fe640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa500c9500 0x7ffa5006e580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c036000 tx=0x7ffa3c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.435+0000 7ffa2bfff640 10 client.?.objecter ms_handle_connect 0x7ffa500c9500 2026-02-17T05:55:45.446 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.435+0000 7ffa2bfff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:55:45.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 1 -- 10.20.193.16:0/82205504 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c046020 con 0x7ffa500c9500 2026-02-17T05:55:45.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:45.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:45.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 10 monclient: dump: 2026-02-17T05:55:45.448 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 10 monclient: _finish_auth 0 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:55:45.449 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 1 -- 10.20.193.16:0/82205504 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3c0039d0 con 0x7ffa500c9500 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 1 -- 10.20.193.16:0/82205504 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c03c040 con 0x7ffa500c9500 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa2bfff640 10 monclient: dump: 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:45.450 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa4dffb640 4 set_mon_vals no callback set 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa55b0a640 5 monclient: authenticate success, global_id 4310 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa55b0a640 10 monclient: _renew_subs 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa55b0a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa55b0a640 1 -- 10.20.193.16:0/82205504 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5015fc40 con 0x7ffa500c9500 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa55b0a640 10 client.4310.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa55b0a640 10 monclient: _renew_subs 2026-02-17T05:55:45.451 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa55b0a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:45.452 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa55b0a640 1 -- 10.20.193.16:0/82205504 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa501f4980 con 0x7ffa500c9500 2026-02-17T05:55:45.452 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.437+0000 7ffa55b0a640 1 librados: init done 2026-02-17T05:55:45.452 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.438+0000 7ffa55b0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:45.452 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.438+0000 7ffa55b0a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:45.452 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.438+0000 7ffa55b0a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:45.452 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.438+0000 7ffa55b0a640 1 -- 10.20.193.16:0/82205504 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa50160160 con 0x7ffa500c9500 2026-02-17T05:55:45.453 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.452+0000 7ffa2bfff640 1 -- 10.20.193.16:0/82205504 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7ffa3c037440 con 0x7ffa500c9500 2026-02-17T05:55:45.453 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.452+0000 7ffa2bfff640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffa200790e0 0x7ffa2007b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:45.453 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.452+0000 7ffa4effd640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffa200790e0 0x7ffa2007b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:45.454 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.452+0000 7ffa4effd640 10 monclient: get_auth_request con 0x7ffa200790e0 auth_method 0 2026-02-17T05:55:45.454 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.452+0000 7ffa4effd640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffa200790e0 0x7ffa2007b5a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffa5006f4b0 tx=0x7ffa40007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:45.454 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.452+0000 7ffa2bfff640 10 client.4310.objecter ms_handle_connect 0x7ffa200790e0 2026-02-17T05:55:45.454 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.454+0000 7ffa2bfff640 1 -- 10.20.193.16:0/82205504 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7ffa3c003da0 con 0x7ffa500c9500 2026-02-17T05:55:45.455 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.454+0000 7ffa2bfff640 10 client.4310.objecter ms_dispatch2 0x7ffa50000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:55:45.455 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.454+0000 7ffa2bfff640 3 client.4310.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:55:45.455 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.454+0000 7ffa2bfff640 3 client.4310.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:55:45.455 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.454+0000 7ffa2bfff640 20 client.4310.objecter dump_active .. 0 homeless 2026-02-17T05:55:45.564 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.564+0000 7ffa2bfff640 1 -- 10.20.193.16:0/82205504 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ffa50160160 con 0x7ffa500c9500 2026-02-17T05:55:45.564 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.564+0000 7ffa2bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:45.564 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.564+0000 7ffa2bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:45.656 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.655+0000 7ffa55b0a640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:45.656 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.655+0000 7ffa55b0a640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:45.656 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.655+0000 7ffa55b0a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:55:45.656 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.655+0000 7ffa55b0a640 1 -- 10.20.193.16:0/82205504 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa50073bc0 con 0x7ffa500c9500 2026-02-17T05:55:45.667 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.667+0000 7ffa2bfff640 1 -- 10.20.193.16:0/82205504 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7ffa3c050020 con 0x7ffa500c9500 2026-02-17T05:55:45.667 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.667+0000 7ffa2bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:45.667 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.667+0000 7ffa2bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:45.668 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:55:45.668 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:55:45.670 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 10 librados: watch_flush enter 2026-02-17T05:55:45.670 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 10 librados: watch_flush exit 2026-02-17T05:55:45.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 20 client.4310.objecter shutdown clearing up homeless session... 2026-02-17T05:55:45.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 10 client.4310.objecter successfully canceled tick 2026-02-17T05:55:45.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 -- 10.20.193.16:0/82205504 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffa200790e0 msgr2=0x7ffa2007b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:45.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffa200790e0 0x7ffa2007b5a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffa5006f4b0 tx=0x7ffa40007dd0 comp rx=0 tx=0).stop 2026-02-17T05:55:45.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 10 monclient: shutdown 2026-02-17T05:55:45.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:45.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 -- 10.20.193.16:0/82205504 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa500c9500 msgr2=0x7ffa5006e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:45.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa500c9500 0x7ffa5006e580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c036000 tx=0x7ffa3c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:45.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 -- 10.20.193.16:0/82205504 shutdown_connections 2026-02-17T05:55:45.671 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffa200790e0 0x7ffa2007b5a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:45.672 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffa5015c170 0x7ffa501f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:45.672 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffa501562d0 0x7ffa501f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:45.672 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 --2- 10.20.193.16:0/82205504 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa500c9500 0x7ffa5006e580 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:45.672 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 -- 10.20.193.16:0/82205504 >> 10.20.193.16:0/82205504 conn(0x7ffa500c9f40 msgr2=0x7ffa500ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:45.672 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 -- 10.20.193.16:0/82205504 shutdown_connections 2026-02-17T05:55:45.672 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 -- 10.20.193.16:0/82205504 wait complete. 2026-02-17T05:55:45.672 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:45.669+0000 7ffa55b0a640 1 librados: shutdown 2026-02-17T05:55:45.678 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:55:53.289 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:55:53.355 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.354+0000 7f8cb16b6640 10 monclient: get_monmap_and_config 2026-02-17T05:55:53.355 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.354+0000 7f8cb16b6640 10 monclient: build_initial_monmap 2026-02-17T05:55:53.355 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.354+0000 7f8cb16b6640 10 monclient: monmap: 2026-02-17T05:55:53.355 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:53.355 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:53.355 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:55:53.355111+0000 2026-02-17T05:55:53.355 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:55:53.355111+0000 2026-02-17T05:55:53.355 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.354+0000 7f8cb16b6640 1 Processor -- start 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.354+0000 7f8cb16b6640 1 -- start start 2026-02-17T05:55:53.356 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.354+0000 7f8cb16b6640 10 monclient: init 2026-02-17T05:55:53.359 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 10 monclient: _add_conns ranks=[4,0,3,2,1,5] 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f8cac15c3b0 0x7f8cac15c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 10 monclient(hunting): picked mon.noname-b con 0x7f8cac15c3b0 addr v2:10.20.193.128:3300/0 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8cac070c60 addr v1:10.20.193.16:6789/0 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f8cac0c6b30 0x7f8cac06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 10 monclient(hunting): picked mon.noname-a con 0x7f8cac0c6b30 addr v2:10.20.193.16:3300/0 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cac0c7270 con 0x7f8cac070c60 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8cac0c7550 con 0x7f8cac0c6b30 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:53.360 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f8cac0c7830 con 0x7f8cac15c3b0 2026-02-17T05:55:53.361 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:53.361 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cb16b6640 10 monclient(hunting): authenticate will time out at 1005.924185s 2026-02-17T05:55:53.361 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cab7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f8cac0c6b30 0x7f8cac06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:53.361 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8caaffd640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f8cac15c3b0 0x7f8cac15c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:53.361 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8caaffd640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f8cac15c3b0 0x7f8cac15c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:55544/0 (socket says 10.20.193.16:55544) 2026-02-17T05:55:53.361 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8caaffd640 1 -- 10.20.193.16:0/2008156509 learned_addr learned my addr 10.20.193.16:0/2008156509 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:55:53.361 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8caaffd640 10 monclient(hunting): get_auth_request con 0x7f8cac15c3b0 auth_method 0 2026-02-17T05:55:53.361 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8caaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:53.361 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8caaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.355+0000 7f8cab7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f8cac0c6b30 0x7f8cac06dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:54334/0 (socket says 10.20.193.16:54334) 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8caaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8cab7fe640 10 monclient(hunting): get_auth_request con 0x7f8cac0c6b30 auth_method 0 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8cab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8cab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8cab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8caaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8caaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8caaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8cab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8cab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.356+0000 7f8cab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.357+0000 7f8caaffd640 10 monclient(hunting): handle_auth_done global_id 4288 payload 274 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.357+0000 7f8caaffd640 1 -- 10.20.193.16:0/2008156509 >> v2:10.20.193.16:3300/0 conn(0x7f8cac0c6b30 msgr2=0x7f8cac06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.357+0000 7f8caaffd640 1 --2- 10.20.193.16:0/2008156509 >> v2:10.20.193.16:3300/0 conn(0x7f8cac0c6b30 0x7f8cac06dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.357+0000 7f8caaffd640 1 -- 10.20.193.16:0/2008156509 >> v1:10.20.193.16:6789/0 conn(0x7f8cac070c60 legacy=0x7f8cac0c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.357+0000 7f8caaffd640 10 monclient: _finish_hunting 0 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.357+0000 7f8caaffd640 1 monclient: found mon.noname-b 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.357+0000 7f8caaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.357+0000 7f8caaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.357+0000 7f8caaffd640 1 -- 10.20.193.16:0/2008156509 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac06d4d0 con 0x7f8cac15c3b0 2026-02-17T05:55:53.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.359+0000 7f8caaffd640 1 --2- 10.20.193.16:0/2008156509 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 0x7f8cac15c790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009970 tx=0x7f8ca002eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=959be3c1049dfdcb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 1 -- 10.20.193.16:0/2008156509 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca003c070 con 0x7f8cac15c3b0 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 10 monclient: dump: 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 1 -- 10.20.193.16:0/2008156509 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca002fbc0 con 0x7f8cac15c3b0 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 5 monclient: authenticate success, global_id 4288 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 1 -- 10.20.193.16:0/2008156509 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0037440 con 0x7f8cac15c3b0 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 4 set_mon_vals no callback set 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8ca9ffb640 10 monclient: dump: 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:53.364 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 10 monclient: shutdown 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 1 -- 10.20.193.16:0/2008156509 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 msgr2=0x7f8cac15c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 1 --2- 10.20.193.16:0/2008156509 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 0x7f8cac15c790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009970 tx=0x7f8ca002eef0 comp rx=0 tx=0).stop 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 1 -- 10.20.193.16:0/2008156509 shutdown_connections 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 1 --2- 10.20.193.16:0/2008156509 >> v2:10.20.193.16:3300/0 conn(0x7f8cac0c6b30 0x7f8cac06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 1 --2- 10.20.193.16:0/2008156509 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 0x7f8cac15c790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 1 -- 10.20.193.16:0/2008156509 >> 10.20.193.16:0/2008156509 conn(0x7f8cac0c9f40 msgr2=0x7f8cac0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 1 -- 10.20.193.16:0/2008156509 shutdown_connections 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.361+0000 7f8cb16b6640 1 -- 10.20.193.16:0/2008156509 wait complete. 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient: build_initial_monmap 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient: monmap: 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 librados: starting msgr at 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 librados: starting objecter 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 Processor -- start 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 -- start start 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 librados: setting wanted keys 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 librados: calling monclient init 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient: init 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8cac070c60 0x7f8cac1f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient(hunting): picked mon.noname-c con 0x7f8cac070c60 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8cac0c6b30 0x7f8cac1f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:53.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient(hunting): picked mon.noname-a con 0x7f8cac0c6b30 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 0x7f8cac0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient(hunting): picked mon.noname-b con 0x7f8cac15c3b0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac06e4f0 con 0x7f8cac0c6b30 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac0c6f80 con 0x7f8cac15c3b0 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac06e670 con 0x7f8cac070c60 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.362+0000 7f8cb16b6640 10 monclient(hunting): authenticate will time out at 1005.931944s 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8cab7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 0x7f8cac0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8cab7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 0x7f8cac0c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:55556/0 (socket says 10.20.193.16:55556) 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8cab7fe640 1 -- 10.20.193.16:0/1051810645 learned_addr learned my addr 10.20.193.16:0/1051810645 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8cab7fe640 10 monclient(hunting): get_auth_request con 0x7f8cac15c3b0 auth_method 0 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8cab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8cab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8cab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caa7fc640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8cac0c6b30 0x7f8cac1f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caaffd640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8cac070c60 0x7f8cac1f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caa7fc640 10 monclient(hunting): get_auth_request con 0x7f8cac0c6b30 auth_method 0 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8cab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8cab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8cab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caaffd640 10 monclient(hunting): get_auth_request con 0x7f8cac070c60 auth_method 0 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.363+0000 7f8caa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.364+0000 7f8caaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.364+0000 7f8caaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:53.366 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.364+0000 7f8caaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.365+0000 7f8cab7fe640 10 monclient(hunting): handle_auth_done global_id 4291 payload 824 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.365+0000 7f8cab7fe640 1 -- 10.20.193.16:0/1051810645 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8cac070c60 msgr2=0x7f8cac1f10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.365+0000 7f8cab7fe640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8cac070c60 0x7f8cac1f10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.365+0000 7f8cab7fe640 1 -- 10.20.193.16:0/1051810645 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8cac0c6b30 msgr2=0x7f8cac1f15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.365+0000 7f8cab7fe640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8cac0c6b30 0x7f8cac1f15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.365+0000 7f8cab7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.365+0000 7f8cab7fe640 1 monclient: found mon.noname-b 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.365+0000 7f8cab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.365+0000 7f8cab7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.365+0000 7f8cab7fe640 1 -- 10.20.193.16:0/1051810645 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac14a0c0 con 0x7f8cac15c3b0 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.366+0000 7f8cab7fe640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 0x7f8cac0c8ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00efc0 tx=0x7f8c9c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.366+0000 7f8c8b7fe640 10 client.?.objecter ms_handle_connect 0x7f8cac15c3b0 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.366+0000 7f8c8b7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:55:53.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 1 -- 10.20.193.16:0/1051810645 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c018020 con 0x7f8cac15c3b0 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 10 monclient: dump: 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 1 -- 10.20.193.16:0/1051810645 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c014480 con 0x7f8cac15c3b0 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 1 -- 10.20.193.16:0/1051810645 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c014870 con 0x7f8cac15c3b0 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8c8b7fe640 10 monclient: dump: 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8ca97fa640 4 set_mon_vals no callback set 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8cb16b6640 5 monclient: authenticate success, global_id 4291 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8cb16b6640 10 monclient: _renew_subs 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8cb16b6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8cb16b6640 1 -- 10.20.193.16:0/1051810645 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac14a3a0 con 0x7f8cac15c3b0 2026-02-17T05:55:53.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8cb16b6640 10 client.4291.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:55:53.369 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8cb16b6640 10 monclient: _renew_subs 2026-02-17T05:55:53.369 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8cb16b6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:53.369 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8cb16b6640 1 -- 10.20.193.16:0/1051810645 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac1f08b0 con 0x7f8cac15c3b0 2026-02-17T05:55:53.369 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.367+0000 7f8cb16b6640 1 librados: init done 2026-02-17T05:55:53.369 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.368+0000 7f8cb16b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:53.369 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.368+0000 7f8cb16b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:53.369 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.368+0000 7f8cb16b6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:53.369 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.368+0000 7f8cb16b6640 1 -- 10.20.193.16:0/1051810645 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cac06df90 con 0x7f8cac15c3b0 2026-02-17T05:55:53.380 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.380+0000 7f8c8b7fe640 1 -- 10.20.193.16:0/1051810645 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f8c9c014a10 con 0x7f8cac15c3b0 2026-02-17T05:55:53.381 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.380+0000 7f8c8b7fe640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8c780790b0 0x7f8c7807b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:53.381 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.381+0000 7f8caaffd640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8c780790b0 0x7f8c7807b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:53.381 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.381+0000 7f8caaffd640 10 monclient: get_auth_request con 0x7f8c780790b0 auth_method 0 2026-02-17T05:55:53.381 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.381+0000 7f8c8b7fe640 10 client.4291.objecter ms_handle_connect 0x7f8c780790b0 2026-02-17T05:55:53.381 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.381+0000 7f8caaffd640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8c780790b0 0x7f8c7807b570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cac06f3a0 tx=0x7f8ca0039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:53.383 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.382+0000 7f8c8b7fe640 1 -- 10.20.193.16:0/1051810645 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f8c9c05f170 con 0x7f8cac15c3b0 2026-02-17T05:55:53.383 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.382+0000 7f8c8b7fe640 10 client.4291.objecter ms_dispatch2 0x7f8cac000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:55:53.383 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.382+0000 7f8c8b7fe640 3 client.4291.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:55:53.383 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.382+0000 7f8c8b7fe640 3 client.4291.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:55:53.383 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.382+0000 7f8c8b7fe640 20 client.4291.objecter dump_active .. 0 homeless 2026-02-17T05:55:53.489 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.489+0000 7f8c8b7fe640 1 -- 10.20.193.16:0/1051810645 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8c9c022030 con 0x7f8cac15c3b0 2026-02-17T05:55:53.489 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.489+0000 7f8c8b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:53.489 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.489+0000 7f8c8b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:53.581 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.581+0000 7f8cb16b6640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:53.581 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.581+0000 7f8cb16b6640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:53.581 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.581+0000 7f8cb16b6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:55:53.582 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.581+0000 7f8cb16b6640 1 -- 10.20.193.16:0/1051810645 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cac14d300 con 0x7f8cac15c3b0 2026-02-17T05:55:53.593 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.592+0000 7f8c8b7fe640 1 -- 10.20.193.16:0/1051810645 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f8c9c010040 con 0x7f8cac15c3b0 2026-02-17T05:55:53.593 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.592+0000 7f8c8b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:55:53.593 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.592+0000 7f8c8b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:53.593 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:55:53.594 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:55:53.596 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 10 librados: watch_flush enter 2026-02-17T05:55:53.596 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 10 librados: watch_flush exit 2026-02-17T05:55:53.596 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 20 client.4291.objecter shutdown clearing up homeless session... 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 10 client.4291.objecter successfully canceled tick 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 -- 10.20.193.16:0/1051810645 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8c780790b0 msgr2=0x7f8c7807b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8c780790b0 0x7f8c7807b570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cac06f3a0 tx=0x7f8ca0039040 comp rx=0 tx=0).stop 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 10 monclient: shutdown 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 -- 10.20.193.16:0/1051810645 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 msgr2=0x7f8cac0c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 0x7f8cac0c8ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00efc0 tx=0x7f8c9c00c490 comp rx=0 tx=0).stop 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 -- 10.20.193.16:0/1051810645 shutdown_connections 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8c780790b0 0x7f8c7807b570 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8cac15c3b0 0x7f8cac0c8ac0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:53.597 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8cac0c6b30 0x7f8cac1f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:53.598 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 --2- 10.20.193.16:0/1051810645 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8cac070c60 0x7f8cac1f10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:53.598 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 -- 10.20.193.16:0/1051810645 >> 10.20.193.16:0/1051810645 conn(0x7f8cac0c9f40 msgr2=0x7f8cac0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:53.598 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 -- 10.20.193.16:0/1051810645 shutdown_connections 2026-02-17T05:55:53.598 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 -- 10.20.193.16:0/1051810645 wait complete. 2026-02-17T05:55:53.598 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:55:53.595+0000 7f8cb16b6640 1 librados: shutdown 2026-02-17T05:55:53.604 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:56:01.218 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient: get_monmap_and_config 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient: build_initial_monmap 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient: monmap: 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:56:01.282030+0000 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:56:01.282030+0000 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:01.281 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 1 Processor -- start 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 1 -- start start 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient: init 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient: _add_conns ranks=[1,4,3,2,5,0] 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fd52815d3e0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd52815d7c0 0x7fd5280c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient(hunting): picked mon.noname-b con 0x7fd52815d7c0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fd5280c52d0 0x7fd5280c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient(hunting): picked mon.noname-a con 0x7fd5280c52d0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5280c7ee0 con 0x7fd52815d3e0 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd5280c8dd0 con 0x7fd5280c52d0 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fd5280c81c0 con 0x7fd52815d7c0 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.281+0000 7fd530185640 10 monclient(hunting): authenticate will time out at 1013.850977s 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52e6fb640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fd5280c52d0 0x7fd5280c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52d6f9640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd52815d7c0 0x7fd5280c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52defa640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fd52815d3e0 0x7fd5280c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:34378/0 (socket says 10.20.193.16:34378) 2026-02-17T05:56:01.282 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52defa640 1 -- 10.20.193.16:0/2227146901 learned_addr learned my addr 10.20.193.16:0/2227146901 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52e6fb640 10 monclient(hunting): get_auth_request con 0x7fd5280c52d0 auth_method 0 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52e6fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52e6fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52e6fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52d6f9640 10 monclient(hunting): get_auth_request con 0x7fd52815d7c0 auth_method 0 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52d6f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52d6f9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.282+0000 7fd52d6f9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.283+0000 7fd52e6fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.283+0000 7fd52e6fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.283+0000 7fd52e6fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.283+0000 7fd52d6f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.283+0000 7fd52d6f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:01.283 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.283+0000 7fd52d6f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:01.284 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.284+0000 7fd52e6fb640 10 monclient(hunting): handle_auth_done global_id 4275 payload 274 2026-02-17T05:56:01.284 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.284+0000 7fd52e6fb640 1 -- 10.20.193.16:0/2227146901 >> v2:10.20.193.128:3300/0 conn(0x7fd52815d7c0 msgr2=0x7fd5280c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:01.285 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.284+0000 7fd52e6fb640 1 --2- 10.20.193.16:0/2227146901 >> v2:10.20.193.128:3300/0 conn(0x7fd52815d7c0 0x7fd5280c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:01.285 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.284+0000 7fd52e6fb640 1 -- 10.20.193.16:0/2227146901 >> v1:10.20.193.128:6789/0 conn(0x7fd52815d3e0 legacy=0x7fd5280c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:01.285 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.284+0000 7fd52e6fb640 10 monclient: _finish_hunting 0 2026-02-17T05:56:01.285 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.284+0000 7fd52e6fb640 1 monclient: found mon.noname-a 2026-02-17T05:56:01.285 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.284+0000 7fd52e6fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:01.285 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.284+0000 7fd52e6fb640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:01.285 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.284+0000 7fd52e6fb640 1 -- 10.20.193.16:0/2227146901 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5280c84a0 con 0x7fd5280c52d0 2026-02-17T05:56:01.285 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.285+0000 7fd52e6fb640 1 --2- 10.20.193.16:0/2227146901 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 0x7fd5280c5710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd528069c50 tx=0x7fd52402f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec63689b33577a5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:01.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 1 -- 10.20.193.16:0/2227146901 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52403c070 con 0x7fd5280c52d0 2026-02-17T05:56:01.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:01.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 10 monclient: dump: 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:01.293 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 10 monclient: _finish_auth 0 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 10 monclient: _check_auth_tickets 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 1 -- 10.20.193.16:0/2227146901 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd524037440 con 0x7fd5280c52d0 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd530185640 5 monclient: authenticate success, global_id 4275 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd530185640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 1 -- 10.20.193.16:0/2227146901 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524037810 con 0x7fd5280c52d0 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd530185640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd530185640 4 set_mon_vals no callback set 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:01.294 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd52cef8640 10 monclient: dump: 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:01.295 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd530185640 10 monclient: shutdown 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd530185640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd530185640 1 -- 10.20.193.16:0/2227146901 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 msgr2=0x7fd5280c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.287+0000 7fd530185640 1 --2- 10.20.193.16:0/2227146901 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 0x7fd5280c5710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd528069c50 tx=0x7fd52402f1c0 comp rx=0 tx=0).stop 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 -- 10.20.193.16:0/2227146901 shutdown_connections 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 --2- 10.20.193.16:0/2227146901 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 0x7fd5280c5710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 --2- 10.20.193.16:0/2227146901 >> v2:10.20.193.128:3300/0 conn(0x7fd52815d7c0 0x7fd5280c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 -- 10.20.193.16:0/2227146901 >> 10.20.193.16:0/2227146901 conn(0x7fd5280c9f40 msgr2=0x7fd5280ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 -- 10.20.193.16:0/2227146901 shutdown_connections 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 -- 10.20.193.16:0/2227146901 wait complete. 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 10 monclient: build_initial_monmap 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 10 monclient: monmap: 2026-02-17T05:56:01.296 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 librados: starting msgr at 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 librados: starting objecter 2026-02-17T05:56:01.297 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 Processor -- start 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 -- start start 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 librados: setting wanted keys 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 1 librados: calling monclient init 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.288+0000 7fd530185640 10 monclient: init 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 0x7fd5281469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 10 monclient(hunting): picked mon.noname-a con 0x7fd5280c52d0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd52815d3e0 0x7fd5281f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 10 monclient(hunting): picked mon.noname-b con 0x7fd52815d3e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd52815d7c0 0x7fd5281f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:01.298 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 10 monclient(hunting): picked mon.noname-c con 0x7fd52815d7c0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280c5c50 con 0x7fd5280c52d0 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280c8a60 con 0x7fd52815d3e0 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd52815b070 con 0x7fd52815d7c0 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd530185640 10 monclient(hunting): authenticate will time out at 1013.858151s 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52defa640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 0x7fd5281469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52d6f9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd52815d3e0 0x7fd5281f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52defa640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 0x7fd5281469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:35968/0 (socket says 10.20.193.16:35968) 2026-02-17T05:56:01.299 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52defa640 1 -- 10.20.193.16:0/2517157576 learned_addr learned my addr 10.20.193.16:0/2517157576 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52defa640 10 monclient(hunting): get_auth_request con 0x7fd5280c52d0 auth_method 0 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52defa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52defa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52defa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52d6f9640 10 monclient(hunting): get_auth_request con 0x7fd52815d3e0 auth_method 0 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52d6f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52d6f9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52d6f9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52defa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52defa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52defa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:01.300 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.289+0000 7fd52e6fb640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd52815d7c0 0x7fd5281f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.290+0000 7fd52d6f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.290+0000 7fd52d6f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.290+0000 7fd52d6f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.290+0000 7fd52e6fb640 10 monclient(hunting): get_auth_request con 0x7fd52815d7c0 auth_method 0 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.290+0000 7fd52e6fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.290+0000 7fd52e6fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.290+0000 7fd52e6fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.290+0000 7fd52e6fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.290+0000 7fd52e6fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.290+0000 7fd52e6fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.291+0000 7fd52defa640 10 monclient(hunting): handle_auth_done global_id 4278 payload 824 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.291+0000 7fd52defa640 1 -- 10.20.193.16:0/2517157576 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd52815d7c0 msgr2=0x7fd5281f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:01.301 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.291+0000 7fd52defa640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd52815d7c0 0x7fd5281f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:01.302 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.291+0000 7fd52defa640 1 -- 10.20.193.16:0/2517157576 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd52815d3e0 msgr2=0x7fd5281f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:01.302 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.291+0000 7fd52defa640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd52815d3e0 0x7fd5281f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:01.302 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.291+0000 7fd52defa640 10 monclient: _finish_hunting 0 2026-02-17T05:56:01.302 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.291+0000 7fd52defa640 1 monclient: found mon.noname-a 2026-02-17T05:56:01.302 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.291+0000 7fd52defa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:01.302 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.291+0000 7fd52defa640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:01.302 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.291+0000 7fd52defa640 1 -- 10.20.193.16:0/2517157576 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd528145fd0 con 0x7fd5280c52d0 2026-02-17T05:56:01.304 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.292+0000 7fd52defa640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 0x7fd5281469a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c760 tx=0x7fd51800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:01.304 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.292+0000 7fd5167fc640 10 client.?.objecter ms_handle_connect 0x7fd5280c52d0 2026-02-17T05:56:01.304 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.292+0000 7fd5167fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:01.304 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 1 -- 10.20.193.16:0/2517157576 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518017070 con 0x7fd5280c52d0 2026-02-17T05:56:01.304 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:01.304 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:01.304 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 10 monclient: dump: 2026-02-17T05:56:01.304 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:01.305 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd530185640 5 monclient: authenticate success, global_id 4278 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 1 -- 10.20.193.16:0/2517157576 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd518005b90 con 0x7fd5280c52d0 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 1 -- 10.20.193.16:0/2517157576 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518007070 con 0x7fd5280c52d0 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd517fff640 4 set_mon_vals no callback set 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd5167fc640 10 monclient: dump: 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:01.306 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd530185640 10 monclient: _renew_subs 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd530185640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd530185640 1 -- 10.20.193.16:0/2517157576 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd528146f90 con 0x7fd5280c52d0 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd530185640 10 client.4278.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd530185640 10 monclient: _renew_subs 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd530185640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd530185640 1 -- 10.20.193.16:0/2517157576 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd528147550 con 0x7fd5280c52d0 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.294+0000 7fd530185640 1 librados: init done 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.295+0000 7fd530185640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:01.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.295+0000 7fd530185640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:01.308 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.295+0000 7fd530185640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:01.308 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.295+0000 7fd530185640 1 -- 10.20.193.16:0/2517157576 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd528147750 con 0x7fd5280c52d0 2026-02-17T05:56:01.308 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.306+0000 7fd5167fc640 1 -- 10.20.193.16:0/2517157576 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7fd518007210 con 0x7fd5280c52d0 2026-02-17T05:56:01.309 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.307+0000 7fd5167fc640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd4f80790e0 0x7fd4f807b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:01.309 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.307+0000 7fd52d6f9640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd4f80790e0 0x7fd4f807b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:01.309 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.307+0000 7fd52d6f9640 10 monclient: get_auth_request con 0x7fd4f80790e0 auth_method 0 2026-02-17T05:56:01.309 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.307+0000 7fd5167fc640 10 client.4278.objecter ms_handle_connect 0x7fd4f80790e0 2026-02-17T05:56:01.309 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.307+0000 7fd52d6f9640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd4f80790e0 0x7fd4f807b5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd528146710 tx=0x7fd51c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:01.310 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.309+0000 7fd5167fc640 1 -- 10.20.193.16:0/2517157576 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7fd51805d680 con 0x7fd5280c52d0 2026-02-17T05:56:01.310 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.309+0000 7fd5167fc640 10 client.4278.objecter ms_dispatch2 0x7fd528000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:56:01.310 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.309+0000 7fd5167fc640 3 client.4278.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:56:01.310 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.309+0000 7fd5167fc640 3 client.4278.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:56:01.310 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.309+0000 7fd5167fc640 20 client.4278.objecter dump_active .. 0 homeless 2026-02-17T05:56:01.418 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.418+0000 7fd5167fc640 1 -- 10.20.193.16:0/2517157576 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd528147750 con 0x7fd5280c52d0 2026-02-17T05:56:01.419 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.418+0000 7fd5167fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:01.419 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.418+0000 7fd5167fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:01.510 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.509+0000 7fd530185640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:01.510 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.509+0000 7fd530185640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:01.510 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.509+0000 7fd530185640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:01.511 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.509+0000 7fd530185640 1 -- 10.20.193.16:0/2517157576 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5280c5710 con 0x7fd5280c52d0 2026-02-17T05:56:01.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.522+0000 7fd5167fc640 1 -- 10.20.193.16:0/2517157576 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7fd518021020 con 0x7fd5280c52d0 2026-02-17T05:56:01.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.522+0000 7fd5167fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:01.522 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.522+0000 7fd5167fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:01.522 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:56:01.523 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:56:01.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 10 librados: watch_flush enter 2026-02-17T05:56:01.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 10 librados: watch_flush exit 2026-02-17T05:56:01.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 20 client.4278.objecter shutdown clearing up homeless session... 2026-02-17T05:56:01.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 10 client.4278.objecter successfully canceled tick 2026-02-17T05:56:01.525 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 -- 10.20.193.16:0/2517157576 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd4f80790e0 msgr2=0x7fd4f807b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd4f80790e0 0x7fd4f807b5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd528146710 tx=0x7fd51c009290 comp rx=0 tx=0).stop 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 10 monclient: shutdown 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 -- 10.20.193.16:0/2517157576 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 msgr2=0x7fd5281469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 0x7fd5281469a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c760 tx=0x7fd51800cc30 comp rx=0 tx=0).stop 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 -- 10.20.193.16:0/2517157576 shutdown_connections 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd4f80790e0 0x7fd4f807b5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd52815d3e0 0x7fd5281f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd52815d7c0 0x7fd5281f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 --2- 10.20.193.16:0/2517157576 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd5280c52d0 0x7fd5281469a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 -- 10.20.193.16:0/2517157576 >> 10.20.193.16:0/2517157576 conn(0x7fd5280c9f40 msgr2=0x7fd5280ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 -- 10.20.193.16:0/2517157576 shutdown_connections 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 -- 10.20.193.16:0/2517157576 wait complete. 2026-02-17T05:56:01.526 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:01.524+0000 7fd530185640 1 librados: shutdown 2026-02-17T05:56:01.534 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:56:03.231 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:01:42.059 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:03.233 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:01:42.061 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:03.252 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:01:42.079 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:03.262 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:01:42.051 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:03.263 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:01:42.089 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:03.263 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:01:42.052 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:03.264 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:01:42.090 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:03.281 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:01:42.070 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:03.283 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:01:42.109 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:09.143 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:56:09.207 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.205+0000 7f55c1a91640 10 monclient: get_monmap_and_config 2026-02-17T05:56:09.208 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.205+0000 7f55c1a91640 10 monclient: build_initial_monmap 2026-02-17T05:56:09.208 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.205+0000 7f55c1a91640 10 monclient: monmap: 2026-02-17T05:56:09.208 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:09.208 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:09.208 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:56:09.206279+0000 2026-02-17T05:56:09.208 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:56:09.206279+0000 2026-02-17T05:56:09.208 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:09.208 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:09.209 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:09.209 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:09.209 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:09.209 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:09.209 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:09.209 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:09.209 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:09.209 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.205+0000 7f55c1a91640 1 Processor -- start 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.205+0000 7f55c1a91640 1 -- start start 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.205+0000 7f55c1a91640 10 monclient: init 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 10 monclient: _add_conns ranks=[4,2,5,1,0,3] 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f55bc156300 0x7f55bc15c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 10 monclient(hunting): picked mon.noname-b con 0x7f55bc156300 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f55bc0d2480 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f55bc0c9d10 0x7f55bc15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 10 monclient(hunting): picked mon.noname-c con 0x7f55bc0c9d10 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55bc15d050 con 0x7f55bc0d2480 2026-02-17T05:56:09.210 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f55bc15c7b0 con 0x7f55bc156300 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f55bc15ca90 con 0x7f55bc0c9d10 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55c1a91640 10 monclient(hunting): authenticate will time out at 1021.775248s 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55bb7fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f55bc156300 0x7f55bc15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.206+0000 7f55bbfff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f55bc0c9d10 0x7f55bc15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55baffd640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f55bc0d2480 0x7f55bc0c9600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:41760/0 (socket says 10.20.193.16:41760) 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55baffd640 1 -- 10.20.193.16:0/1550852574 learned_addr learned my addr 10.20.193.16:0/1550852574 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bb7fe640 10 monclient(hunting): get_auth_request con 0x7f55bc156300 auth_method 0 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:09.211 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:09.212 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:09.212 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bbfff640 10 monclient(hunting): get_auth_request con 0x7f55bc0c9d10 auth_method 0 2026-02-17T05:56:09.212 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:09.212 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:09.212 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:09.212 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:09.212 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:09.212 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.207+0000 7f55bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.208+0000 7f55bb7fe640 10 monclient(hunting): handle_auth_done global_id 4303 payload 274 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.208+0000 7f55bb7fe640 1 -- 10.20.193.16:0/1550852574 >> v2:10.20.193.148:3300/0 conn(0x7f55bc0c9d10 msgr2=0x7f55bc15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.208+0000 7f55bb7fe640 1 --2- 10.20.193.16:0/1550852574 >> v2:10.20.193.148:3300/0 conn(0x7f55bc0c9d10 0x7f55bc15f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.208+0000 7f55bb7fe640 1 -- 10.20.193.16:0/1550852574 >> v1:10.20.193.148:6789/0 conn(0x7f55bc0d2480 legacy=0x7f55bc0c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.208+0000 7f55bb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.208+0000 7f55bb7fe640 1 monclient: found mon.noname-b 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.208+0000 7f55bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.208+0000 7f55bb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.208+0000 7f55bb7fe640 1 -- 10.20.193.16:0/1550852574 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc15cd70 con 0x7f55bc156300 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.209+0000 7f55bb7fe640 1 --2- 10.20.193.16:0/1550852574 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc156300 0x7f55bc15c150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f55bc08fc70 tx=0x7f55a802ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad4a0d007c11c296 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 1 -- 10.20.193.16:0/1550852574 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a803c070 con 0x7f55bc156300 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 10 monclient: dump: 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:09.218 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55c1a91640 5 monclient: authenticate success, global_id 4303 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55c1a91640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 1 -- 10.20.193.16:0/1550852574 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a802faf0 con 0x7f55bc156300 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 1 -- 10.20.193.16:0/1550852574 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8037440 con 0x7f55bc156300 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55ba7fc640 10 monclient: dump: 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55c1a91640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.211+0000 7f55c1a91640 4 set_mon_vals no callback set 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 10 monclient: shutdown 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 -- 10.20.193.16:0/1550852574 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc156300 msgr2=0x7f55bc15c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 --2- 10.20.193.16:0/1550852574 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc156300 0x7f55bc15c150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f55bc08fc70 tx=0x7f55a802ede0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 -- 10.20.193.16:0/1550852574 shutdown_connections 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 --2- 10.20.193.16:0/1550852574 >> v2:10.20.193.148:3300/0 conn(0x7f55bc0c9d10 0x7f55bc15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 --2- 10.20.193.16:0/1550852574 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc156300 0x7f55bc15c150 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 -- 10.20.193.16:0/1550852574 >> 10.20.193.16:0/1550852574 conn(0x7f55bc0d1960 msgr2=0x7f55bc0d1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 -- 10.20.193.16:0/1550852574 shutdown_connections 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 -- 10.20.193.16:0/1550852574 wait complete. 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 10 monclient: build_initial_monmap 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 10 monclient: monmap: 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:56:09.219 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 librados: starting msgr at 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 librados: starting objecter 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 Processor -- start 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 -- start start 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 librados: setting wanted keys 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 1 librados: calling monclient init 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.212+0000 7f55c1a91640 10 monclient: init 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc0c9d10 0x7f55bc15e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 10 monclient(hunting): picked mon.noname-b con 0x7f55bc0c9d10 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55bc0d2480 0x7f55bc1f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 10 monclient(hunting): picked mon.noname-a con 0x7f55bc0d2480 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f55bc156300 0x7f55bc1f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 10 monclient(hunting): picked mon.noname-c con 0x7f55bc156300 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc06e8c0 con 0x7f55bc0d2480 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc0c90e0 con 0x7f55bc0c9d10 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc08e3a0 con 0x7f55bc156300 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55c1a91640 10 monclient(hunting): authenticate will time out at 1021.782395s 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bb7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc0c9d10 0x7f55bc15e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bbfff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f55bc156300 0x7f55bc1f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bb7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc0c9d10 0x7f55bc15e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:48202/0 (socket says 10.20.193.16:48202) 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bb7fe640 1 -- 10.20.193.16:0/736707088 learned_addr learned my addr 10.20.193.16:0/736707088 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bb7fe640 10 monclient(hunting): get_auth_request con 0x7f55bc0c9d10 auth_method 0 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bbfff640 10 monclient(hunting): get_auth_request con 0x7f55bc156300 auth_method 0 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.213+0000 7f55bbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55baffd640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55bc0d2480 0x7f55bc1f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:09.220 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55baffd640 10 monclient(hunting): get_auth_request con 0x7f55bc0d2480 auth_method 0 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55baffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.214+0000 7f55baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.215+0000 7f55bb7fe640 10 monclient(hunting): handle_auth_done global_id 4306 payload 824 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.215+0000 7f55bb7fe640 1 -- 10.20.193.16:0/736707088 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f55bc156300 msgr2=0x7f55bc1f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.215+0000 7f55bb7fe640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f55bc156300 0x7f55bc1f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.215+0000 7f55bb7fe640 1 -- 10.20.193.16:0/736707088 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55bc0d2480 msgr2=0x7f55bc1f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.215+0000 7f55bb7fe640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55bc0d2480 0x7f55bc1f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.215+0000 7f55bb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.215+0000 7f55bb7fe640 1 monclient: found mon.noname-b 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.215+0000 7f55bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.215+0000 7f55bb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.215+0000 7f55bb7fe640 1 -- 10.20.193.16:0/736707088 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc0ce120 con 0x7f55bc0c9d10 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.216+0000 7f55bb7fe640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc0c9d10 0x7f55bc15e8f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009130 tx=0x7f55a80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.216+0000 7f5597fff640 10 client.?.objecter ms_handle_connect 0x7f55bc0c9d10 2026-02-17T05:56:09.221 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.216+0000 7f5597fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 1 -- 10.20.193.16:0/736707088 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a803c040 con 0x7f55bc0c9d10 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 10 monclient: dump: 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 1 -- 10.20.193.16:0/736707088 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a8042610 con 0x7f55bc0c9d10 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 1 -- 10.20.193.16:0/736707088 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a80429e0 con 0x7f55bc0c9d10 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f55b9ffb640 4 set_mon_vals no callback set 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f5597fff640 10 monclient: dump: 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f55c1a91640 5 monclient: authenticate success, global_id 4306 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f55c1a91640 10 monclient: _renew_subs 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f55c1a91640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f55c1a91640 1 -- 10.20.193.16:0/736707088 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc0cd230 con 0x7f55bc0c9d10 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f55c1a91640 10 client.4306.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:09.222 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f55c1a91640 10 monclient: _renew_subs 2026-02-17T05:56:09.223 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f55c1a91640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:09.223 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f55c1a91640 1 -- 10.20.193.16:0/736707088 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc0cd7f0 con 0x7f55bc0c9d10 2026-02-17T05:56:09.223 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.218+0000 7f55c1a91640 1 librados: init done 2026-02-17T05:56:09.223 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.219+0000 7f55c1a91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:09.223 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.219+0000 7f55c1a91640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:09.223 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.219+0000 7f55c1a91640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:09.223 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.219+0000 7f55c1a91640 1 -- 10.20.193.16:0/736707088 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc15c150 con 0x7f55bc0c9d10 2026-02-17T05:56:09.231 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.231+0000 7f5597fff640 1 -- 10.20.193.16:0/736707088 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f55a804b450 con 0x7f55bc0c9d10 2026-02-17T05:56:09.231 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.231+0000 7f5597fff640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f558c0790e0 0x7f558c07b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:09.231 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.231+0000 7f55baffd640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f558c0790e0 0x7f558c07b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:09.231 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.231+0000 7f55baffd640 10 monclient: get_auth_request con 0x7f558c0790e0 auth_method 0 2026-02-17T05:56:09.231 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.231+0000 7f5597fff640 10 client.4306.objecter ms_handle_connect 0x7f558c0790e0 2026-02-17T05:56:09.231 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.231+0000 7f55baffd640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f558c0790e0 0x7f558c07b5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f55bc06c1a0 tx=0x7f55ac002890 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:09.279 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.279+0000 7f5597fff640 1 -- 10.20.193.16:0/736707088 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f55a802fa70 con 0x7f55bc0c9d10 2026-02-17T05:56:09.279 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.279+0000 7f5597fff640 10 client.4306.objecter ms_dispatch2 0x7f55bc000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:56:09.279 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.279+0000 7f5597fff640 3 client.4306.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:56:09.279 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.279+0000 7f5597fff640 3 client.4306.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:56:09.279 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.279+0000 7f5597fff640 20 client.4306.objecter dump_active .. 0 homeless 2026-02-17T05:56:09.341 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.341+0000 7f5597fff640 1 -- 10.20.193.16:0/736707088 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f55a8086b80 con 0x7f55bc0c9d10 2026-02-17T05:56:09.342 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.341+0000 7f5597fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:09.342 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.341+0000 7f5597fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:09.433 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.432+0000 7f55c1a91640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:09.433 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.432+0000 7f55c1a91640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:09.433 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.432+0000 7f55c1a91640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:09.433 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.432+0000 7f55c1a91640 1 -- 10.20.193.16:0/736707088 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55bc15fec0 con 0x7f55bc0c9d10 2026-02-17T05:56:09.443 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.443+0000 7f5597fff640 1 -- 10.20.193.16:0/736707088 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f55a808ae80 con 0x7f55bc0c9d10 2026-02-17T05:56:09.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.443+0000 7f5597fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:09.444 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.443+0000 7f5597fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:09.444 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:56:09.444 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:56:09.447 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 10 librados: watch_flush enter 2026-02-17T05:56:09.447 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 10 librados: watch_flush exit 2026-02-17T05:56:09.447 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 20 client.4306.objecter shutdown clearing up homeless session... 2026-02-17T05:56:09.447 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 10 client.4306.objecter successfully canceled tick 2026-02-17T05:56:09.447 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 -- 10.20.193.16:0/736707088 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f558c0790e0 msgr2=0x7f558c07b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f558c0790e0 0x7f558c07b5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f55bc06c1a0 tx=0x7f55ac002890 comp rx=0 tx=0).stop 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 10 monclient: shutdown 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 -- 10.20.193.16:0/736707088 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc0c9d10 msgr2=0x7f55bc15e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc0c9d10 0x7f55bc15e8f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009130 tx=0x7f55a80047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 -- 10.20.193.16:0/736707088 shutdown_connections 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f558c0790e0 0x7f558c07b5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f55bc156300 0x7f55bc1f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55bc0d2480 0x7f55bc1f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 --2- 10.20.193.16:0/736707088 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55bc0c9d10 0x7f55bc15e8f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:09.448 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 -- 10.20.193.16:0/736707088 >> 10.20.193.16:0/736707088 conn(0x7f55bc0d1960 msgr2=0x7f55bc153160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:09.449 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 -- 10.20.193.16:0/736707088 shutdown_connections 2026-02-17T05:56:09.449 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 -- 10.20.193.16:0/736707088 wait complete. 2026-02-17T05:56:09.449 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:09.446+0000 7f55c1a91640 1 librados: shutdown 2026-02-17T05:56:09.454 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:56:17.065 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:56:17.126 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.125+0000 7f5627577640 10 monclient: get_monmap_and_config 2026-02-17T05:56:17.126 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.125+0000 7f5627577640 10 monclient: build_initial_monmap 2026-02-17T05:56:17.126 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.125+0000 7f5627577640 10 monclient: monmap: 2026-02-17T05:56:17.126 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:17.126 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:17.126 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:56:17.126428+0000 2026-02-17T05:56:17.127 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:56:17.126428+0000 2026-02-17T05:56:17.127 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:17.127 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:17.127 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:17.127 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:17.127 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:17.127 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:17.127 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:17.127 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:17.127 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:17.130 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 1 Processor -- start 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 1 -- start start 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 10 monclient: init 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 10 monclient: _add_conns ranks=[2,5,4,0,3,1] 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f562806d830 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f562806dc10 0x7f56280cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 10 monclient(hunting): picked mon.noname-c con 0x7f562806dc10 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f562806e3f0 0x7f562815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 10 monclient(hunting): picked mon.noname-b con 0x7f562806e3f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56280cc6b0 con 0x7f562806d830 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:17.131 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f56280cf100 con 0x7f562806e3f0 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f56280cc990 con 0x7f562806dc10 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.126+0000 7f5627577640 10 monclient(hunting): authenticate will time out at 1029.695705s 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5626d76640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f562806e3f0 0x7f562815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5625d74640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f562806dc10 0x7f56280cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5626d76640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f562806e3f0 0x7f562815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:41658/0 (socket says 10.20.193.16:41658) 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5626d76640 1 -- 10.20.193.16:0/2031363792 learned_addr learned my addr 10.20.193.16:0/2031363792 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5626d76640 10 monclient(hunting): get_auth_request con 0x7f562806e3f0 auth_method 0 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5626d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5626d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:17.132 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5626d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5625d74640 10 monclient(hunting): get_auth_request con 0x7f562806dc10 auth_method 0 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5625d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5625d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5625d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5626d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5626d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.127+0000 7f5626d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5625d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5625d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5625d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5626d76640 10 monclient(hunting): handle_auth_done global_id 4309 payload 274 2026-02-17T05:56:17.133 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5626d76640 1 -- 10.20.193.16:0/2031363792 >> v2:10.20.193.148:3300/0 conn(0x7f562806dc10 msgr2=0x7f56280cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:17.134 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5626d76640 1 --2- 10.20.193.16:0/2031363792 >> v2:10.20.193.148:3300/0 conn(0x7f562806dc10 0x7f56280cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:17.134 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5626d76640 1 -- 10.20.193.16:0/2031363792 >> v1:10.20.193.148:6789/0 conn(0x7f562806d830 legacy=0x7f562815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:17.134 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5626d76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:17.134 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5626d76640 1 monclient: found mon.noname-b 2026-02-17T05:56:17.134 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5626d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:17.134 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5626d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:17.134 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.128+0000 7f5626d76640 1 -- 10.20.193.16:0/2031363792 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56280ccc70 con 0x7f562806e3f0 2026-02-17T05:56:17.134 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.129+0000 7f5626d76640 1 --2- 10.20.193.16:0/2031363792 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806e3f0 0x7f562815d040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56280ceb90 tx=0x7f561802f270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d9b904d29dc0e1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 1 -- 10.20.193.16:0/2031363792 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561802fd70 con 0x7f562806e3f0 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 10 monclient: dump: 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 10 monclient: _finish_auth 0 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 10 monclient: _check_auth_tickets 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 1 -- 10.20.193.16:0/2031363792 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56180373f0 con 0x7f562806e3f0 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 5 monclient: authenticate success, global_id 4309 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 1 -- 10.20.193.16:0/2031363792 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56180377c0 con 0x7f562806e3f0 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 4 set_mon_vals no callback set 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5625573640 10 monclient: dump: 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:17.139 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 10 monclient: shutdown 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 1 -- 10.20.193.16:0/2031363792 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806e3f0 msgr2=0x7f562815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 1 --2- 10.20.193.16:0/2031363792 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806e3f0 0x7f562815d040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56280ceb90 tx=0x7f561802f270 comp rx=0 tx=0).stop 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 1 -- 10.20.193.16:0/2031363792 shutdown_connections 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 1 --2- 10.20.193.16:0/2031363792 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806e3f0 0x7f562815d040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 1 --2- 10.20.193.16:0/2031363792 >> v2:10.20.193.148:3300/0 conn(0x7f562806dc10 0x7f56280cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 1 -- 10.20.193.16:0/2031363792 >> 10.20.193.16:0/2031363792 conn(0x7f5628068950 msgr2=0x7f5628068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 1 -- 10.20.193.16:0/2031363792 shutdown_connections 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.131+0000 7f5627577640 1 -- 10.20.193.16:0/2031363792 wait complete. 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient: build_initial_monmap 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient: monmap: 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 librados: starting msgr at 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 librados: starting objecter 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 Processor -- start 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 -- start start 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 librados: setting wanted keys 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 librados: calling monclient init 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient: init 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f562806d830 0x7f562814a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient(hunting): picked mon.noname-a con 0x7f562806d830 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806dc10 0x7f56281f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient(hunting): picked mon.noname-b con 0x7f562806dc10 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:17.140 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f562806e3f0 0x7f56281f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient(hunting): picked mon.noname-c con 0x7f562806e3f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56280cd510 con 0x7f562806d830 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f56280cd230 con 0x7f562806dc10 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f562806e860 con 0x7f562806e3f0 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.132+0000 7f5627577640 10 monclient(hunting): authenticate will time out at 1029.701985s 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5625d74640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806dc10 0x7f56281f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626d76640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f562806e3f0 0x7f56281f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5625d74640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806dc10 0x7f56281f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:41662/0 (socket says 10.20.193.16:41662) 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5625d74640 1 -- 10.20.193.16:0/3660118570 learned_addr learned my addr 10.20.193.16:0/3660118570 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5625d74640 10 monclient(hunting): get_auth_request con 0x7f562806dc10 auth_method 0 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5625d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5625d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5625d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626575640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f562806d830 0x7f562814a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626d76640 10 monclient(hunting): get_auth_request con 0x7f562806e3f0 auth_method 0 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5625d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5625d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5625d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626575640 10 monclient(hunting): get_auth_request con 0x7f562806d830 auth_method 0 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.133+0000 7f5626d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.134+0000 7f5626575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.134+0000 7f5626575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.134+0000 7f5626575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.135+0000 7f5625d74640 10 monclient(hunting): handle_auth_done global_id 4312 payload 824 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.135+0000 7f5625d74640 1 -- 10.20.193.16:0/3660118570 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f562806e3f0 msgr2=0x7f56281f6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.135+0000 7f5625d74640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f562806e3f0 0x7f56281f6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.135+0000 7f5625d74640 1 -- 10.20.193.16:0/3660118570 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f562806d830 msgr2=0x7f562814a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.135+0000 7f5625d74640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f562806d830 0x7f562814a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.135+0000 7f5625d74640 10 monclient: _finish_hunting 0 2026-02-17T05:56:17.141 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.135+0000 7f5625d74640 1 monclient: found mon.noname-b 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.135+0000 7f5625d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.135+0000 7f5625d74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.135+0000 7f5625d74640 1 -- 10.20.193.16:0/3660118570 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562814a940 con 0x7f562806dc10 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.136+0000 7f5625d74640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806dc10 0x7f56281f62e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f561000ec10 tx=0x7f561000c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.136+0000 7f560effd640 10 client.?.objecter ms_handle_connect 0x7f562806dc10 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.136+0000 7f560effd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 1 -- 10.20.193.16:0/3660118570 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610017070 con 0x7f562806dc10 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 10 monclient: dump: 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 10 monclient: _finish_auth 0 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 1 -- 10.20.193.16:0/3660118570 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561000cc40 con 0x7f562806dc10 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 1 -- 10.20.193.16:0/3660118570 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610013470 con 0x7f562806dc10 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f5624d72640 4 set_mon_vals no callback set 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f560effd640 10 monclient: dump: 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f5627577640 5 monclient: authenticate success, global_id 4312 2026-02-17T05:56:17.142 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f5627577640 10 monclient: _renew_subs 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f5627577640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f5627577640 1 -- 10.20.193.16:0/3660118570 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5628149080 con 0x7f562806dc10 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f5627577640 10 client.4312.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f5627577640 10 monclient: _renew_subs 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f5627577640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f5627577640 1 -- 10.20.193.16:0/3660118570 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5628149640 con 0x7f562806dc10 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.137+0000 7f5627577640 1 librados: init done 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.138+0000 7f5627577640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.138+0000 7f5627577640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.138+0000 7f5627577640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:17.143 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.138+0000 7f5627577640 1 -- 10.20.193.16:0/3660118570 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562814f060 con 0x7f562806dc10 2026-02-17T05:56:17.150 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.150+0000 7f560effd640 1 -- 10.20.193.16:0/3660118570 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f5610024060 con 0x7f562806dc10 2026-02-17T05:56:17.151 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.150+0000 7f560effd640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55f40790e0 0x7f55f407b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:17.151 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.150+0000 7f5626575640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55f40790e0 0x7f55f407b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:17.151 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.150+0000 7f5626575640 10 monclient: get_auth_request con 0x7f55f40790e0 auth_method 0 2026-02-17T05:56:17.151 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.151+0000 7f560effd640 10 client.4312.objecter ms_handle_connect 0x7f55f40790e0 2026-02-17T05:56:17.152 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.151+0000 7f5626575640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55f40790e0 0x7f55f407b5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f561c005110 tx=0x7f561c007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:17.153 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.153+0000 7f560effd640 1 -- 10.20.193.16:0/3660118570 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f561005e420 con 0x7f562806dc10 2026-02-17T05:56:17.153 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.153+0000 7f560effd640 10 client.4312.objecter ms_dispatch2 0x7f5628000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:56:17.153 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.153+0000 7f560effd640 3 client.4312.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:56:17.153 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.153+0000 7f560effd640 3 client.4312.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:56:17.153 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.153+0000 7f560effd640 20 client.4312.objecter dump_active .. 0 homeless 2026-02-17T05:56:17.263 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.262+0000 7f560effd640 1 -- 10.20.193.16:0/3660118570 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f562814f060 con 0x7f562806dc10 2026-02-17T05:56:17.263 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.262+0000 7f560effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:17.263 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.262+0000 7f560effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:17.354 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.354+0000 7f5627577640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:17.355 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.354+0000 7f5627577640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:17.355 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.354+0000 7f5627577640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:17.355 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.354+0000 7f5627577640 1 -- 10.20.193.16:0/3660118570 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56280cc050 con 0x7f562806dc10 2026-02-17T05:56:17.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.367+0000 7f560effd640 1 -- 10.20.193.16:0/3660118570 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f56100623e0 con 0x7f562806dc10 2026-02-17T05:56:17.367 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.367+0000 7f560effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:17.368 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.367+0000 7f560effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:17.368 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:56:17.368 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:56:17.371 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 10 librados: watch_flush enter 2026-02-17T05:56:17.371 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 10 librados: watch_flush exit 2026-02-17T05:56:17.371 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 20 client.4312.objecter shutdown clearing up homeless session... 2026-02-17T05:56:17.371 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 10 client.4312.objecter successfully canceled tick 2026-02-17T05:56:17.371 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 1 -- 10.20.193.16:0/3660118570 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55f40790e0 msgr2=0x7f55f407b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:17.371 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55f40790e0 0x7f55f407b5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f561c005110 tx=0x7f561c007f30 comp rx=0 tx=0).stop 2026-02-17T05:56:17.371 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 10 monclient: shutdown 2026-02-17T05:56:17.371 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:17.371 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 1 -- 10.20.193.16:0/3660118570 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806dc10 msgr2=0x7f56281f62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:17.372 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806dc10 0x7f56281f62e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f561000ec10 tx=0x7f561000c520 comp rx=0 tx=0).stop 2026-02-17T05:56:17.372 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 1 -- 10.20.193.16:0/3660118570 shutdown_connections 2026-02-17T05:56:17.372 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55f40790e0 0x7f55f407b5a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:17.372 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f562806dc10 0x7f56281f62e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:17.372 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f562806e3f0 0x7f56281f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:17.372 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 1 --2- 10.20.193.16:0/3660118570 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f562806d830 0x7f562814a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:17.372 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.369+0000 7f5627577640 1 -- 10.20.193.16:0/3660118570 >> 10.20.193.16:0/3660118570 conn(0x7f5628068950 msgr2=0x7f562814e950 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:17.372 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.370+0000 7f5627577640 1 -- 10.20.193.16:0/3660118570 shutdown_connections 2026-02-17T05:56:17.372 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.370+0000 7f5627577640 1 -- 10.20.193.16:0/3660118570 wait complete. 2026-02-17T05:56:17.372 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:17.370+0000 7f5627577640 1 librados: shutdown 2026-02-17T05:56:17.379 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:56:23.195 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:02:02.022 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:23.225 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:02:02.051 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:23.226 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:02:02.054 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:23.227 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:02:02.053 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:23.235 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:02:02.062 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:23.235 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:02:02.061 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:23.256 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:02:02.044 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:23.257 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:02:02.046 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:23.264 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:02:02.053 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:24.993 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:56:25.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.056+0000 7f1534c91640 10 monclient: get_monmap_and_config 2026-02-17T05:56:25.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.056+0000 7f1534c91640 10 monclient: build_initial_monmap 2026-02-17T05:56:25.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.056+0000 7f1534c91640 10 monclient: monmap: 2026-02-17T05:56:25.057 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:25.057 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:56:25.057780+0000 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:56:25.057780+0000 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:25.058 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:25.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 1 Processor -- start 2026-02-17T05:56:25.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 1 -- start start 2026-02-17T05:56:25.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 10 monclient: init 2026-02-17T05:56:25.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:25.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 10 monclient: _add_conns ranks=[1,2,0,3,4,5] 2026-02-17T05:56:25.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f153015c080 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:25.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f15300cba00 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:25.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f153015c460 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:25.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15300c9580 con 0x7f153015c460 2026-02-17T05:56:25.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15300c9860 con 0x7f153015c080 2026-02-17T05:56:25.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15300c9b40 con 0x7f15300cba00 2026-02-17T05:56:25.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:25.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.057+0000 7f1534c91640 10 monclient(hunting): authenticate will time out at 1037.626864s 2026-02-17T05:56:25.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.058+0000 7f152e575640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f153015c080 0x7f15300d21f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:47500/0 (socket says 10.20.193.16:47500) 2026-02-17T05:56:25.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.058+0000 7f152e575640 1 -- 10.20.193.16:0/1692739480 learned_addr learned my addr 10.20.193.16:0/1692739480 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:25.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.059+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 240960151 0 0) 0x7f15300c9580 con 0x7f153015c460 2026-02-17T05:56:25.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.059+0000 7f152d573640 10 monclient(hunting): _negotiate 2026-02-17T05:56:25.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.059+0000 7f152d573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.059+0000 7f152d573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.059+0000 7f152d573640 10 monclient(hunting): my global_id is 4287 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f151c003620 con 0x7f153015c460 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 476762052 0 0) 0x7f15300c9860 con 0x7f153015c080 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 10 monclient(hunting): _negotiate 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 10 monclient(hunting): my global_id is 4315 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15300c9580 con 0x7f153015c080 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3049678571 0 0) 0x7f15300c9b40 con 0x7f15300cba00 2026-02-17T05:56:25.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 10 monclient(hunting): _negotiate 2026-02-17T05:56:25.064 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:25.064 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:25.064 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 10 monclient(hunting): my global_id is 4337 2026-02-17T05:56:25.064 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.060+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15300c9860 con 0x7f15300cba00 2026-02-17T05:56:25.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 <== mon.0 v1:10.20.193.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4247446748 0 0) 0x7f151c003620 con 0x7f153015c460 2026-02-17T05:56:25.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 >> v1:10.20.193.148:6789/0 conn(0x7f15300cba00 legacy=0x7f15300c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:25.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 >> v1:10.20.193.128:6789/0 conn(0x7f153015c080 legacy=0x7f15300d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:25.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 10 monclient: _finish_hunting 0 2026-02-17T05:56:25.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.16:6789/0 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 --> v1:10.20.193.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15300c9e20 con 0x7f153015c460 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 10 monclient: _finish_auth 0 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 10 monclient: _check_auth_tickets 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 <== mon.0 v1:10.20.193.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f1520003280 con 0x7f153015c460 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 10 monclient: dump: 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:25.074 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] but i'm connected to v1:10.20.193.16:6789/0, reconnecting 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 >> v1:10.20.193.16:6789/0 conn(0x7f153015c460 legacy=0x7f153015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.061+0000 7f152d573640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f151c003fe0 0x7f151c004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 10 monclient(hunting): picked mon.a con 0x7f151c003fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f151c004e40 0x7f151c0052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 10 monclient(hunting): picked mon.b con 0x7f151c004e40 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:25.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 0x7f151c00bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 10 monclient(hunting): picked mon.c con 0x7f151c009ab0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f151c00c570 con 0x7f151c003fe0 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f151c00c6f0 con 0x7f151c004e40 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f151c00c8f0 con 0x7f151c009ab0 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.062+0000 7f152d573640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.063+0000 7f152ed76640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 0x7f151c00bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.063+0000 7f152e575640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f151c003fe0 0x7f151c004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.063+0000 7f152e575640 10 monclient(hunting): get_auth_request con 0x7f151c003fe0 auth_method 0 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.063+0000 7f152e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.063+0000 7f152e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:25.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.063+0000 7f152e575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.064+0000 7f152ed76640 10 monclient(hunting): get_auth_request con 0x7f151c009ab0 auth_method 0 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.064+0000 7f152ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.064+0000 7f152ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.064+0000 7f152ed76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.064+0000 7f152e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.064+0000 7f152e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.064+0000 7f152e575640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.064+0000 7f152ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.064+0000 7f152ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.064+0000 7f152ed76640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152dd74640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f151c004e40 0x7f151c0052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152dd74640 10 monclient(hunting): get_auth_request con 0x7f151c004e40 auth_method 0 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152dd74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:25.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152dd74640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 10 monclient(hunting): handle_auth_done global_id 4287 payload 293 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 1 -- 10.20.193.16:0/1692739480 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f151c004e40 msgr2=0x7f151c0052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f151c004e40 0x7f151c0052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 1 -- 10.20.193.16:0/1692739480 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f151c003fe0 msgr2=0x7f151c004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f151c003fe0 0x7f151c004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 1 monclient: found mon.c 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 1 -- 10.20.193.16:0/1692739480 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c003620 con 0x7f151c009ab0 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 10 monclient: _finish_auth 0 2026-02-17T05:56:25.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 10 monclient: _check_auth_tickets 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f152ed76640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f1534c91640 5 monclient: authenticate success, global_id 4287 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.065+0000 7f1534c91640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.066+0000 7f152ed76640 1 -- 10.20.193.16:0/1692739480 reap_dead start 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.066+0000 7f152ed76640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 0x7f151c00bea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f152000a7b0 tx=0x7f152002fb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c9c06bd5c8dacbf server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.067+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520037070 con 0x7f151c009ab0 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.067+0000 7f152d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.068+0000 7f152d573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.068+0000 7f152d573640 10 monclient: dump: 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:25.079 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.068+0000 7f1534c91640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1520033650 con 0x7f151c009ab0 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f152d573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f152d573640 1 -- 10.20.193.16:0/1692739480 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152000a3d0 con 0x7f151c009ab0 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 4 set_mon_vals no callback set 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f152d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f152d573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f152d573640 10 monclient: dump: 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:25.080 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 10 monclient: shutdown 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 1 -- 10.20.193.16:0/1692739480 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 msgr2=0x7f151c00bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 0x7f151c00bea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f152000a7b0 tx=0x7f152002fb60 comp rx=0 tx=0).stop 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 1 -- 10.20.193.16:0/1692739480 shutdown_connections 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 1 --2- 10.20.193.16:0/1692739480 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 0x7f151c00bea0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 1 -- 10.20.193.16:0/1692739480 >> 10.20.193.16:0/1692739480 conn(0x7f15300740d0 msgr2=0x7f15300744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 1 -- 10.20.193.16:0/1692739480 shutdown_connections 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 1 -- 10.20.193.16:0/1692739480 wait complete. 2026-02-17T05:56:25.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 10 monclient: build_initial_monmap 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.069+0000 7f1534c91640 10 monclient: monmap: 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 librados: starting msgr at 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 librados: starting objecter 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 Processor -- start 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 -- start start 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 librados: setting wanted keys 2026-02-17T05:56:25.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 librados: calling monclient init 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient: init 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 0x7f15301f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient(hunting): picked mon.noname-c con 0x7f151c009ab0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f15301f3580 0x7f15301f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient(hunting): picked mon.noname-b con 0x7f15301f3580 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f15301f69c0 0x7f15301f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient(hunting): picked mon.noname-a con 0x7f15301f69c0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15300d21f0 con 0x7f15301f69c0 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f15300ca3e0 con 0x7f15301f3580 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f15300d2370 con 0x7f151c009ab0 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:25.083 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f1534c91640 10 monclient(hunting): authenticate will time out at 1037.639575s 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f152ed76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f15301f69c0 0x7f15301f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f152e575640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 0x7f15301f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.070+0000 7f152dd74640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f15301f3580 0x7f15301f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152e575640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 0x7f15301f3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:41078/0 (socket says 10.20.193.16:41078) 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152e575640 1 -- 10.20.193.16:0/4048746775 learned_addr learned my addr 10.20.193.16:0/4048746775 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152e575640 10 monclient(hunting): get_auth_request con 0x7f151c009ab0 auth_method 0 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152ed76640 10 monclient(hunting): get_auth_request con 0x7f15301f69c0 auth_method 0 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152dd74640 10 monclient(hunting): get_auth_request con 0x7f15301f3580 auth_method 0 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:25.084 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.071+0000 7f152dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.072+0000 7f152ed76640 10 monclient(hunting): handle_auth_done global_id 4290 payload 824 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.072+0000 7f152ed76640 1 -- 10.20.193.16:0/4048746775 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 msgr2=0x7f15301f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.072+0000 7f152ed76640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 0x7f15301f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.072+0000 7f152ed76640 1 -- 10.20.193.16:0/4048746775 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f15301f3580 msgr2=0x7f15301f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.072+0000 7f152ed76640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f15301f3580 0x7f15301f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.072+0000 7f152ed76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.072+0000 7f152ed76640 1 monclient: found mon.noname-a 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.072+0000 7f152ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.072+0000 7f152ed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:25.085 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.072+0000 7f152ed76640 1 -- 10.20.193.16:0/4048746775 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15301f4e20 con 0x7f15301f69c0 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.074+0000 7f152ed76640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f15301f69c0 0x7f15301f3a20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f15300d2900 tx=0x7f15300c9750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.074+0000 7f1516ffd640 10 client.?.objecter ms_handle_connect 0x7f15301f69c0 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.074+0000 7f1516ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 1 -- 10.20.193.16:0/4048746775 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152004c070 con 0x7f15301f69c0 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 10 monclient: dump: 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:25.087 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 1 -- 10.20.193.16:0/4048746775 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1520033800 con 0x7f15301f69c0 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 1 -- 10.20.193.16:0/4048746775 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152000a3d0 con 0x7f15301f69c0 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f152cd72640 4 set_mon_vals no callback set 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1516ffd640 10 monclient: dump: 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:25.088 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1534c91640 5 monclient: authenticate success, global_id 4290 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1534c91640 10 monclient: _renew_subs 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1534c91640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1534c91640 1 -- 10.20.193.16:0/4048746775 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15301f5100 con 0x7f15301f69c0 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1534c91640 10 client.4290.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1534c91640 10 monclient: _renew_subs 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1534c91640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1534c91640 1 -- 10.20.193.16:0/4048746775 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15301f56c0 con 0x7f15301f69c0 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.075+0000 7f1534c91640 1 librados: init done 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.076+0000 7f1534c91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.076+0000 7f1534c91640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.076+0000 7f1534c91640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:25.089 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.076+0000 7f1534c91640 1 -- 10.20.193.16:0/4048746775 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15300c8ce0 con 0x7f15301f69c0 2026-02-17T05:56:25.090 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.088+0000 7f1516ffd640 1 -- 10.20.193.16:0/4048746775 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f1520033bf0 con 0x7f15301f69c0 2026-02-17T05:56:25.090 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.088+0000 7f1516ffd640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f14fc0790e0 0x7f14fc07b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:25.090 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.088+0000 7f152e575640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f14fc0790e0 0x7f14fc07b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:25.090 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.088+0000 7f152e575640 10 monclient: get_auth_request con 0x7f14fc0790e0 auth_method 0 2026-02-17T05:56:25.090 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.089+0000 7f1516ffd640 10 client.4290.objecter ms_handle_connect 0x7f14fc0790e0 2026-02-17T05:56:25.090 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.089+0000 7f152e575640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f14fc0790e0 0x7f14fc07b5a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1524002910 tx=0x7f1524007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:25.091 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.090+0000 7f1516ffd640 1 -- 10.20.193.16:0/4048746775 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f15200bcea0 con 0x7f15301f69c0 2026-02-17T05:56:25.091 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.090+0000 7f1516ffd640 10 client.4290.objecter ms_dispatch2 0x7f1530000b90 osd_map(18..18 src has 1..18) 2026-02-17T05:56:25.091 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.090+0000 7f1516ffd640 3 client.4290.objecter handle_osd_map got epochs [18,18] > 0 2026-02-17T05:56:25.091 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.090+0000 7f1516ffd640 3 client.4290.objecter handle_osd_map decoding full epoch 18 2026-02-17T05:56:25.091 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.090+0000 7f1516ffd640 20 client.4290.objecter dump_active .. 0 homeless 2026-02-17T05:56:25.201 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.201+0000 7f1516ffd640 1 -- 10.20.193.16:0/4048746775 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1520055060 con 0x7f15301f69c0 2026-02-17T05:56:25.202 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.201+0000 7f1516ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:25.202 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.201+0000 7f1516ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:25.291 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.291+0000 7f1534c91640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:25.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.291+0000 7f1534c91640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:25.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.291+0000 7f1534c91640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:25.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.291+0000 7f1534c91640 1 -- 10.20.193.16:0/4048746775 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f153014ba50 con 0x7f15301f69c0 2026-02-17T05:56:25.303 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.302+0000 7f1516ffd640 1 -- 10.20.193.16:0/4048746775 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f15200875c0 con 0x7f15301f69c0 2026-02-17T05:56:25.303 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.302+0000 7f1516ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:25.303 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.302+0000 7f1516ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:25.303 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:56:25.304 INFO:teuthology.misc.health.trial016.stdout:{"epoch":18,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:54:55.671400+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:56:25.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.304+0000 7f1534c91640 10 librados: watch_flush enter 2026-02-17T05:56:25.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.304+0000 7f1534c91640 10 librados: watch_flush exit 2026-02-17T05:56:25.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.304+0000 7f1534c91640 20 client.4290.objecter shutdown clearing up homeless session... 2026-02-17T05:56:25.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.304+0000 7f1534c91640 10 client.4290.objecter successfully canceled tick 2026-02-17T05:56:25.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 -- 10.20.193.16:0/4048746775 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f14fc0790e0 msgr2=0x7f14fc07b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:25.306 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f14fc0790e0 0x7f14fc07b5a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1524002910 tx=0x7f1524007eb0 comp rx=0 tx=0).stop 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 10 monclient: shutdown 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 -- 10.20.193.16:0/4048746775 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f15301f69c0 msgr2=0x7f15301f3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f15301f69c0 0x7f15301f3a20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f15300d2900 tx=0x7f15300c9750 comp rx=0 tx=0).stop 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 -- 10.20.193.16:0/4048746775 shutdown_connections 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f14fc0790e0 0x7f14fc07b5a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f15301f3580 0x7f15301f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f15301f69c0 0x7f15301f3a20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 --2- 10.20.193.16:0/4048746775 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f151c009ab0 0x7f15301f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 -- 10.20.193.16:0/4048746775 >> 10.20.193.16:0/4048746775 conn(0x7f15300740d0 msgr2=0x7f153015b250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 -- 10.20.193.16:0/4048746775 shutdown_connections 2026-02-17T05:56:25.307 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 -- 10.20.193.16:0/4048746775 wait complete. 2026-02-17T05:56:25.308 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:25.305+0000 7f1534c91640 1 librados: shutdown 2026-02-17T05:56:25.313 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-17T05:56:26.256 INFO:tasks.ceph.osd.10.trial148.stderr:2026-02-17T05:56:26.253+0000 77b6940 -1 osd.10 0 log_to_monitors true 2026-02-17T05:56:26.538 INFO:tasks.ceph.osd.2.trial148.stderr:2026-02-17T05:56:26.535+0000 77b6940 -1 osd.2 0 log_to_monitors true 2026-02-17T05:56:26.902 INFO:tasks.ceph.osd.6.trial148.stderr:2026-02-17T05:56:26.899+0000 77b6940 -1 osd.6 0 log_to_monitors true 2026-02-17T05:56:27.060 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:27.057+0000 77b6940 -1 osd.14 0 log_to_monitors true 2026-02-17T05:56:27.871 INFO:tasks.ceph.osd.15.trial164.stderr:2026-02-17T05:56:27.868+0000 77b6940 -1 osd.15 0 log_to_monitors true 2026-02-17T05:56:28.124 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:28.120+0000 77b6940 -1 osd.7 0 log_to_monitors true 2026-02-17T05:56:28.583 INFO:tasks.ceph.osd.11.trial164.stderr:2026-02-17T05:56:28.580+0000 77b6940 -1 osd.11 0 log_to_monitors true 2026-02-17T05:56:28.778 INFO:tasks.ceph.osd.3.trial164.stderr:2026-02-17T05:56:28.775+0000 77b6940 -1 osd.3 0 log_to_monitors true 2026-02-17T05:56:28.791 INFO:tasks.ceph.osd.9.trial128.stderr:2026-02-17T05:56:28.789+0000 77b6940 -1 osd.9 0 log_to_monitors true 2026-02-17T05:56:28.852 INFO:tasks.ceph.osd.10.trial148.stderr:2026-02-17T05:56:28.851+0000 deb0640 -1 osd.10 0 waiting for initial osdmap 2026-02-17T05:56:28.858 INFO:tasks.ceph.osd.1.trial128.stderr:2026-02-17T05:56:28.855+0000 77b6940 -1 osd.1 0 log_to_monitors true 2026-02-17T05:56:28.872 INFO:tasks.ceph.osd.6.trial148.stderr:2026-02-17T05:56:28.870+0000 deb0640 -1 osd.6 0 waiting for initial osdmap 2026-02-17T05:56:28.883 INFO:tasks.ceph.osd.2.trial148.stderr:2026-02-17T05:56:28.880+0000 ce9c640 -1 osd.2 0 waiting for initial osdmap 2026-02-17T05:56:29.001 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:28.995+0000 77b6940 -1 osd.5 0 log_to_monitors true 2026-02-17T05:56:29.077 INFO:tasks.ceph.osd.6.trial148.stderr:2026-02-17T05:56:29.069+0000 11c74640 -1 osd.6 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:29.084 INFO:tasks.ceph.osd.10.trial148.stderr:2026-02-17T05:56:29.077+0000 11c74640 -1 osd.10 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:29.112 INFO:tasks.ceph.osd.2.trial148.stderr:2026-02-17T05:56:29.105+0000 11c74640 -1 osd.2 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:29.190 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:29.188+0000 77b6940 -1 osd.13 0 log_to_monitors true 2026-02-17T05:56:29.846 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:29.843+0000 77b6940 -1 osd.12 0 log_to_monitors true 2026-02-17T05:56:29.955 INFO:tasks.ceph.osd.15.trial164.stderr:2026-02-17T05:56:29.952+0000 deb0640 -1 osd.15 0 waiting for initial osdmap 2026-02-17T05:56:30.175 INFO:tasks.ceph.osd.15.trial164.stderr:2026-02-17T05:56:30.166+0000 11c74640 -1 osd.15 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:30.218 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:30.216+0000 77b6940 -1 osd.0 0 log_to_monitors true 2026-02-17T05:56:30.483 INFO:tasks.ceph.osd.8.trial016.stderr:2026-02-17T05:56:30.481+0000 77b6940 -1 osd.8 0 log_to_monitors true 2026-02-17T05:56:30.519 INFO:tasks.ceph.osd.4.trial016.stderr:2026-02-17T05:56:30.517+0000 77b6940 -1 osd.4 0 log_to_monitors true 2026-02-17T05:56:31.178 INFO:tasks.ceph.osd.11.trial164.stderr:2026-02-17T05:56:31.175+0000 ce9c640 -1 osd.11 0 waiting for initial osdmap 2026-02-17T05:56:31.183 INFO:tasks.ceph.osd.3.trial164.stderr:2026-02-17T05:56:31.180+0000 deb0640 -1 osd.3 0 waiting for initial osdmap 2026-02-17T05:56:31.204 INFO:tasks.ceph.osd.1.trial128.stderr:2026-02-17T05:56:31.201+0000 deb0640 -1 osd.1 0 waiting for initial osdmap 2026-02-17T05:56:31.229 INFO:tasks.ceph.osd.9.trial128.stderr:2026-02-17T05:56:31.227+0000 ce9c640 -1 osd.9 0 waiting for initial osdmap 2026-02-17T05:56:31.426 INFO:tasks.ceph.osd.3.trial164.stderr:2026-02-17T05:56:31.418+0000 11c74640 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:31.437 INFO:tasks.ceph.osd.11.trial164.stderr:2026-02-17T05:56:31.429+0000 11c74640 -1 osd.11 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:31.474 INFO:tasks.ceph.osd.1.trial128.stderr:2026-02-17T05:56:31.466+0000 11c74640 -1 osd.1 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:31.480 INFO:tasks.ceph.osd.9.trial128.stderr:2026-02-17T05:56:31.473+0000 11c74640 -1 osd.9 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:32.192 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.149+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 2 2026-02-17T05:56:32.215 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.186+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.232 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.191+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.234 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.203+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.234 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.214+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.237 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.232+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.245 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.237+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.257 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.245+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.271 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.257+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.272 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.267+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.272 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.271+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.288 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.286+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 2 2026-02-17T05:56:32.321 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.302+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.332 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.321+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.338 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.332+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.386 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.353+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 2 2026-02-17T05:56:32.398 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.386+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.413 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.407+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.414 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.413+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.418 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.418+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 4 2026-02-17T05:56:32.481 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.442+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 2 2026-02-17T05:56:32.481 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.461+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.481 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.475+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.490 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.489+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.505 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.496+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.518 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.506+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 2 2026-02-17T05:56:32.529 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.517+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.555 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.528+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.583 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.555+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.589 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.583+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.591 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.588+0000 77b6940 -1 bdev(0x16f72f20 /var/lib/ceph/osd/ceph-14/block) aio_submit retries 1 2026-02-17T05:56:32.925 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:56:32.979 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:32.976+0000 deb0640 -1 osd.14 0 waiting for initial osdmap 2026-02-17T05:56:32.985 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.985+0000 7f2c8f8bf640 10 monclient: get_monmap_and_config 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.985+0000 7f2c8f8bf640 10 monclient: build_initial_monmap 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.985+0000 7f2c8f8bf640 10 monclient: monmap: 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:56:32.986425+0000 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:56:32.986425+0000 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:32.986 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:32.987 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:32.987 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:32.987 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:32.988 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.985+0000 7f2c8f8bf640 1 Processor -- start 2026-02-17T05:56:32.988 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.985+0000 7f2c8f8bf640 1 -- start start 2026-02-17T05:56:32.988 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.985+0000 7f2c8f8bf640 10 monclient: init 2026-02-17T05:56:32.988 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:32.988 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 10 monclient: _add_conns ranks=[1,2,5,0,3,4] 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f2c8815d3e0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2c8815d7c0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f2c880c52d0 0x7f2c880c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 10 monclient(hunting): picked mon.noname-c con 0x7f2c880c52d0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c880c78a0 con 0x7f2c8815d3e0 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c880c7ee0 con 0x7f2c8815d7c0 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f2c880c8dd0 con 0x7f2c880c52d0 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.986+0000 7f2c8f8bf640 10 monclient(hunting): authenticate will time out at 1045.555473s 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.988+0000 7f2c8d634640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f2c8815d3e0 0x7f2c880c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:60682/0 (socket says 10.20.193.16:60682) 2026-02-17T05:56:32.989 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.988+0000 7f2c8d634640 1 -- 10.20.193.16:0/2214832803 learned_addr learned my addr 10.20.193.16:0/2214832803 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:32.990 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.989+0000 7f2c8de35640 1 --2- 10.20.193.16:0/2214832803 >> v2:10.20.193.148:3300/0 conn(0x7f2c880c52d0 0x7f2c880c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:32.991 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.991+0000 7f2c8de35640 10 monclient(hunting): get_auth_request con 0x7f2c880c52d0 auth_method 0 2026-02-17T05:56:32.991 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.991+0000 7f2c8de35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:32.992 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.991+0000 7f2c8de35640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:32.992 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.991+0000 7f2c8de35640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:32.992 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.992+0000 7f2c8de35640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:32.993 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.992+0000 7f2c8de35640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:32.993 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.992+0000 7f2c8de35640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:32.994 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.994+0000 7f2c8de35640 10 monclient(hunting): handle_auth_done global_id 4388 payload 274 2026-02-17T05:56:32.994 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.994+0000 7f2c8de35640 1 -- 10.20.193.16:0/2214832803 >> v1:10.20.193.148:6789/0 conn(0x7f2c8815d7c0 legacy=0x7f2c880c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:32.994 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.994+0000 7f2c8de35640 1 -- 10.20.193.16:0/2214832803 >> v1:10.20.193.128:6789/0 conn(0x7f2c8815d3e0 legacy=0x7f2c880c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:32.994 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.994+0000 7f2c8de35640 10 monclient: _finish_hunting 0 2026-02-17T05:56:32.994 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.994+0000 7f2c8de35640 1 monclient: found mon.noname-c 2026-02-17T05:56:32.995 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.994+0000 7f2c8de35640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:32.995 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.994+0000 7f2c8de35640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:32.995 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.994+0000 7f2c8de35640 1 -- 10.20.193.16:0/2214832803 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c880c81c0 con 0x7f2c880c52d0 2026-02-17T05:56:32.996 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:32.996+0000 7f2c8de35640 1 --2- 10.20.193.16:0/2214832803 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 0x7f2c880c5710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009d90 tx=0x7f2c8402f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3531f0a7ee99d99 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:33.038 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 1 -- 10.20.193.16:0/2214832803 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8403c070 con 0x7f2c880c52d0 2026-02-17T05:56:33.039 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:33.039 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:33.039 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 10 monclient: dump: 2026-02-17T05:56:33.039 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:33.040 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:33.040 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:33.040 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:33.040 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:33.040 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:33.040 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:33.041 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:33.041 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:33.041 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:33.041 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:33.041 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:33.041 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:33.042 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 1 -- 10.20.193.16:0/2214832803 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c84037440 con 0x7f2c880c52d0 2026-02-17T05:56:33.042 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:33.042 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c8f8bf640 5 monclient: authenticate success, global_id 4388 2026-02-17T05:56:33.042 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 1 -- 10.20.193.16:0/2214832803 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84037810 con 0x7f2c880c52d0 2026-02-17T05:56:33.042 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:33.042 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:33.043 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c77fff640 10 monclient: dump: 2026-02-17T05:56:33.043 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:33.043 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:33.043 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:33.043 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:33.043 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:33.044 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:33.044 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:33.044 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:33.044 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:33.044 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:33.044 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c8f8bf640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:33.045 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c8f8bf640 4 set_mon_vals no callback set 2026-02-17T05:56:33.045 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c8f8bf640 10 monclient: shutdown 2026-02-17T05:56:33.045 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c8f8bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:33.045 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/2214832803 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 msgr2=0x7f2c880c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:33.045 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.029+0000 7f2c8f8bf640 1 --2- 10.20.193.16:0/2214832803 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 0x7f2c880c5710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009d90 tx=0x7f2c8402f230 comp rx=0 tx=0).stop 2026-02-17T05:56:33.045 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/2214832803 shutdown_connections 2026-02-17T05:56:33.046 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 --2- 10.20.193.16:0/2214832803 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 0x7f2c880c5710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:33.046 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/2214832803 >> 10.20.193.16:0/2214832803 conn(0x7f2c880c9f40 msgr2=0x7f2c880ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:33.046 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/2214832803 shutdown_connections 2026-02-17T05:56:33.046 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/2214832803 wait complete. 2026-02-17T05:56:33.046 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 10 monclient: build_initial_monmap 2026-02-17T05:56:33.046 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 10 monclient: monmap: 2026-02-17T05:56:33.046 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:33.047 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:33.047 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:33.047 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:56:33.047 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:33.047 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:33.047 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:33.048 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:33.048 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:33.048 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:33.048 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 librados: starting msgr at 2026-02-17T05:56:33.048 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 librados: starting objecter 2026-02-17T05:56:33.048 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 Processor -- start 2026-02-17T05:56:33.049 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 -- start start 2026-02-17T05:56:33.049 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 librados: setting wanted keys 2026-02-17T05:56:33.049 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 1 librados: calling monclient init 2026-02-17T05:56:33.049 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.030+0000 7f2c8f8bf640 10 monclient: init 2026-02-17T05:56:33.049 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:33.049 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:33.050 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 0x7f2c881f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:33.050 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 10 monclient(hunting): picked mon.noname-c con 0x7f2c880c52d0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:33.050 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2c8815d3e0 0x7f2c881f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:33.050 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 10 monclient(hunting): picked mon.noname-b con 0x7f2c8815d3e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:33.050 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c8815d7c0 0x7f2c880736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:33.051 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 10 monclient(hunting): picked mon.noname-a con 0x7f2c8815d7c0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:33.051 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:33.051 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c880c8a60 con 0x7f2c8815d7c0 2026-02-17T05:56:33.051 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:33.051 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f2c880c8780 con 0x7f2c8815d3e0 2026-02-17T05:56:33.051 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:33.051 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2c880c5c50 con 0x7f2c880c52d0 2026-02-17T05:56:33.052 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:33.052 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8f8bf640 10 monclient(hunting): authenticate will time out at 1045.600223s 2026-02-17T05:56:33.052 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8d634640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 0x7f2c881f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:33.052 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8d634640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 0x7f2c881f7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:41550/0 (socket says 10.20.193.16:41550) 2026-02-17T05:56:33.052 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8d634640 1 -- 10.20.193.16:0/4131160706 learned_addr learned my addr 10.20.193.16:0/4131160706 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:33.052 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8d634640 10 monclient(hunting): get_auth_request con 0x7f2c880c52d0 auth_method 0 2026-02-17T05:56:33.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8d634640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:33.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8d634640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:33.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.031+0000 7f2c8d634640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:33.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.032+0000 7f2c8d634640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:33.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.032+0000 7f2c8d634640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:33.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.032+0000 7f2c8d634640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:33.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.033+0000 7f2c8d634640 10 monclient(hunting): handle_auth_done global_id 4391 payload 824 2026-02-17T05:56:33.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.033+0000 7f2c8d634640 1 -- 10.20.193.16:0/4131160706 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2c8815d3e0 msgr2=0x7f2c881f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:33.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.033+0000 7f2c8d634640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2c8815d3e0 0x7f2c881f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:33.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.033+0000 7f2c8d634640 1 -- 10.20.193.16:0/4131160706 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c8815d7c0 msgr2=0x7f2c880736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:33.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.033+0000 7f2c8d634640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c8815d7c0 0x7f2c880736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:33.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.033+0000 7f2c8d634640 10 monclient: _finish_hunting 0 2026-02-17T05:56:33.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.033+0000 7f2c8d634640 1 monclient: found mon.noname-c 2026-02-17T05:56:33.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.033+0000 7f2c8d634640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:33.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.033+0000 7f2c8d634640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:33.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.033+0000 7f2c8d634640 1 -- 10.20.193.16:0/4131160706 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c881f4a90 con 0x7f2c880c52d0 2026-02-17T05:56:33.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.034+0000 7f2c8d634640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 0x7f2c881f7ae0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c730 tx=0x7f2c7800cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:33.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.034+0000 7f2c75ffb640 10 client.?.objecter ms_handle_connect 0x7f2c880c52d0 2026-02-17T05:56:33.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.034+0000 7f2c75ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:33.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 1 -- 10.20.193.16:0/4131160706 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78017070 con 0x7f2c880c52d0 2026-02-17T05:56:33.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:33.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:33.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 10 monclient: dump: 2026-02-17T05:56:33.056 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:33.057 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:33.057 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:33.057 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:33.057 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:33.057 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:33.057 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:33.057 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:33.058 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:33.058 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:33.058 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:33.058 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:33.058 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:33.058 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 1 -- 10.20.193.16:0/4131160706 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c78005bb0 con 0x7f2c880c52d0 2026-02-17T05:56:33.059 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:33.059 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 1 -- 10.20.193.16:0/4131160706 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78007070 con 0x7f2c880c52d0 2026-02-17T05:56:33.059 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:33.059 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:33.059 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c777fe640 4 set_mon_vals no callback set 2026-02-17T05:56:33.059 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c75ffb640 10 monclient: dump: 2026-02-17T05:56:33.059 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:33.060 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:33.060 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:33.060 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:33.060 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:33.060 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:33.060 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:33.060 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:33.061 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:33.061 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:33.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 5 monclient: authenticate success, global_id 4391 2026-02-17T05:56:33.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 10 monclient: _renew_subs 2026-02-17T05:56:33.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:33.061 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/4131160706 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c881f44d0 con 0x7f2c880c52d0 2026-02-17T05:56:33.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 10 client.4391.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:33.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 10 monclient: _renew_subs 2026-02-17T05:56:33.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:33.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/4131160706 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c88072be0 con 0x7f2c880c52d0 2026-02-17T05:56:33.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 1 librados: init done 2026-02-17T05:56:33.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:33.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:33.062 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:33.063 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.036+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/4131160706 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8815c1f0 con 0x7f2c880c52d0 2026-02-17T05:56:33.065 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.047+0000 7f2c75ffb640 1 -- 10.20.193.16:0/4131160706 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 4) ==== 100001+0+0 (secure 0 0 0) 0x7f2c78007210 con 0x7f2c880c52d0 2026-02-17T05:56:33.065 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.048+0000 7f2c75ffb640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c58079100 0x7f2c5807b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:33.065 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.048+0000 7f2c8ce33640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c58079100 0x7f2c5807b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:33.065 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.048+0000 7f2c8ce33640 10 monclient: get_auth_request con 0x7f2c58079100 auth_method 0 2026-02-17T05:56:33.066 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.048+0000 7f2c75ffb640 10 client.4391.objecter ms_handle_connect 0x7f2c58079100 2026-02-17T05:56:33.066 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.048+0000 7f2c8ce33640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c58079100 0x7f2c5807b5c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c002770 tx=0x7f2c7c007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:33.066 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.049+0000 7f2c75ffb640 1 -- 10.20.193.16:0/4131160706 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6291+0+0 (secure 0 0 0) 0x7f2c7805d6c0 con 0x7f2c880c52d0 2026-02-17T05:56:33.066 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.049+0000 7f2c75ffb640 10 client.4391.objecter ms_dispatch2 0x7f2c88000b90 osd_map(23..23 src has 1..23) 2026-02-17T05:56:33.066 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.049+0000 7f2c75ffb640 3 client.4391.objecter handle_osd_map got epochs [23,23] > 0 2026-02-17T05:56:33.066 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.049+0000 7f2c75ffb640 3 client.4391.objecter handle_osd_map decoding full epoch 23 2026-02-17T05:56:33.066 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.049+0000 7f2c75ffb640 20 client.4391.objecter dump_active .. 0 homeless 2026-02-17T05:56:33.157 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.157+0000 7f2c75ffb640 1 -- 10.20.193.16:0/4131160706 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2c78022020 con 0x7f2c880c52d0 2026-02-17T05:56:33.158 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.157+0000 7f2c75ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:33.158 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.157+0000 7f2c75ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:33.246 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.244+0000 7f2c8f8bf640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:33.246 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.244+0000 7f2c8f8bf640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:33.246 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.244+0000 7f2c8f8bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:33.246 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.244+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/4131160706 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c880c99b0 con 0x7f2c880c52d0 2026-02-17T05:56:33.288 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T05:56:33.280+0000 15c7c640 -1 osd.14 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:33.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.361+0000 7f2c75ffb640 1 -- 10.20.193.16:0/4131160706 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18522 (secure 0 0 0) 0x7f2c78061d80 con 0x7f2c880c52d0 2026-02-17T05:56:33.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.361+0000 7f2c75ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:33.362 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.361+0000 7f2c75ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:33.362 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:56:33.362 INFO:teuthology.misc.health.trial016.stdout:{"epoch":24,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:56:33.051007+0000","last_up_change":"2026-02-17T05:56:31.937975+0000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:56:30.047883+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6807","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6809","nonce":1618049789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6810","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6811","nonce":1618049789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6814","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6815","nonce":1618049789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6812","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6813","nonce":1618049789}]},"public_addr":"10.20.193.128:6809/1618049789","cluster_addr":"10.20.193.128:6811/1618049789","heartbeat_back_addr":"10.20.193.128:6815/1618049789","heartbeat_front_addr":"10.20.193.128:6813/1618049789","state":["exists","up"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6816","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6817","nonce":1588035471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6818","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6819","nonce":1588035471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6822","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6823","nonce":1588035471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6820","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6821","nonce":1588035471}]},"public_addr":"10.20.193.148:6817/1588035471","cluster_addr":"10.20.193.148:6819/1588035471","heartbeat_back_addr":"10.20.193.148:6823/1588035471","heartbeat_front_addr":"10.20.193.148:6821/1588035471","state":["exists","up"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6824","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6825","nonce":3772404125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6826","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6827","nonce":3772404125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6830","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6831","nonce":3772404125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6828","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6829","nonce":3772404125}]},"public_addr":"10.20.193.164:6825/3772404125","cluster_addr":"10.20.193.164:6827/3772404125","heartbeat_back_addr":"10.20.193.164:6831/3772404125","heartbeat_front_addr":"10.20.193.164:6829/3772404125","state":["exists","up"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6824","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6825","nonce":2491860138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6826","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6827","nonce":2491860138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6830","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6831","nonce":2491860138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6828","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6829","nonce":2491860138}]},"public_addr":"10.20.193.148:6825/2491860138","cluster_addr":"10.20.193.148:6827/2491860138","heartbeat_back_addr":"10.20.193.148:6831/2491860138","heartbeat_front_addr":"10.20.193.148:6829/2491860138","state":["exists","up"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6800","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6801","nonce":1813385411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6802","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6803","nonce":1813385411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6806","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6808","nonce":1813385411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6804","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6805","nonce":1813385411}]},"public_addr":"10.20.193.128:6801/1813385411","cluster_addr":"10.20.193.128:6803/1813385411","heartbeat_back_addr":"10.20.193.128:6808/1813385411","heartbeat_front_addr":"10.20.193.128:6805/1813385411","state":["exists","up"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6808","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6809","nonce":2531105113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6810","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6811","nonce":2531105113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6814","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6815","nonce":2531105113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6812","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6813","nonce":2531105113}]},"public_addr":"10.20.193.148:6809/2531105113","cluster_addr":"10.20.193.148:6811/2531105113","heartbeat_back_addr":"10.20.193.148:6815/2531105113","heartbeat_front_addr":"10.20.193.148:6813/2531105113","state":["exists","up"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6808","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6809","nonce":188912883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6810","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6811","nonce":188912883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6814","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6815","nonce":188912883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6812","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6813","nonce":188912883}]},"public_addr":"10.20.193.164:6809/188912883","cluster_addr":"10.20.193.164:6811/188912883","heartbeat_back_addr":"10.20.193.164:6815/188912883","heartbeat_front_addr":"10.20.193.164:6813/188912883","state":["exists","up"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6800","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6801","nonce":1881720827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6802","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6803","nonce":1881720827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6806","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6807","nonce":1881720827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6804","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6805","nonce":1881720827}]},"public_addr":"10.20.193.164:6801/1881720827","cluster_addr":"10.20.193.164:6803/1881720827","heartbeat_back_addr":"10.20.193.164:6807/1881720827","heartbeat_front_addr":"10.20.193.164:6805/1881720827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.248604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:27.916170+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:28.238512+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.224894+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:27.660706+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.933443+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.312258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:56:33.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 10 librados: watch_flush enter 2026-02-17T05:56:33.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 10 librados: watch_flush exit 2026-02-17T05:56:33.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 20 client.4391.objecter shutdown clearing up homeless session... 2026-02-17T05:56:33.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 10 client.4391.objecter successfully canceled tick 2026-02-17T05:56:33.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/4131160706 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c58079100 msgr2=0x7f2c5807b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:33.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c58079100 0x7f2c5807b5c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c002770 tx=0x7f2c7c007ab0 comp rx=0 tx=0).stop 2026-02-17T05:56:33.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 10 monclient: shutdown 2026-02-17T05:56:33.364 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/4131160706 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 msgr2=0x7f2c881f7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 0x7f2c881f7ae0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c730 tx=0x7f2c7800cc00 comp rx=0 tx=0).stop 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/4131160706 shutdown_connections 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c58079100 0x7f2c5807b5c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2c8815d3e0 0x7f2c881f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c8815d7c0 0x7f2c880736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 --2- 10.20.193.16:0/4131160706 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c880c52d0 0x7f2c881f7ae0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/4131160706 >> 10.20.193.16:0/4131160706 conn(0x7f2c880c9f40 msgr2=0x7f2c880ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/4131160706 shutdown_connections 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 -- 10.20.193.16:0/4131160706 wait complete. 2026-02-17T05:56:33.365 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:33.364+0000 7f2c8f8bf640 1 librados: shutdown 2026-02-17T05:56:33.372 DEBUG:teuthology.misc:8 of 16 OSDs are up 2026-02-17T05:56:33.390 INFO:tasks.ceph.osd.4.trial016.stderr:2026-02-17T05:56:33.388+0000 deb0640 -1 osd.4 0 waiting for initial osdmap 2026-02-17T05:56:33.430 INFO:tasks.ceph.osd.8.trial016.stderr:2026-02-17T05:56:33.428+0000 deb0640 -1 osd.8 0 waiting for initial osdmap 2026-02-17T05:56:33.670 INFO:tasks.ceph.osd.4.trial016.stderr:2026-02-17T05:56:33.661+0000 11c74640 -1 osd.4 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:33.725 INFO:tasks.ceph.osd.8.trial016.stderr:2026-02-17T05:56:33.718+0000 11c74640 -1 osd.8 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:34.488 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.466+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.498 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.486+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.498 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.491+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.514 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.490+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.514 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.508+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.521 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.509+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.531 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.513+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.540 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.520+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.542 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.524+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.548 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.538+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.549 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.541+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.552 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.532+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.558 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.546+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.563 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.539+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.567 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.551+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.572 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.557+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.572 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.563+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.578 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.548+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.584 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.566+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.592 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.561+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.592 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.571+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.595 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.570+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.598 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.579+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.601 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.577+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.601 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.583+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.604 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.591+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.607 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.593+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.607 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.599+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.612 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.594+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.615 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.596+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.619 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.605+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.621 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.602+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.623 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.600+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.633 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.613+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.636 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.634+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 3 2026-02-17T05:56:34.646 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.611+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.646 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.639+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.647 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.645+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.647 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.617+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.648 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.641+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.656 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.655+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.659 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.645+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.661 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.654+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.671 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.659+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 2 2026-02-17T05:56:34.674 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.646+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.677 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.670+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.677 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.669+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.683 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.675+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.687 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.658+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.690 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.673+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.694 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.681+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.696 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.684+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.705 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.692+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.707 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.704+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.719 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.705+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.719 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.685+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.719 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.701+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.721 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.704+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 3 2026-02-17T05:56:34.731 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.718+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 3 2026-02-17T05:56:34.737 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.730+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.738 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.720+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.741 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.734+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.745 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.740+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 2 2026-02-17T05:56:34.749 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.744+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 3 2026-02-17T05:56:34.750 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.737+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.756 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.750+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.757 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.744+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.758 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.756+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 3 2026-02-17T05:56:34.764 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.764+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.772 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.755+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.778 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.770+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.780 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.778+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.785 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.770+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.793 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.790+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.799 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.783+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.800 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.766+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.803 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.801+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 2 2026-02-17T05:56:34.815 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.795+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.817 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.797+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.817 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.814+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.822 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.799+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.822 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.816+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.833 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.821+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.837 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.815+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.840 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.821+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.844 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.832+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.845 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.836+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.851 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.839+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.856 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.844+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.859 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.855+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.863 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.850+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.867 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.843+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.868 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.867+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 2 2026-02-17T05:56:34.876 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.874+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.879 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.878+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 2 2026-02-17T05:56:34.890 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.857+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.915 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.889+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.917 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.889+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.923 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.907+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.923 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.916+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.924 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.883+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.928 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.921+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.933 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.889+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.933 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.901+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.933 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.923+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.934 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.927+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.934 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:34.923+0000 77b6940 -1 bdev(0x1ae8efd0 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2026-02-17T05:56:34.943 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.932+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.944 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:34.942+0000 77b6940 -1 bdev(0x16f4ca50 /var/lib/ceph/osd/ceph-13/block) aio_submit retries 1 2026-02-17T05:56:34.969 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.932+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.969 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.959+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:34.980 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.967+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:35.011 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:34.999+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:35.018 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:35.013+0000 77b6940 -1 bdev(0x1a701bb0 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2026-02-17T05:56:35.335 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:35.332+0000 deb0640 -1 osd.7 0 waiting for initial osdmap 2026-02-17T05:56:35.339 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:35.334+0000 deb0640 -1 osd.13 0 waiting for initial osdmap 2026-02-17T05:56:35.424 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:35.419+0000 ce9c640 -1 osd.5 0 waiting for initial osdmap 2026-02-17T05:56:35.566 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T05:56:35.558+0000 15c7c640 -1 osd.7 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:35.578 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T05:56:35.570+0000 15c7c640 -1 osd.13 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:35.601 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.600+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.611 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.581+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 2 2026-02-17T05:56:35.613 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.606+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.630 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.613+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.649 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.614+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.660 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.630+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.660 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.639+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.666 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T05:56:35.658+0000 15c7c640 -1 osd.5 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:35.669 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.648+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.671 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.669+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.675 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.672+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.679 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.675+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.682 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.660+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.694 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.671+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.700 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.683+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.711 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.694+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.711 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.700+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.715 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.712+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 2 2026-02-17T05:56:35.728 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.691+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.744 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.732+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 2 2026-02-17T05:56:35.787 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.744+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.792 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.755+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.800 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.716+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.800 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.754+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 2 2026-02-17T05:56:35.801 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.762+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.806 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.775+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.808 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.801+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 2 2026-02-17T05:56:35.808 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.787+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.815 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.809+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.820 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.815+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.826 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.821+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.831 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.793+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.832 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.801+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.832 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.806+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.832 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.832+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.833 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.826+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.852 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.852+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.885 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.882+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.897 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.885+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.905 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.867+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 4 2026-02-17T05:56:35.906 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.886+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.910 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.904+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.920 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.910+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.922 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.897+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.933 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.905+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.939 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.916+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.946 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.933+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.946 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.940+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.955 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.936+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.955 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.948+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.955 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.955+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.963 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.946+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.969 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:35.969+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:35.979 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.963+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.981 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.980+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:35.988 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.981+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:36.007 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:35.988+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:36.013 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:36.007+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:36.016 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:36.010+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 2 2026-02-17T05:56:36.025 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:36.013+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 1 2026-02-17T05:56:36.027 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:36.016+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:36.038 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:36.027+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:36.055 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:36.038+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:36.062 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:36.048+0000 77b6940 -1 bdev(0x16f4a180 /var/lib/ceph/osd/ceph-12/block) aio_submit retries 4 2026-02-17T05:56:36.074 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:36.065+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:36.075 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:36.071+0000 77b6940 -1 bdev(0x1a704aa0 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2026-02-17T05:56:36.466 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:36.464+0000 deb0640 -1 osd.0 0 waiting for initial osdmap 2026-02-17T05:56:36.475 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:36.473+0000 ce9c640 -1 osd.12 0 waiting for initial osdmap 2026-02-17T05:56:36.708 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T05:56:36.701+0000 15c7c640 -1 osd.0 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:36.721 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T05:56:36.715+0000 15c7c640 -1 osd.12 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:56:40.987 DEBUG:teuthology.orchestra.run.trial016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:56:41.047 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.047+0000 7fddb5711640 10 monclient: get_monmap_and_config 2026-02-17T05:56:41.048 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.047+0000 7fddb5711640 10 monclient: build_initial_monmap 2026-02-17T05:56:41.048 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.047+0000 7fddb5711640 10 monclient: monmap: 2026-02-17T05:56:41.048 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:41.048 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.048 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:56:41.048248+0000 2026-02-17T05:56:41.048 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:56:41.048248+0000 2026-02-17T05:56:41.048 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:41.048 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.049 INFO:teuthology.misc.health.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:41.049 INFO:teuthology.misc.health.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:41.049 INFO:teuthology.misc.health.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:41.049 INFO:teuthology.misc.health.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:41.049 INFO:teuthology.misc.health.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:41.049 INFO:teuthology.misc.health.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:41.049 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:41.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.047+0000 7fddb5711640 1 Processor -- start 2026-02-17T05:56:41.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.047+0000 7fddb5711640 1 -- start start 2026-02-17T05:56:41.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.047+0000 7fddb5711640 10 monclient: init 2026-02-17T05:56:41.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:41.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 10 monclient: _add_conns ranks=[3,2,4,1,5,0] 2026-02-17T05:56:41.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fddb015d3e0 0x7fddb015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 10 monclient(hunting): picked mon.noname-a con 0x7fddb015d3e0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:41.053 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fddb00c9260 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:41.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fddb00c52d0 0x7fddb00c5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 10 monclient(hunting): picked mon.noname-b con 0x7fddb00c52d0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:41.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb00c78a0 con 0x7fddb00c9260 2026-02-17T05:56:41.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fddb00c8dd0 con 0x7fddb015d3e0 2026-02-17T05:56:41.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fddb00c5c80 con 0x7fddb00c52d0 2026-02-17T05:56:41.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:41.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddb5711640 10 monclient(hunting): authenticate will time out at 1053.617312s 2026-02-17T05:56:41.054 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddaeffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fddb015d3e0 0x7fddb015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddae7fc640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fddb00c9260 0x7fddb00c7190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46386/0 (socket says 10.20.193.16:46386) 2026-02-17T05:56:41.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddae7fc640 1 -- 10.20.193.16:0/2261281038 learned_addr learned my addr 10.20.193.16:0/2261281038 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:41.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.048+0000 7fddaf7fe640 1 --2- 10.20.193.16:0/2261281038 >> v2:10.20.193.128:3300/0 conn(0x7fddb00c52d0 0x7fddb00c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaeffd640 10 monclient(hunting): get_auth_request con 0x7fddb015d3e0 auth_method 0 2026-02-17T05:56:41.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaf7fe640 10 monclient(hunting): get_auth_request con 0x7fddb00c52d0 auth_method 0 2026-02-17T05:56:41.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.055 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.049+0000 7fddaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddadffb640 1 -- 10.20.193.16:0/2261281038 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309164123 0 0) 0x7fddb00c78a0 con 0x7fddb00c9260 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddadffb640 10 monclient(hunting): _negotiate 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddadffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.056 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddadffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddadffb640 10 monclient(hunting): my global_id is 4400 2026-02-17T05:56:41.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddadffb640 1 -- 10.20.193.16:0/2261281038 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd9c003620 con 0x7fddb00c9260 2026-02-17T05:56:41.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddaeffd640 10 monclient(hunting): handle_auth_done global_id 4338 payload 274 2026-02-17T05:56:41.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddaeffd640 1 -- 10.20.193.16:0/2261281038 >> v2:10.20.193.128:3300/0 conn(0x7fddb00c52d0 msgr2=0x7fddb00c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:41.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddaeffd640 1 --2- 10.20.193.16:0/2261281038 >> v2:10.20.193.128:3300/0 conn(0x7fddb00c52d0 0x7fddb00c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddaeffd640 1 -- 10.20.193.16:0/2261281038 >> v1:10.20.193.148:6789/0 conn(0x7fddb00c9260 legacy=0x7fddb00c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddaeffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:41.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddaeffd640 1 monclient: found mon.noname-a 2026-02-17T05:56:41.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddaeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:41.057 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddaeffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:41.058 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.050+0000 7fddaeffd640 1 -- 10.20.193.16:0/2261281038 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb00c7ee0 con 0x7fddb015d3e0 2026-02-17T05:56:41.058 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.051+0000 7fddaeffd640 1 --2- 10.20.193.16:0/2261281038 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fddb015d3e0 0x7fddb015d7c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fddb00c4790 tx=0x7fdda402ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67431722ae3d09a3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:41.065 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 1 -- 10.20.193.16:0/2261281038 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda403c070 con 0x7fddb015d3e0 2026-02-17T05:56:41.066 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.066 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:41.066 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 10 monclient: dump: 2026-02-17T05:56:41.066 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:41.066 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.066 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.066 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.066 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.067 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.067 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.067 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.067 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.067 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:41.067 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:41.067 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:41.067 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:41.067 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 1 -- 10.20.193.16:0/2261281038 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda402faf0 con 0x7fddb015d3e0 2026-02-17T05:56:41.068 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 5 monclient: authenticate success, global_id 4338 2026-02-17T05:56:41.068 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:41.068 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 1 -- 10.20.193.16:0/2261281038 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4037440 con 0x7fddb015d3e0 2026-02-17T05:56:41.068 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:41.068 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 4 set_mon_vals no callback set 2026-02-17T05:56:41.068 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.068 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:41.068 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddadffb640 10 monclient: dump: 2026-02-17T05:56:41.068 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:41.068 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.069 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.069 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.069 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.069 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.069 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.069 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.069 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.069 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:41.069 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 10 monclient: shutdown 2026-02-17T05:56:41.069 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:41.070 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 1 -- 10.20.193.16:0/2261281038 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fddb015d3e0 msgr2=0x7fddb015d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.070 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 1 --2- 10.20.193.16:0/2261281038 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fddb015d3e0 0x7fddb015d7c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fddb00c4790 tx=0x7fdda402ede0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.070 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 1 -- 10.20.193.16:0/2261281038 shutdown_connections 2026-02-17T05:56:41.070 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 1 --2- 10.20.193.16:0/2261281038 >> v2:10.20.193.128:3300/0 conn(0x7fddb00c52d0 0x7fddb00c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.070 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 1 --2- 10.20.193.16:0/2261281038 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fddb015d3e0 0x7fddb015d7c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.070 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 1 -- 10.20.193.16:0/2261281038 >> 10.20.193.16:0/2261281038 conn(0x7fddb00c9f40 msgr2=0x7fddb00ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:41.070 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 1 -- 10.20.193.16:0/2261281038 shutdown_connections 2026-02-17T05:56:41.070 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.053+0000 7fddb5711640 1 -- 10.20.193.16:0/2261281038 wait complete. 2026-02-17T05:56:41.070 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.054+0000 7fddb5711640 10 monclient: build_initial_monmap 2026-02-17T05:56:41.070 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.054+0000 7fddb5711640 10 monclient: monmap: 2026-02-17T05:56:41.071 INFO:teuthology.misc.health.trial016.stderr:epoch 0 2026-02-17T05:56:41.071 INFO:teuthology.misc.health.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:41.071 INFO:teuthology.misc.health.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:41.071 INFO:teuthology.misc.health.trial016.stderr:created 0.000000 2026-02-17T05:56:41.071 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:41.071 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.071 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:41.071 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:41.071 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:41.071 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:41.072 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.054+0000 7fddb5711640 1 librados: starting msgr at 2026-02-17T05:56:41.072 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.054+0000 7fddb5711640 1 librados: starting objecter 2026-02-17T05:56:41.072 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.054+0000 7fddb5711640 1 Processor -- start 2026-02-17T05:56:41.072 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.054+0000 7fddb5711640 1 -- start start 2026-02-17T05:56:41.072 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.054+0000 7fddb5711640 1 librados: setting wanted keys 2026-02-17T05:56:41.072 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.054+0000 7fddb5711640 1 librados: calling monclient init 2026-02-17T05:56:41.072 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.054+0000 7fddb5711640 10 monclient: init 2026-02-17T05:56:41.072 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:41.072 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:41.072 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fddb00c52d0 0x7fddb0143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 10 monclient(hunting): picked mon.noname-b con 0x7fddb00c52d0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fddb00c9260 0x7fddb0072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 10 monclient(hunting): picked mon.noname-c con 0x7fddb00c9260 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fddb015d3e0 0x7fddb0145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 10 monclient(hunting): picked mon.noname-a con 0x7fddb015d3e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddb00c8780 con 0x7fddb015d3e0 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fddb00c84a0 con 0x7fddb00c52d0 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fddb00c8a60 con 0x7fddb00c9260 2026-02-17T05:56:41.073 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:41.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddb5711640 10 monclient(hunting): authenticate will time out at 1053.624075s 2026-02-17T05:56:41.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddae7fc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fddb00c9260 0x7fddb0072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaeffd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fddb00c52d0 0x7fddb0143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddae7fc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fddb00c9260 0x7fddb0072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56688/0 (socket says 10.20.193.16:56688) 2026-02-17T05:56:41.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddae7fc640 1 -- 10.20.193.16:0/2152362558 learned_addr learned my addr 10.20.193.16:0/2152362558 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:41.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddae7fc640 10 monclient(hunting): get_auth_request con 0x7fddb00c9260 auth_method 0 2026-02-17T05:56:41.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddae7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddae7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.074 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaeffd640 10 monclient(hunting): get_auth_request con 0x7fddb00c52d0 auth_method 0 2026-02-17T05:56:41.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaf7fe640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fddb015d3e0 0x7fddb0145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaf7fe640 10 monclient(hunting): get_auth_request con 0x7fddb015d3e0 auth_method 0 2026-02-17T05:56:41.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.075 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.055+0000 7fddaeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.056+0000 7fddaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.056+0000 7fddaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.056+0000 7fddaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.057+0000 7fddae7fc640 10 monclient(hunting): handle_auth_done global_id 4403 payload 824 2026-02-17T05:56:41.076 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.057+0000 7fddae7fc640 1 -- 10.20.193.16:0/2152362558 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fddb00c52d0 msgr2=0x7fddb0143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.057+0000 7fddae7fc640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fddb00c52d0 0x7fddb0143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.057+0000 7fddae7fc640 1 -- 10.20.193.16:0/2152362558 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fddb015d3e0 msgr2=0x7fddb0145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.057+0000 7fddae7fc640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fddb015d3e0 0x7fddb0145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.057+0000 7fddae7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:56:41.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.057+0000 7fddae7fc640 1 monclient: found mon.noname-c 2026-02-17T05:56:41.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.057+0000 7fddae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:41.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.057+0000 7fddae7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.057+0000 7fddae7fc640 1 -- 10.20.193.16:0/2152362558 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb0073b90 con 0x7fddb00c9260 2026-02-17T05:56:41.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.058+0000 7fddae7fc640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fddb00c9260 0x7fddb0072660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c730 tx=0x7fdd9800cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:41.077 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.058+0000 7fdd8f7fe640 10 client.?.objecter ms_handle_connect 0x7fddb00c9260 2026-02-17T05:56:41.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.058+0000 7fdd8f7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:41.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fdd8f7fe640 1 -- 10.20.193.16:0/2152362558 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98017070 con 0x7fddb00c9260 2026-02-17T05:56:41.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fdd8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fdd8f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:41.078 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fdd8f7fe640 10 monclient: dump: 2026-02-17T05:56:41.078 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:41.078 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.078 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.078 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.078 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.079 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.079 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.079 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.079 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.079 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:41.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fdd8f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:41.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fdd8f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:41.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fdd8f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:41.079 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fddb5711640 5 monclient: authenticate success, global_id 4403 2026-02-17T05:56:41.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fddb5711640 10 monclient: _renew_subs 2026-02-17T05:56:41.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fddb5711640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fddb5711640 1 -- 10.20.193.16:0/2152362558 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb0073e30 con 0x7fddb00c9260 2026-02-17T05:56:41.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fddb5711640 10 client.4403.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:41.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fddb5711640 10 monclient: _renew_subs 2026-02-17T05:56:41.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fddb5711640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fddb5711640 1 -- 10.20.193.16:0/2152362558 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb00743f0 con 0x7fddb00c9260 2026-02-17T05:56:41.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.059+0000 7fddb5711640 1 librados: init done 2026-02-17T05:56:41.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fdd8f7fe640 1 -- 10.20.193.16:0/2152362558 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd98005bb0 con 0x7fddb00c9260 2026-02-17T05:56:41.080 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fdd8f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fdd8f7fe640 1 -- 10.20.193.16:0/2152362558 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98021060 con 0x7fddb00c9260 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fdd8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fdd8f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fddad7fa640 4 set_mon_vals no callback set 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fdd8f7fe640 10 monclient: dump: 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:epoch 1 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.081 INFO:teuthology.misc.health.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.082 INFO:teuthology.misc.health.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.082 INFO:teuthology.misc.health.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.082 INFO:teuthology.misc.health.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.082 INFO:teuthology.misc.health.trial016.stderr: 2026-02-17T05:56:41.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fddb5711640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:41.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fddb5711640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:41.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fddb5711640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.082 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.060+0000 7fddb5711640 1 -- 10.20.193.16:0/2152362558 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb00c9660 con 0x7fddb00c9260 2026-02-17T05:56:41.169 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.169+0000 7fdd8f7fe640 1 -- 10.20.193.16:0/2152362558 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fdd98024080 con 0x7fddb00c9260 2026-02-17T05:56:41.170 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.170+0000 7fdd8f7fe640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd7c079140 0x7fdd7c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.171 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.170+0000 7fddaeffd640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd7c079140 0x7fdd7c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.171 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.170+0000 7fddaeffd640 10 monclient: get_auth_request con 0x7fdd7c079140 auth_method 0 2026-02-17T05:56:41.171 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.170+0000 7fddaeffd640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd7c079140 0x7fdd7c07b600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdda4039000 tx=0x7fdda4033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:41.171 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.170+0000 7fdd8f7fe640 10 client.4403.objecter ms_handle_connect 0x7fdd7c079140 2026-02-17T05:56:41.172 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.172+0000 7fdd8f7fe640 1 -- 10.20.193.16:0/2152362558 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8563+0+0 (secure 0 0 0) 0x7fdd9805d7f0 con 0x7fddb00c9260 2026-02-17T05:56:41.172 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.172+0000 7fdd8f7fe640 10 client.4403.objecter ms_dispatch2 0x7fddb0000b90 osd_map(29..29 src has 1..29) 2026-02-17T05:56:41.172 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.172+0000 7fdd8f7fe640 3 client.4403.objecter handle_osd_map got epochs [29,29] > 0 2026-02-17T05:56:41.172 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.172+0000 7fdd8f7fe640 3 client.4403.objecter handle_osd_map decoding full epoch 29 2026-02-17T05:56:41.172 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.172+0000 7fdd8f7fe640 20 client.4403.objecter dump_active .. 0 homeless 2026-02-17T05:56:41.178 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.178+0000 7fdd8f7fe640 1 -- 10.20.193.16:0/2152362558 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdd980617b0 con 0x7fddb00c9260 2026-02-17T05:56:41.178 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.178+0000 7fdd8f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:41.178 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.178+0000 7fdd8f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:41.262 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.261+0000 7fddb5711640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:41.262 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.261+0000 7fddb5711640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:41.262 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.261+0000 7fddb5711640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.263 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.261+0000 7fddb5711640 1 -- 10.20.193.16:0/2152362558 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddb00c99b0 con 0x7fddb00c9260 2026-02-17T05:56:41.291 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.290+0000 7fdd8f7fe640 1 -- 10.20.193.16:0/2152362558 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+22814 (secure 0 0 0) 0x7fdd98065ab0 con 0x7fddb00c9260 2026-02-17T05:56:41.291 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.290+0000 7fdd8f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:41.291 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.290+0000 7fdd8f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:41.291 INFO:teuthology.misc.health.trial016.stdout: 2026-02-17T05:56:41.291 INFO:teuthology.misc.health.trial016.stdout:{"epoch":29,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:56:38.632172+0000","last_up_change":"2026-02-17T05:56:37.615172+0000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:56:30.047883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6826","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6827","nonce":821011033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6828","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6829","nonce":821011033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6832","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6833","nonce":821011033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6830","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6831","nonce":821011033}]},"public_addr":"10.20.193.16:6827/821011033","cluster_addr":"10.20.193.16:6829/821011033","heartbeat_back_addr":"10.20.193.16:6833/821011033","heartbeat_front_addr":"10.20.193.16:6831/821011033","state":["exists","up"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6807","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6809","nonce":1618049789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6810","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6811","nonce":1618049789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6814","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6815","nonce":1618049789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6812","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6813","nonce":1618049789}]},"public_addr":"10.20.193.128:6809/1618049789","cluster_addr":"10.20.193.128:6811/1618049789","heartbeat_back_addr":"10.20.193.128:6815/1618049789","heartbeat_front_addr":"10.20.193.128:6813/1618049789","state":["exists","up"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6816","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6817","nonce":1588035471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6818","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6819","nonce":1588035471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6822","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6823","nonce":1588035471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6820","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6821","nonce":1588035471}]},"public_addr":"10.20.193.148:6817/1588035471","cluster_addr":"10.20.193.148:6819/1588035471","heartbeat_back_addr":"10.20.193.148:6823/1588035471","heartbeat_front_addr":"10.20.193.148:6821/1588035471","state":["exists","up"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6824","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6825","nonce":3772404125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6826","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6827","nonce":3772404125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6830","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6831","nonce":3772404125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6828","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6829","nonce":3772404125}]},"public_addr":"10.20.193.164:6825/3772404125","cluster_addr":"10.20.193.164:6827/3772404125","heartbeat_back_addr":"10.20.193.164:6831/3772404125","heartbeat_front_addr":"10.20.193.164:6829/3772404125","state":["exists","up"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6818","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6819","nonce":1157611076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6820","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6821","nonce":1157611076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6824","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6825","nonce":1157611076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6822","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6823","nonce":1157611076}]},"public_addr":"10.20.193.16:6819/1157611076","cluster_addr":"10.20.193.16:6821/1157611076","heartbeat_back_addr":"10.20.193.16:6825/1157611076","heartbeat_front_addr":"10.20.193.16:6823/1157611076","state":["exists","up"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6824","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6825","nonce":1593443319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6826","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6827","nonce":1593443319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6830","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6831","nonce":1593443319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6828","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6829","nonce":1593443319}]},"public_addr":"10.20.193.128:6825/1593443319","cluster_addr":"10.20.193.128:6827/1593443319","heartbeat_back_addr":"10.20.193.128:6831/1593443319","heartbeat_front_addr":"10.20.193.128:6829/1593443319","state":["exists","up"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6824","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6825","nonce":2491860138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6826","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6827","nonce":2491860138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6830","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6831","nonce":2491860138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6828","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6829","nonce":2491860138}]},"public_addr":"10.20.193.148:6825/2491860138","cluster_addr":"10.20.193.148:6827/2491860138","heartbeat_back_addr":"10.20.193.148:6831/2491860138","heartbeat_front_addr":"10.20.193.148:6829/2491860138","state":["exists","up"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6816","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6817","nonce":3942637745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6818","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6819","nonce":3942637745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6822","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6823","nonce":3942637745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6820","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6821","nonce":3942637745}]},"public_addr":"10.20.193.164:6817/3942637745","cluster_addr":"10.20.193.164:6819/3942637745","heartbeat_back_addr":"10.20.193.164:6823/3942637745","heartbeat_front_addr":"10.20.193.164:6821/3942637745","state":["exists","up"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6802","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6803","nonce":15364298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6804","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6805","nonce":15364298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6808","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6809","nonce":15364298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6806","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6807","nonce":15364298}]},"public_addr":"10.20.193.16:6803/15364298","cluster_addr":"10.20.193.16:6805/15364298","heartbeat_back_addr":"10.20.193.16:6809/15364298","heartbeat_front_addr":"10.20.193.16:6807/15364298","state":["exists","up"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6800","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6801","nonce":1813385411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6802","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6803","nonce":1813385411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6806","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6808","nonce":1813385411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6804","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6805","nonce":1813385411}]},"public_addr":"10.20.193.128:6801/1813385411","cluster_addr":"10.20.193.128:6803/1813385411","heartbeat_back_addr":"10.20.193.128:6808/1813385411","heartbeat_front_addr":"10.20.193.128:6805/1813385411","state":["exists","up"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6808","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6809","nonce":2531105113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6810","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6811","nonce":2531105113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6814","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6815","nonce":2531105113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6812","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6813","nonce":2531105113}]},"public_addr":"10.20.193.148:6809/2531105113","cluster_addr":"10.20.193.148:6811/2531105113","heartbeat_back_addr":"10.20.193.148:6815/2531105113","heartbeat_front_addr":"10.20.193.148:6813/2531105113","state":["exists","up"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6808","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6809","nonce":188912883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6810","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6811","nonce":188912883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6814","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6815","nonce":188912883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6812","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6813","nonce":188912883}]},"public_addr":"10.20.193.164:6809/188912883","cluster_addr":"10.20.193.164:6811/188912883","heartbeat_back_addr":"10.20.193.164:6815/188912883","heartbeat_front_addr":"10.20.193.164:6813/188912883","state":["exists","up"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6810","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6811","nonce":3960967243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6812","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6813","nonce":3960967243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6816","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6817","nonce":3960967243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6814","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6815","nonce":3960967243}]},"public_addr":"10.20.193.16:6811/3960967243","cluster_addr":"10.20.193.16:6813/3960967243","heartbeat_back_addr":"10.20.193.16:6817/3960967243","heartbeat_front_addr":"10.20.193.16:6815/3960967243","state":["exists","up"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6816","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6817","nonce":1434515412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6818","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6819","nonce":1434515412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6822","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6823","nonce":1434515412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6820","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6821","nonce":1434515412}]},"public_addr":"10.20.193.128:6817/1434515412","cluster_addr":"10.20.193.128:6819/1434515412","heartbeat_back_addr":"10.20.193.128:6823/1434515412","heartbeat_front_addr":"10.20.193.128:6821/1434515412","state":["exists","up"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6800","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6801","nonce":1855767579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6802","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6803","nonce":1855767579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6806","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6807","nonce":1855767579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6804","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6805","nonce":1855767579}]},"public_addr":"10.20.193.148:6801/1855767579","cluster_addr":"10.20.193.148:6803/1855767579","heartbeat_back_addr":"10.20.193.148:6807/1855767579","heartbeat_front_addr":"10.20.193.148:6805/1855767579","state":["exists","up"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6800","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6801","nonce":1881720827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6802","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6803","nonce":1881720827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6806","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6807","nonce":1881720827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6804","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6805","nonce":1881720827}]},"public_addr":"10.20.193.164:6801/1881720827","cluster_addr":"10.20.193.164:6803/1881720827","heartbeat_back_addr":"10.20.193.164:6807/1881720827","heartbeat_front_addr":"10.20.193.164:6805/1881720827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.650447+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.248604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:27.916170+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.151671+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.881661+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.409956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:28.238512+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.558291+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.920088+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.224894+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:27.660706+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.933443+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.293415+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.603293+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:28.448673+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.312258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 10 librados: watch_flush enter 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 10 librados: watch_flush exit 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 20 client.4403.objecter shutdown clearing up homeless session... 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 10 client.4403.objecter successfully canceled tick 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 -- 10.20.193.16:0/2152362558 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd7c079140 msgr2=0x7fdd7c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd7c079140 0x7fdd7c07b600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdda4039000 tx=0x7fdda4033000 comp rx=0 tx=0).stop 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 10 monclient: shutdown 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 -- 10.20.193.16:0/2152362558 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fddb00c9260 msgr2=0x7fddb0072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fddb00c9260 0x7fddb0072660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c730 tx=0x7fdd9800cc00 comp rx=0 tx=0).stop 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 -- 10.20.193.16:0/2152362558 shutdown_connections 2026-02-17T05:56:41.292 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd7c079140 0x7fdd7c07b600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.293 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fddb015d3e0 0x7fddb0145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.293 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fddb00c9260 0x7fddb0072660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.293 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 --2- 10.20.193.16:0/2152362558 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fddb00c52d0 0x7fddb0143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.293 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 -- 10.20.193.16:0/2152362558 >> 10.20.193.16:0/2152362558 conn(0x7fddb00c9f40 msgr2=0x7fddb00cb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:41.293 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 -- 10.20.193.16:0/2152362558 shutdown_connections 2026-02-17T05:56:41.293 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 -- 10.20.193.16:0/2152362558 wait complete. 2026-02-17T05:56:41.293 INFO:teuthology.misc.health.trial016.stderr:2026-02-17T05:56:41.292+0000 7fddb5711640 1 librados: shutdown 2026-02-17T05:56:41.300 DEBUG:teuthology.misc:16 of 16 OSDs are up 2026-02-17T05:56:41.300 INFO:tasks.ceph:Creating RBD pool 2026-02-17T05:56:41.300 DEBUG:teuthology.orchestra.run.trial016:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-17T05:56:41.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.364+0000 7f6d60d87640 10 monclient: get_monmap_and_config 2026-02-17T05:56:41.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.364+0000 7f6d60d87640 10 monclient: build_initial_monmap 2026-02-17T05:56:41.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.364+0000 7f6d60d87640 10 monclient: monmap: 2026-02-17T05:56:41.364 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:41.365641+0000 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:41.365641+0000 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 1 Processor -- start 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 1 -- start start 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 10 monclient: init 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 10 monclient: _add_conns ranks=[1,0,5,2,4,3] 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f6d5c14f020 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6d5c14f400 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:41.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6d5c15caf0 0x7f6d5c0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 10 monclient(hunting): picked mon.noname-c con 0x7f6d5c15caf0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5c0c5b10 con 0x7f6d5c14f400 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5c06e620 con 0x7f6d5c14f020 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5c06e900 con 0x7f6d5c15caf0 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.365+0000 7f6d60d87640 10 monclient(hunting): authenticate will time out at 1053.934711s 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.366+0000 7f6d59d74640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f6d5c14f400 0x7f6d5c15c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:55984/0 (socket says 10.20.193.16:55984) 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.366+0000 7f6d59d74640 1 -- 10.20.193.16:0/3641688668 learned_addr learned my addr 10.20.193.16:0/3641688668 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.366+0000 7f6d5ad76640 1 --2- 10.20.193.16:0/3641688668 >> v2:10.20.193.148:3300/0 conn(0x7f6d5c15caf0 0x7f6d5c0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.366+0000 7f6d5ad76640 10 monclient(hunting): get_auth_request con 0x7f6d5c15caf0 auth_method 0 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.366+0000 7f6d5ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.366+0000 7f6d5ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.366+0000 7f6d5ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.366+0000 7f6d5ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.366+0000 7f6d5ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.366+0000 7f6d5ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d59573640 1 -- 10.20.193.16:0/3641688668 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1222833705 0 0) 0x7f6d5c0c5b10 con 0x7f6d5c14f400 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d59573640 10 monclient(hunting): _negotiate 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d59573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d59573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d59573640 10 monclient(hunting): my global_id is 4344 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d59573640 1 -- 10.20.193.16:0/3641688668 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d48003620 con 0x7f6d5c14f400 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d5ad76640 10 monclient(hunting): handle_auth_done global_id 4406 payload 274 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d5ad76640 1 -- 10.20.193.16:0/3641688668 >> v1:10.20.193.128:6789/0 conn(0x7f6d5c14f020 legacy=0x7f6d5c06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d5ad76640 1 -- 10.20.193.16:0/3641688668 >> v1:10.20.193.16:6789/0 conn(0x7f6d5c14f400 legacy=0x7f6d5c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d5ad76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:41.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d5ad76640 1 monclient: found mon.noname-c 2026-02-17T05:56:41.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d5ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:41.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d5ad76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.367+0000 7f6d5ad76640 1 -- 10.20.193.16:0/3641688668 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c06ebe0 con 0x7f6d5c15caf0 2026-02-17T05:56:41.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.368+0000 7f6d5ad76640 1 --2- 10.20.193.16:0/3641688668 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5c15caf0 0x7f6d5c0c6010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009d90 tx=0x7f6d4c02f2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e88a16d7e19ee31a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:41.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.369+0000 7f6d59573640 1 -- 10.20.193.16:0/3641688668 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c03c070 con 0x7f6d5c15caf0 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.369+0000 7f6d59573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.369+0000 7f6d59573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.369+0000 7f6d59573640 10 monclient: dump: 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.369+0000 7f6d59573640 10 monclient: _finish_auth 0 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.369+0000 7f6d59573640 10 monclient: _check_auth_tickets 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.369+0000 7f6d59573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.369+0000 7f6d60d87640 5 monclient: authenticate success, global_id 4406 2026-02-17T05:56:41.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.369+0000 7f6d60d87640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d59573640 1 -- 10.20.193.16:0/3641688668 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4c037440 con 0x7f6d5c15caf0 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d59573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d59573640 1 -- 10.20.193.16:0/3641688668 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0377f0 con 0x7f6d5c15caf0 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d59573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d59573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d59573640 10 monclient: dump: 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 4 set_mon_vals no callback set 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 10 monclient: shutdown 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 1 -- 10.20.193.16:0/3641688668 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5c15caf0 msgr2=0x7f6d5c0c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 1 --2- 10.20.193.16:0/3641688668 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5c15caf0 0x7f6d5c0c6010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009d90 tx=0x7f6d4c02f2c0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 1 -- 10.20.193.16:0/3641688668 shutdown_connections 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 1 --2- 10.20.193.16:0/3641688668 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5c15caf0 0x7f6d5c0c6010 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 1 -- 10.20.193.16:0/3641688668 >> 10.20.193.16:0/3641688668 conn(0x7f6d5c0cd770 msgr2=0x7f6d5c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 1 -- 10.20.193.16:0/3641688668 shutdown_connections 2026-02-17T05:56:41.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.370+0000 7f6d60d87640 1 -- 10.20.193.16:0/3641688668 wait complete. 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient: build_initial_monmap 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient: monmap: 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:41.372 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 librados: starting msgr at 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 librados: starting objecter 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 Processor -- start 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 -- start start 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 librados: setting wanted keys 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 librados: calling monclient init 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient: init 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5c14f020 0x7f6d5c1f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient(hunting): picked mon.noname-c con 0x7f6d5c14f020 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5c14f400 0x7f6d5c070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient(hunting): picked mon.noname-a con 0x7f6d5c14f400 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5c15caf0 0x7f6d5c0737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient(hunting): picked mon.noname-b con 0x7f6d5c15caf0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c06e2f0 con 0x7f6d5c14f400 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c06f1a0 con 0x7f6d5c15caf0 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c0c6550 con 0x7f6d5c14f020 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.371+0000 7f6d60d87640 10 monclient(hunting): authenticate will time out at 1053.940954s 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d59d74640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5c14f400 0x7f6d5c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5a575640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5c14f020 0x7f6d5c1f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5ad76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5c15caf0 0x7f6d5c0737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d59d74640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5c14f400 0x7f6d5c070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59462/0 (socket says 10.20.193.16:59462) 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d59d74640 1 -- 10.20.193.16:0/538755379 learned_addr learned my addr 10.20.193.16:0/538755379 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d59d74640 10 monclient(hunting): get_auth_request con 0x7f6d5c14f400 auth_method 0 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d59d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d59d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d59d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5a575640 10 monclient(hunting): get_auth_request con 0x7f6d5c14f020 auth_method 0 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5ad76640 10 monclient(hunting): get_auth_request con 0x7f6d5c15caf0 auth_method 0 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d59d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d59d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d59d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.372+0000 7f6d5ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.374+0000 7f6d5ad76640 10 monclient(hunting): handle_auth_done global_id 4378 payload 824 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.374+0000 7f6d5ad76640 1 -- 10.20.193.16:0/538755379 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5c14f020 msgr2=0x7f6d5c1f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.374+0000 7f6d5ad76640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5c14f020 0x7f6d5c1f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.374+0000 7f6d5ad76640 1 -- 10.20.193.16:0/538755379 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5c14f400 msgr2=0x7f6d5c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.374+0000 7f6d5ad76640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5c14f400 0x7f6d5c070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.374+0000 7f6d5ad76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:41.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.374+0000 7f6d5ad76640 1 monclient: found mon.noname-b 2026-02-17T05:56:41.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.374+0000 7f6d5ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:41.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.374+0000 7f6d5ad76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:41.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.374+0000 7f6d5ad76640 1 -- 10.20.193.16:0/538755379 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c072840 con 0x7f6d5c15caf0 2026-02-17T05:56:41.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.375+0000 7f6d5ad76640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5c15caf0 0x7f6d5c0737a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0c5d00 tx=0x7f6d4c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:41.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.375+0000 7f6d42ffd640 10 client.?.objecter ms_handle_connect 0x7f6d5c15caf0 2026-02-17T05:56:41.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.375+0000 7f6d42ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:41.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d42ffd640 1 -- 10.20.193.16:0/538755379 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c03c050 con 0x7f6d5c15caf0 2026-02-17T05:56:41.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d42ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d42ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d42ffd640 10 monclient: dump: 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d42ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d42ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 5 monclient: authenticate success, global_id 4378 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 10 monclient: _renew_subs 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 1 -- 10.20.193.16:0/538755379 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c072b20 con 0x7f6d5c15caf0 2026-02-17T05:56:41.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 10 client.4378.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 10 monclient: _renew_subs 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 1 -- 10.20.193.16:0/538755379 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c0730e0 con 0x7f6d5c15caf0 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 1 librados: init done 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.376+0000 7f6d60d87640 1 -- 10.20.193.16:0/538755379 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c0c6930 con 0x7f6d5c15caf0 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.377+0000 7f6d42ffd640 1 -- 10.20.193.16:0/538755379 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4c048070 con 0x7f6d5c15caf0 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.377+0000 7f6d42ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.377+0000 7f6d42ffd640 1 -- 10.20.193.16:0/538755379 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c004240 con 0x7f6d5c15caf0 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.377+0000 7f6d42ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.377+0000 7f6d42ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.377+0000 7f6d58d72640 4 set_mon_vals no callback set 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.377+0000 7f6d42ffd640 10 monclient: dump: 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.379 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.380 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.380 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.485+0000 7f6d42ffd640 1 -- 10.20.193.16:0/538755379 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f6d4c0043e0 con 0x7f6d5c15caf0 2026-02-17T05:56:41.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.486+0000 7f6d42ffd640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079140 0x7f6d2807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.486+0000 7f6d5a575640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079140 0x7f6d2807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.486+0000 7f6d5a575640 10 monclient: get_auth_request con 0x7f6d28079140 auth_method 0 2026-02-17T05:56:41.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.486+0000 7f6d42ffd640 10 client.4378.objecter ms_handle_connect 0x7f6d28079140 2026-02-17T05:56:41.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.486+0000 7f6d5a575640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079140 0x7f6d2807b600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6d50002770 tx=0x7f6d50007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:41.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.488+0000 7f6d42ffd640 1 -- 10.20.193.16:0/538755379 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8563+0+0 (secure 0 0 0) 0x7f6d4c084380 con 0x7f6d5c15caf0 2026-02-17T05:56:41.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.488+0000 7f6d42ffd640 10 client.4378.objecter ms_dispatch2 0x7f6d5c000b90 osd_map(29..29 src has 1..29) 2026-02-17T05:56:41.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.488+0000 7f6d42ffd640 3 client.4378.objecter handle_osd_map got epochs [29,29] > 0 2026-02-17T05:56:41.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.488+0000 7f6d42ffd640 3 client.4378.objecter handle_osd_map decoding full epoch 29 2026-02-17T05:56:41.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.488+0000 7f6d42ffd640 20 client.4378.objecter dump_active .. 0 homeless 2026-02-17T05:56:41.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.494+0000 7f6d42ffd640 1 -- 10.20.193.16:0/538755379 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6d4c088340 con 0x7f6d5c15caf0 2026-02-17T05:56:41.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.494+0000 7f6d42ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:41.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.494+0000 7f6d42ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:41.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.576+0000 7f6d60d87640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-17T05:56:41.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.576+0000 7f6d60d87640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-17T05:56:41.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.576+0000 7f6d60d87640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:41.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.576+0000 7f6d60d87640 1 -- 10.20.193.16:0/538755379 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f6d5c0c8010 con 0x7f6d5c15caf0 2026-02-17T05:56:41.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.858+0000 7f6d42ffd640 1 -- 10.20.193.16:0/538755379 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v30) ==== 109+0+0 (secure 0 0 0) 0x7f6d4c08c640 con 0x7f6d5c15caf0 2026-02-17T05:56:41.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.858+0000 7f6d42ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-17T05:56:41.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.858+0000 7f6d42ffd640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2026-02-17T05:56:41.860 INFO:teuthology.orchestra.run.trial016.stderr:pool 'rbd' created 2026-02-17T05:56:41.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 10 librados: watch_flush enter 2026-02-17T05:56:41.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 10 librados: watch_flush exit 2026-02-17T05:56:41.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 20 client.4378.objecter shutdown clearing up homeless session... 2026-02-17T05:56:41.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 10 client.4378.objecter successfully canceled tick 2026-02-17T05:56:41.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 -- 10.20.193.16:0/538755379 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079140 msgr2=0x7f6d2807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079140 0x7f6d2807b600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6d50002770 tx=0x7f6d50007ef0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 10 monclient: shutdown 2026-02-17T05:56:41.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:41.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 -- 10.20.193.16:0/538755379 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5c15caf0 msgr2=0x7f6d5c0737a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5c15caf0 0x7f6d5c0737a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0c5d00 tx=0x7f6d4c004b40 comp rx=0 tx=0).stop 2026-02-17T05:56:41.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 -- 10.20.193.16:0/538755379 shutdown_connections 2026-02-17T05:56:41.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079140 0x7f6d2807b600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5c15caf0 0x7f6d5c0737a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5c14f400 0x7f6d5c070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 --2- 10.20.193.16:0/538755379 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5c14f020 0x7f6d5c1f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 -- 10.20.193.16:0/538755379 >> 10.20.193.16:0/538755379 conn(0x7f6d5c0cd770 msgr2=0x7f6d5c0c8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:41.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 -- 10.20.193.16:0/538755379 shutdown_connections 2026-02-17T05:56:41.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 -- 10.20.193.16:0/538755379 wait complete. 2026-02-17T05:56:41.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.861+0000 7f6d60d87640 1 librados: shutdown 2026-02-17T05:56:41.870 DEBUG:teuthology.orchestra.run.trial016:> rbd --cluster ceph pool init rbd 2026-02-17T05:56:41.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.900+0000 7f00904500c0 10 monclient: get_monmap_and_config 2026-02-17T05:56:41.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.900+0000 7f00904500c0 10 monclient: build_initial_monmap 2026-02-17T05:56:41.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.900+0000 7f00904500c0 10 monclient: monmap: 2026-02-17T05:56:41.902 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:41.902 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.902 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:41.901667+0000 2026-02-17T05:56:41.902 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:41.901667+0000 2026-02-17T05:56:41.902 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:41.902 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.902 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:41.902 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:41.902 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:41.903 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:41.903 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:41.903 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:41.903 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 1 Processor -- start 2026-02-17T05:56:41.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 1 -- start start 2026-02-17T05:56:41.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 10 monclient: init 2026-02-17T05:56:41.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 10 monclient: _reopen_session rank -1 2026-02-17T05:56:41.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 10 monclient: _add_conns ranks=[4,5,1,2,0,3] 2026-02-17T05:56:41.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 1 --2- >> v2:10.20.193.128:3300/0 conn(0x55cace1ac0b0 0x55cace125cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 10 monclient(hunting): picked mon.noname-b con 0x55cace1ac0b0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:41.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 1 --2- >> v2:10.20.193.148:3300/0 conn(0x55cace125540 0x55cace1b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 10 monclient(hunting): picked mon.noname-c con 0x55cace125540 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:41.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 10 monclient(hunting): picked mon.noname-b-legacy con 0x55cace1b6fc0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:41.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55cacdf47d20 con 0x55cace1b6fc0 2026-02-17T05:56:41.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x55cacdf74720 con 0x55cace1ac0b0 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x55cacdfc8e80 con 0x55cace125540 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 10 monclient(hunting): _renew_subs 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.901+0000 7f00904500c0 10 monclient(hunting): authenticate will time out at 1054.470905s 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008e715640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x55cace125540 0x55cace1b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008ef16640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x55cace1ac0b0 0x55cace125cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f009019f640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x55cace1b6fc0 0x55cace123150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:53714/0 (socket says 10.20.193.16:53714) 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f009019f640 1 -- 10.20.193.16:0/2253419642 learned_addr learned my addr 10.20.193.16:0/2253419642 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008e715640 10 monclient(hunting): get_auth_request con 0x55cace125540 auth_method 0 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008e715640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008e715640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008e715640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008ef16640 10 monclient(hunting): get_auth_request con 0x55cace1ac0b0 auth_method 0 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008ef16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008ef16640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008ef16640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008e715640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.902+0000 7f008e715640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.903+0000 7f008e715640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.903+0000 7f008ef16640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.903+0000 7f008ef16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.903+0000 7f008ef16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.904+0000 7f008e715640 10 monclient(hunting): handle_auth_done global_id 4412 payload 274 2026-02-17T05:56:41.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.904+0000 7f008e715640 1 -- 10.20.193.16:0/2253419642 >> v2:10.20.193.128:3300/0 conn(0x55cace1ac0b0 msgr2=0x55cace125cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:41.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.904+0000 7f008e715640 1 --2- 10.20.193.16:0/2253419642 >> v2:10.20.193.128:3300/0 conn(0x55cace1ac0b0 0x55cace125cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.904+0000 7f008e715640 1 -- 10.20.193.16:0/2253419642 >> v1:10.20.193.128:6789/0 conn(0x55cace1b6fc0 legacy=0x55cace123150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.904+0000 7f008e715640 10 monclient: _finish_hunting 0 2026-02-17T05:56:41.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.904+0000 7f008e715640 1 monclient: found mon.noname-c 2026-02-17T05:56:41.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.904+0000 7f008e715640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:41.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.904+0000 7f008e715640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.904+0000 7f008e715640 1 -- 10.20.193.16:0/2253419642 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cace1ac490 con 0x55cace125540 2026-02-17T05:56:41.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.905+0000 7f008e715640 1 --2- 10.20.193.16:0/2253419642 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x55cace125540 0x55cace1b6a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55cacdfccbc0 tx=0x7f00780585e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c5cbffca926b18 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:41.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 1 -- 10.20.193.16:0/2253419642 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007806d070 con 0x55cace125540 2026-02-17T05:56:41.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:41.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 10 monclient: dump: 2026-02-17T05:56:41.916 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:41.916 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.916 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.916 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.916 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.916 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 10 monclient: _finish_auth 0 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 10 monclient: _check_auth_tickets 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 1 -- 10.20.193.16:0/2253419642 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0078060cd0 con 0x55cace125540 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 5 monclient: authenticate success, global_id 4412 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:41.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 1 -- 10.20.193.16:0/2253419642 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078068680 con 0x55cace125540 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 10 monclient: get_monmap_and_config success 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 4 set_mon_vals no callback set 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f008df14640 10 monclient: dump: 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.918 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 10 monclient: shutdown 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 1 -- 10.20.193.16:0/2253419642 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x55cace125540 msgr2=0x55cace1b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 1 --2- 10.20.193.16:0/2253419642 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x55cace125540 0x55cace1b6a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55cacdfccbc0 tx=0x7f00780585e0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 1 -- 10.20.193.16:0/2253419642 shutdown_connections 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 1 --2- 10.20.193.16:0/2253419642 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x55cace125540 0x55cace1b6a80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 1 --2- 10.20.193.16:0/2253419642 >> v2:10.20.193.128:3300/0 conn(0x55cace1ac0b0 0x55cace125cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 1 -- 10.20.193.16:0/2253419642 >> 10.20.193.16:0/2253419642 conn(0x55cace0ddec0 msgr2=0x55cace0de2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 1 -- 10.20.193.16:0/2253419642 shutdown_connections 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 1 -- 10.20.193.16:0/2253419642 wait complete. 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 10 monclient: build_initial_monmap 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.908+0000 7f00904500c0 10 monclient: monmap: 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.920 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 librados: starting msgr at 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 librados: starting objecter 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 Processor -- start 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 -- start start 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 librados: setting wanted keys 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 librados: calling monclient init 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient: init 2026-02-17T05:56:41.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient: _reopen_session rank -1 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x55cace125540 0x55cace1c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient(hunting): picked mon.noname-c con 0x55cace125540 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x55cace1ac0b0 0x55cace1c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient(hunting): picked mon.noname-a con 0x55cace1ac0b0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x55cace1b6fc0 0x55cace196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient(hunting): picked mon.noname-b con 0x55cace1b6fc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x55cacdf63e50 con 0x55cace1ac0b0 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x55cacdfc4db0 con 0x55cace1b6fc0 2026-02-17T05:56:41.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x55cace126200 con 0x55cace125540 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient(hunting): _renew_subs 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.909+0000 7f00904500c0 10 monclient(hunting): authenticate will time out at 1054.478672s 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008ef16640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x55cace125540 0x55cace1c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008e715640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x55cace1ac0b0 0x55cace1c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008ef16640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x55cace125540 0x55cace1c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56728/0 (socket says 10.20.193.16:56728) 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008ef16640 1 -- 10.20.193.16:0/215595543 learned_addr learned my addr 10.20.193.16:0/215595543 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008ef16640 10 monclient(hunting): get_auth_request con 0x55cace125540 auth_method 0 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008ef16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008ef16640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008ef16640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008e715640 10 monclient(hunting): get_auth_request con 0x55cace1ac0b0 auth_method 0 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008e715640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008e715640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008e715640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f009019f640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x55cace1b6fc0 0x55cace196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008ef16640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008ef16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008ef16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008e715640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008e715640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f008e715640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f009019f640 10 monclient(hunting): get_auth_request con 0x55cace1b6fc0 auth_method 0 2026-02-17T05:56:41.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f009019f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f009019f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f009019f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f009019f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f009019f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.910+0000 7f009019f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.911+0000 7f008ef16640 10 monclient(hunting): handle_auth_done global_id 4415 payload 824 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.911+0000 7f008ef16640 1 -- 10.20.193.16:0/215595543 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x55cace1b6fc0 msgr2=0x55cace196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.911+0000 7f008ef16640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x55cace1b6fc0 0x55cace196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.911+0000 7f008ef16640 1 -- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x55cace1ac0b0 msgr2=0x55cace1c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.911+0000 7f008ef16640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x55cace1ac0b0 0x55cace1c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.911+0000 7f008ef16640 10 monclient: _finish_hunting 0 2026-02-17T05:56:41.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.911+0000 7f008ef16640 1 monclient: found mon.noname-c 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.911+0000 7f008ef16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.911+0000 7f008ef16640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.911+0000 7f008ef16640 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cace1c5f70 con 0x55cace125540 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.912+0000 7f008ef16640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x55cace125540 0x55cace1c5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f00840087e0 tx=0x7f00840088e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.912+0000 7f0076ffd640 10 client.?.objecter ms_handle_connect 0x55cace125540 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.912+0000 7f0076ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084012790 con 0x55cace125540 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 10 monclient: dump: 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:41.926 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:41.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0084018070 con 0x55cace125540 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084012b30 con 0x55cace125540 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f0076ffd640 10 monclient: dump: 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:41.928 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f00904500c0 5 monclient: authenticate success, global_id 4415 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f00904500c0 10 monclient: _renew_subs 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f00904500c0 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cace0e58e0 con 0x55cace125540 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f008d713640 4 set_mon_vals no callback set 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f00904500c0 10 client.4415.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:41.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f00904500c0 10 monclient: _renew_subs 2026-02-17T05:56:41.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f00904500c0 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cace1c59b0 con 0x55cace125540 2026-02-17T05:56:41.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f00904500c0 1 librados: init done 2026-02-17T05:56:41.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.914+0000 7f00904500c0 10 librados: wait_for_osdmap waiting 2026-02-17T05:56:41.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.926+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f0084012cd0 con 0x55cace125540 2026-02-17T05:56:41.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.926+0000 7f0076ffd640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0070077170 0x7f0070079630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:41.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.926+0000 7f008e715640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0070077170 0x7f0070079630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:41.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.926+0000 7f008e715640 10 monclient: get_auth_request con 0x7f0070077170 auth_method 0 2026-02-17T05:56:41.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.926+0000 7f008e715640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0070077170 0x7f0070079630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f007806a000 tx=0x7f0078064000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:41.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.926+0000 7f0076ffd640 10 client.4415.objecter ms_handle_connect 0x7f0070077170 2026-02-17T05:56:41.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.927+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 8905+0+0 (secure 0 0 0) 0x7f0084096500 con 0x55cace125540 2026-02-17T05:56:41.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.927+0000 7f0076ffd640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 osd_map(30..30 src has 1..30) 2026-02-17T05:56:41.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.927+0000 7f0076ffd640 3 client.4415.objecter handle_osd_map got epochs [30,30] > 0 2026-02-17T05:56:41.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.927+0000 7f0076ffd640 3 client.4415.objecter handle_osd_map decoding full epoch 30 2026-02-17T05:56:41.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.928+0000 7f0076ffd640 20 client.4415.objecter dump_active .. 0 homeless 2026-02-17T05:56:41.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.928+0000 7f00904500c0 10 librados: wait_for_osdmap done waiting 2026-02-17T05:56:41.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.928+0000 7f00904500c0 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-17T05:56:41.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.928+0000 7f00904500c0 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-17T05:56:41.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.928+0000 7f00904500c0 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:41.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:41.928+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x55cace0e5b80 con 0x55cace125540 2026-02-17T05:56:42.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.933+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v31) ==== 141+0+0 (secure 0 0 0) 0x7f0084060b30 con 0x55cace125540 2026-02-17T05:56:42.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.933+0000 7f0076ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-17T05:56:42.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.933+0000 7f0076ffd640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2026-02-17T05:56:42.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.933+0000 7f00904500c0 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:42.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.933+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0078060cd0 con 0x55cace125540 2026-02-17T05:56:42.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.937+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f008405cf30 con 0x55cace125540 2026-02-17T05:56:42.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.937+0000 7f0076ffd640 10 monclient: handle_get_version_reply finishing 1 version 31 2026-02-17T05:56:42.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.937+0000 7f008d713640 10 client.4415.objecter _get_latest_version latest 31, waiting 2026-02-17T05:56:42.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.937+0000 7f008d713640 10 client.4415.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:42.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.937+0000 7f008d713640 10 monclient: _renew_subs 2026-02-17T05:56:42.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.937+0000 7f008d713640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:42.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.937+0000 7f008d713640 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=31}) -- 0x7f0068003c90 con 0x55cace125540 2026-02-17T05:56:42.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 8 ==== osd_map(31..31 src has 1..31) ==== 682+0+0 (secure 0 0 0) 0x7f00840871a0 con 0x55cace125540 2026-02-17T05:56:42.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f0076ffd640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 osd_map(31..31 src has 1..31) 2026-02-17T05:56:42.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f0076ffd640 3 client.4415.objecter handle_osd_map got epochs [31,31] > 30 2026-02-17T05:56:42.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f0076ffd640 3 client.4415.objecter handle_osd_map decoding incremental epoch 31 2026-02-17T05:56:42.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f0076ffd640 20 client.4415.objecter dump_active .. 0 homeless 2026-02-17T05:56:42.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 10 librados: create oid=rbd_trash nspace= 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 10 client.4415.objecter _op_submit op 0x55cace196a90 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 20 client.4415.objecter _calc_target epoch 31 base rbd_trash @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 20 client.4415.objecter _calc_target target rbd_trash @2 -> pgid 2.5a749323 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 10 client.4415.objecter _calc_target raw pgid 2.5a749323 -> actual 2.3 acting [4,15] primary 4 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x55cace0e65b0 0x55cace1b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 20 client.4415.objecter _get_session s=0x55cacdf64230 osd=4 3 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 10 client.4415.objecter _op_submit oid rbd_trash '@2' '@2' [create] tid 1 osd.4 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 20 client.4415.objecter get_session s=0x55cacdf64230 osd=4 3 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 15 client.4415.objecter _session_op_assign 4 1 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 15 client.4415.objecter _send_op 1 to 2.3 on osd.4 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55cace1b7ec0 con 0x55cace0e65b0 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 20 client.4415.objecter put_session s=0x55cacdf64230 osd=4 4 2026-02-17T05:56:42.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.938+0000 7f00904500c0 5 client.4415.objecter 1 in flight 2026-02-17T05:56:42.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.941+0000 7f009019f640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x55cace0e65b0 0x55cace1b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:42.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.942+0000 7f009019f640 10 monclient: get_auth_request con 0x55cace0e65b0 auth_method 0 2026-02-17T05:56:42.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.945+0000 7f009019f640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x55cace0e65b0 0x55cace1b7980 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:42.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:42.945+0000 7f0076ffd640 10 client.4415.objecter ms_handle_connect 0x55cace0e65b0 2026-02-17T05:56:43.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f009019f640 1 -- 10.20.193.16:0/215595543 <== osd.4 v2:10.20.193.16:6818/1157611076 1 ==== osd_op_reply(1 rbd_trash [create] v31'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7f0080001860 con 0x55cace0e65b0 2026-02-17T05:56:43.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f009019f640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 osd_op_reply(1 rbd_trash [create] v31'1 uv1 ondisk = 0) 2026-02-17T05:56:43.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f009019f640 10 client.4415.objecter in handle_osd_op_reply 2026-02-17T05:56:43.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f009019f640 7 client.4415.objecter handle_osd_op_reply 1 ondisk uv 1 in 2.3 attempt 0 2026-02-17T05:56:43.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f009019f640 10 client.4415.objecter op 0 rval 0 len 0 2026-02-17T05:56:43.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f009019f640 15 client.4415.objecter handle_osd_op_reply completed tid 1 2026-02-17T05:56:43.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f009019f640 15 client.4415.objecter _finish_op 1 2026-02-17T05:56:43.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f009019f640 20 client.4415.objecter put_session s=0x55cacdf64230 osd=4 4 2026-02-17T05:56:43.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f009019f640 15 client.4415.objecter _session_op_remove 4 1 2026-02-17T05:56:43.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f009019f640 5 client.4415.objecter 0 in flight 2026-02-17T05:56:43.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.397+0000 7f00904500c0 10 librados: Objecter returned from create r=0 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 10 client.4415.objecter _op_submit op 0x55cace221df0 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 20 client.4415.objecter _calc_target epoch 31 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_read 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 20 client.4415.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 10 client.4415.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x55cace2229a0 0x55cace224e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 20 client.4415.objecter _get_session s=0x55cace222320 osd=3 3 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 10 client.4415.objecter _op_submit oid rbd_info '@2' '@2' [call rbd.metadata_list in=33b] tid 2 osd.3 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 20 client.4415.objecter get_session s=0x55cace222320 osd=3 3 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 15 client.4415.objecter _session_op_assign 3 2 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 15 client.4415.objecter _send_op 2 to 2.2 on osd.3 2026-02-17T05:56:43.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55cace2253b0 con 0x55cace2229a0 2026-02-17T05:56:43.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 20 client.4415.objecter put_session s=0x55cace222320 osd=3 4 2026-02-17T05:56:43.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f00904500c0 5 client.4415.objecter 1 in flight 2026-02-17T05:56:43.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.399+0000 7f008ef16640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x55cace2229a0 0x55cace224e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:43.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.400+0000 7f008ef16640 10 monclient: get_auth_request con 0x55cace2229a0 auth_method 0 2026-02-17T05:56:43.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.402+0000 7f008ef16640 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x55cace2229a0 0x55cace224e30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:43.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.402+0000 7f0076ffd640 10 client.4415.objecter ms_handle_connect 0x55cace2229a0 2026-02-17T05:56:43.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 1 -- 10.20.193.16:0/215595543 <== osd.3 v2:10.20.193.164:6824/3772404125 1 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f00840603f0 con 0x55cace2229a0 2026-02-17T05:56:43.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 10 client.4415.objecter in handle_osd_op_reply 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 7 client.4415.objecter handle_osd_op_reply 2 ondisk uv 0 in 2.2 attempt 0 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 10 client.4415.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 10 client.4415.objecter op 0 rval 0 len 0 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 15 client.4415.objecter handle_osd_op_reply completed tid 2 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 15 client.4415.objecter _finish_op 2 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 20 client.4415.objecter put_session s=0x55cace222320 osd=3 4 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 15 client.4415.objecter _session_op_remove 3 2 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f008ef16640 5 client.4415.objecter 0 in flight 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 10 client.4415.objecter _op_submit op 0x55cace220fd0 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 20 client.4415.objecter _calc_target epoch 31 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_read 2026-02-17T05:56:43.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 20 client.4415.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-17T05:56:43.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 10 client.4415.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-17T05:56:43.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 20 client.4415.objecter _get_session s=0x55cace222320 osd=3 3 2026-02-17T05:56:43.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 10 client.4415.objecter _op_submit oid rbd_info '@2' '@2' [read 0~0] tid 3 osd.3 2026-02-17T05:56:43.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 20 client.4415.objecter get_session s=0x55cace222320 osd=3 3 2026-02-17T05:56:43.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 15 client.4415.objecter _session_op_assign 3 3 2026-02-17T05:56:43.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 15 client.4415.objecter _send_op 3 to 2.2 on osd.3 2026-02-17T05:56:43.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55cace221480 con 0x55cace2229a0 2026-02-17T05:56:43.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 20 client.4415.objecter put_session s=0x55cace222320 osd=3 4 2026-02-17T05:56:43.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.451+0000 7f00904500c0 5 client.4415.objecter 1 in flight 2026-02-17T05:56:43.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 1 -- 10.20.193.16:0/215595543 <== osd.3 v2:10.20.193.164:6824/3772404125 2 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f00840603f0 con 0x55cace2229a0 2026-02-17T05:56:43.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-17T05:56:43.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 10 client.4415.objecter in handle_osd_op_reply 2026-02-17T05:56:43.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 7 client.4415.objecter handle_osd_op_reply 3 ondisk uv 0 in 2.2 attempt 0 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 10 client.4415.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 10 client.4415.objecter op 0 rval 0 len 0 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 15 client.4415.objecter handle_osd_op_reply completed tid 3 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 15 client.4415.objecter _finish_op 3 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 20 client.4415.objecter put_session s=0x55cace222320 osd=3 4 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 15 client.4415.objecter _session_op_remove 3 3 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008ef16640 5 client.4415.objecter 0 in flight 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008cf12640 10 client.4415.objecter allocate_selfmanaged_snap; pool: 2 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008cf12640 10 client.4415.objecter pool_op_submit 4 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008cf12640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:43.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:43.455+0000 7f008cf12640 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7f006c0030d0 con 0x55cace125540 2026-02-17T05:56:44.254 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:02:23.081 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:44.287 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:02:23.115 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:44.293 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:02:23.119 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:44.302 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:02:23.128 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:44.315 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:02:23.104 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:44.321 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:02:23.110 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:44.333 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:02:23.159 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:44.353 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:02:23.180 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:44.356 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:02:23.146 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:56:44.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.794+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 9 ==== pool_op_reply(tid 4 (0) Success v32) ==== 55+0+0 (secure 0 0 0) 0x7f0084083290 con 0x55cace125540 2026-02-17T05:56:44.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.794+0000 7f0076ffd640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 pool_op_reply(tid 4 (0) Success v32) 2026-02-17T05:56:44.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.794+0000 7f0076ffd640 10 client.4415.objecter handle_pool_op_reply pool_op_reply(tid 4 (0) Success v32) 2026-02-17T05:56:44.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.794+0000 7f0076ffd640 10 client.4415.objecter have request 4 at 0x7f006c003060 Op: create unmanaged snap 2026-02-17T05:56:44.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.794+0000 7f0076ffd640 20 client.4415.objecter waiting for client to reach epoch 32 before calling back 2026-02-17T05:56:44.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.794+0000 7f0076ffd640 10 client.4415.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:44.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.794+0000 7f0076ffd640 10 monclient: _renew_subs 2026-02-17T05:56:44.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.794+0000 7f0076ffd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:44.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.794+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=32}) -- 0x7f007007f900 con 0x55cace125540 2026-02-17T05:56:44.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.794+0000 7f0076ffd640 10 client.4415.objecter done 2026-02-17T05:56:44.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 10 ==== osd_map(32..32 src has 1..32) ==== 626+0+0 (secure 0 0 0) 0x7f0084069850 con 0x55cace125540 2026-02-17T05:56:44.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f0076ffd640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 osd_map(32..32 src has 1..32) 2026-02-17T05:56:44.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f0076ffd640 3 client.4415.objecter handle_osd_map got epochs [32,32] > 31 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f0076ffd640 3 client.4415.objecter handle_osd_map decoding incremental epoch 32 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f0076ffd640 20 client.4415.objecter dump_active .. 0 homeless 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 20 librados: queue_aio_write 0x55cace21d380 completion 0x55cacdf885f0 write_seq 1 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 10 client.4415.objecter _op_submit op 0x7f0068006380 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 20 client.4415.objecter _calc_target epoch 32 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 20 client.4415.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 10 client.4415.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 20 client.4415.objecter _get_session s=0x55cace222320 osd=3 3 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 10 client.4415.objecter _op_submit oid rbd_info '@2' '@2' [create,write 0~8 in=8b] tid 5 osd.3 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 20 client.4415.objecter get_session s=0x55cace222320 osd=3 3 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 15 client.4415.objecter _session_op_assign 3 5 2026-02-17T05:56:44.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 15 client.4415.objecter _send_op 5 to 2.2 on osd.3 2026-02-17T05:56:44.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x7f0068006aa0 con 0x55cace2229a0 2026-02-17T05:56:44.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 20 client.4415.objecter put_session s=0x55cace222320 osd=3 4 2026-02-17T05:56:44.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.796+0000 7f008d713640 5 client.4415.objecter 1 in flight 2026-02-17T05:56:44.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.909+0000 7f0075ffb640 10 monclient: tick 2026-02-17T05:56:44.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.909+0000 7f0075ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:44.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.909+0000 7f0075ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:44.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:44.909+0000 7f0075ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:45.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 1 -- 10.20.193.16:0/215595543 <== osd.3 v2:10.20.193.164:6824/3772404125 3 ==== osd_op_reply(5 rbd_info [create,write 0~8] v32'1 uv1 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f00840603f0 con 0x55cace2229a0 2026-02-17T05:56:45.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 osd_op_reply(5 rbd_info [create,write 0~8] v32'1 uv1 ondisk = 0) 2026-02-17T05:56:45.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 10 client.4415.objecter in handle_osd_op_reply 2026-02-17T05:56:45.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 7 client.4415.objecter handle_osd_op_reply 5 ondisk uv 1 in 2.2 attempt 0 2026-02-17T05:56:45.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 10 client.4415.objecter op 0 rval 0 len 0 2026-02-17T05:56:45.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 10 client.4415.objecter op 1 rval 0 len 0 2026-02-17T05:56:45.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 15 client.4415.objecter handle_osd_op_reply completed tid 5 2026-02-17T05:56:45.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 15 client.4415.objecter _finish_op 5 2026-02-17T05:56:45.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 20 client.4415.objecter put_session s=0x55cace222320 osd=3 4 2026-02-17T05:56:45.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 15 client.4415.objecter _session_op_remove 3 5 2026-02-17T05:56:45.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 5 client.4415.objecter 0 in flight 2026-02-17T05:56:45.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008ef16640 20 librados: complete_aio_write 0x55cacdf885f0 2026-02-17T05:56:45.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008cf12640 10 client.4415.objecter delete_selfmanaged_snap; pool: 2; snap: 2 2026-02-17T05:56:45.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008cf12640 10 client.4415.objecter pool_op_submit 6 2026-02-17T05:56:45.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008cf12640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:45.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.042+0000 7f008cf12640 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v32) -- 0x7f006c004520 con 0x55cace125540 2026-02-17T05:56:45.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.451+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 11 ==== pool_op_reply(tid 6 (0) Success v33) ==== 43+0+0 (secure 0 0 0) 0x7f0084069a50 con 0x55cace125540 2026-02-17T05:56:45.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.451+0000 7f0076ffd640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 pool_op_reply(tid 6 (0) Success v33) 2026-02-17T05:56:45.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.451+0000 7f0076ffd640 10 client.4415.objecter handle_pool_op_reply pool_op_reply(tid 6 (0) Success v33) 2026-02-17T05:56:45.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.451+0000 7f0076ffd640 10 client.4415.objecter have request 6 at 0x7f006c003060 Op: delete unmanaged snap 2026-02-17T05:56:45.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.451+0000 7f0076ffd640 20 client.4415.objecter waiting for client to reach epoch 33 before calling back 2026-02-17T05:56:45.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.451+0000 7f0076ffd640 10 client.4415.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:45.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.451+0000 7f0076ffd640 10 monclient: _renew_subs 2026-02-17T05:56:45.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.451+0000 7f0076ffd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:45.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.451+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=33}) -- 0x7f007007fbe0 con 0x55cace125540 2026-02-17T05:56:45.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.451+0000 7f0076ffd640 10 client.4415.objecter done 2026-02-17T05:56:45.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f0076ffd640 1 -- 10.20.193.16:0/215595543 <== mon.2 v2:10.20.193.148:3300/0 12 ==== osd_map(33..33 src has 1..33) ==== 654+0+0 (secure 0 0 0) 0x7f008405fb40 con 0x55cace125540 2026-02-17T05:56:45.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f0076ffd640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 osd_map(33..33 src has 1..33) 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f0076ffd640 3 client.4415.objecter handle_osd_map got epochs [33,33] > 32 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f0076ffd640 3 client.4415.objecter handle_osd_map decoding incremental epoch 33 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f0076ffd640 20 client.4415.objecter dump_active .. 0 homeless 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 20 librados: queue_aio_write 0x55cace21d380 completion 0x7f006c001d60 write_seq 2 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 10 client.4415.objecter _op_submit op 0x7f0068006380 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 20 client.4415.objecter _calc_target epoch 33 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 20 client.4415.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 10 client.4415.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 20 client.4415.objecter _get_session s=0x55cace222320 osd=3 3 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 10 client.4415.objecter _op_submit oid rbd_info '@2' '@2' [write 0~19 in=19b] tid 7 osd.3 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 20 client.4415.objecter get_session s=0x55cace222320 osd=3 3 2026-02-17T05:56:45.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 15 client.4415.objecter _session_op_assign 3 7 2026-02-17T05:56:45.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 15 client.4415.objecter _send_op 7 to 2.2 on osd.3 2026-02-17T05:56:45.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 1 -- 10.20.193.16:0/215595543 --> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x7f0068006e40 con 0x55cace2229a0 2026-02-17T05:56:45.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 20 client.4415.objecter put_session s=0x55cace222320 osd=3 4 2026-02-17T05:56:45.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.452+0000 7f008d713640 5 client.4415.objecter 1 in flight 2026-02-17T05:56:45.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 1 -- 10.20.193.16:0/215595543 <== osd.3 v2:10.20.193.164:6824/3772404125 4 ==== osd_op_reply(7 rbd_info [write 0~19] v33'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00840603f0 con 0x55cace2229a0 2026-02-17T05:56:45.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 10 client.4415.objecter ms_dispatch2 0x55cacdfdf5a0 osd_op_reply(7 rbd_info [write 0~19] v33'2 uv2 ondisk = 0) 2026-02-17T05:56:45.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 10 client.4415.objecter in handle_osd_op_reply 2026-02-17T05:56:45.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 7 client.4415.objecter handle_osd_op_reply 7 ondisk uv 2 in 2.2 attempt 0 2026-02-17T05:56:45.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 10 client.4415.objecter op 0 rval 0 len 0 2026-02-17T05:56:45.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 15 client.4415.objecter handle_osd_op_reply completed tid 7 2026-02-17T05:56:45.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 15 client.4415.objecter _finish_op 7 2026-02-17T05:56:45.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 20 client.4415.objecter put_session s=0x55cace222320 osd=3 4 2026-02-17T05:56:45.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 15 client.4415.objecter _session_op_remove 3 7 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 5 client.4415.objecter 0 in flight 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f008ef16640 20 librados: complete_aio_write 0x7f006c001d60 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 10 librados: watch_flush enter 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 10 librados: watch_flush exit 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 10 client.4415.objecter close_session for osd.3 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x55cace2229a0 msgr2=0x55cace224e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x55cace2229a0 0x55cace224e30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 20 client.4415.objecter put_session s=0x55cace222320 osd=3 1 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 10 client.4415.objecter close_session for osd.4 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x55cace0e65b0 msgr2=0x55cace1b7980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:45.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x55cace0e65b0 0x55cace1b7980 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 20 client.4415.objecter put_session s=0x55cacdf64230 osd=4 1 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 20 client.4415.objecter shutdown clearing up homeless session... 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 10 client.4415.objecter successfully canceled tick 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0070077170 msgr2=0x7f0070079630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0070077170 0x7f0070079630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f007806a000 tx=0x7f0078064000 comp rx=0 tx=0).stop 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 10 monclient: shutdown 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x55cace125540 msgr2=0x55cace1c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 --2- 10.20.193.16:0/215595543 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x55cace125540 0x55cace1c5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f00840087e0 tx=0x7f00840088e0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f009019f640 1 -- 10.20.193.16:0/215595543 reap_dead start 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f009019f640 1 -- 10.20.193.16:0/215595543 reap_dead start 2026-02-17T05:56:45.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 shutdown_connections 2026-02-17T05:56:45.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 >> 10.20.193.16:0/215595543 conn(0x55cace0ddec0 msgr2=0x55cace123050 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:45.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 shutdown_connections 2026-02-17T05:56:45.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 -- 10.20.193.16:0/215595543 wait complete. 2026-02-17T05:56:45.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.559+0000 7f00904500c0 1 librados: shutdown 2026-02-17T05:56:45.565 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-17T05:56:45.566 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-17T05:56:45.566 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-17T05:56:45.566 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-17T05:56:45.567 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-17T05:56:45.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.675+0000 7f9483196640 10 monclient: get_monmap_and_config 2026-02-17T05:56:45.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.675+0000 7f9483196640 10 monclient: build_initial_monmap 2026-02-17T05:56:45.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.675+0000 7f9483196640 10 monclient: monmap: 2026-02-17T05:56:45.677 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:45.677 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:45.677 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:45.676835+0000 2026-02-17T05:56:45.677 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:45.676835+0000 2026-02-17T05:56:45.677 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:45.677 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:45.678 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:45.678 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:45.678 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:45.678 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:45.678 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:45.678 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:45.678 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:45.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 1 Processor -- start 2026-02-17T05:56:45.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 1 -- start start 2026-02-17T05:56:45.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 10 monclient: init 2026-02-17T05:56:45.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:45.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 10 monclient: _add_conns ranks=[4,0,5,3,2,1] 2026-02-17T05:56:45.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f947c152550 0x7f947c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:45.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 10 monclient(hunting): picked mon.noname-b con 0x7f947c152550 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:45.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f947c152930 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:45.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f947c15d930 0x7f947c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:45.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 10 monclient(hunting): picked mon.noname-c con 0x7f947c15d930 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f947c0cca70 con 0x7f947c152930 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f947c0cc850 con 0x7f947c152550 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f947c0cd960 con 0x7f947c15d930 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.676+0000 7f9483196640 10 monclient(hunting): authenticate will time out at 1058.245903s 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f948170c640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f947c15d930 0x7f947c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f9473fff640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f947c152930 0x7f947c15d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:55986/0 (socket says 10.20.193.16:55986) 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f9473fff640 1 -- 10.20.193.16:0/3296519766 learned_addr learned my addr 10.20.193.16:0/3296519766 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f948170c640 10 monclient(hunting): get_auth_request con 0x7f947c15d930 auth_method 0 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f948170c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f948170c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f948170c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:45.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f9480f0b640 1 --2- 10.20.193.16:0/3296519766 >> v2:10.20.193.128:3300/0 conn(0x7f947c152550 0x7f947c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f9480f0b640 10 monclient(hunting): get_auth_request con 0x7f947c152550 auth_method 0 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f9480f0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f9480f0b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f9480f0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f948170c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f948170c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.677+0000 7f948170c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f9480f0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f9480f0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f9480f0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f94737fe640 1 -- 10.20.193.16:0/3296519766 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78413047 0 0) 0x7f947c0cca70 con 0x7f947c152930 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f94737fe640 10 monclient(hunting): _negotiate 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f94737fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f94737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:45.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f94737fe640 10 monclient(hunting): my global_id is 4356 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f94737fe640 1 -- 10.20.193.16:0/3296519766 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9468003620 con 0x7f947c152930 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f948170c640 10 monclient(hunting): handle_auth_done global_id 4421 payload 274 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f948170c640 1 -- 10.20.193.16:0/3296519766 >> v2:10.20.193.128:3300/0 conn(0x7f947c152550 msgr2=0x7f947c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f948170c640 1 --2- 10.20.193.16:0/3296519766 >> v2:10.20.193.128:3300/0 conn(0x7f947c152550 0x7f947c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f948170c640 1 -- 10.20.193.16:0/3296519766 >> v1:10.20.193.16:6789/0 conn(0x7f947c152930 legacy=0x7f947c15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f948170c640 10 monclient: _finish_hunting 0 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f948170c640 1 monclient: found mon.noname-c 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f948170c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f948170c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.678+0000 7f948170c640 1 -- 10.20.193.16:0/3296519766 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947c0ccd50 con 0x7f947c15d930 2026-02-17T05:56:45.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.680+0000 7f948170c640 1 --2- 10.20.193.16:0/3296519766 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f947c15d930 0x7f947c06d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f947c0c8ad0 tx=0x7f947802f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b952799dc615ccf5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:45.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 1 -- 10.20.193.16:0/3296519766 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947803c070 con 0x7f947c15d930 2026-02-17T05:56:45.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 10 monclient: dump: 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:45.689 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f9483196640 5 monclient: authenticate success, global_id 4421 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f9483196640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 1 -- 10.20.193.16:0/3296519766 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9478037440 con 0x7f947c15d930 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 1 -- 10.20.193.16:0/3296519766 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94780377f0 con 0x7f947c15d930 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f94737fe640 10 monclient: dump: 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:45.690 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f9483196640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.681+0000 7f9483196640 4 set_mon_vals no callback set 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 10 monclient: shutdown 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 -- 10.20.193.16:0/3296519766 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f947c15d930 msgr2=0x7f947c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 --2- 10.20.193.16:0/3296519766 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f947c15d930 0x7f947c06d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f947c0c8ad0 tx=0x7f947802f1c0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 -- 10.20.193.16:0/3296519766 shutdown_connections 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 --2- 10.20.193.16:0/3296519766 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f947c15d930 0x7f947c06d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 --2- 10.20.193.16:0/3296519766 >> v2:10.20.193.128:3300/0 conn(0x7f947c152550 0x7f947c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 -- 10.20.193.16:0/3296519766 >> 10.20.193.16:0/3296519766 conn(0x7f947c0d1c50 msgr2=0x7f947c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 -- 10.20.193.16:0/3296519766 shutdown_connections 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 -- 10.20.193.16:0/3296519766 wait complete. 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 10 monclient: build_initial_monmap 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 10 monclient: monmap: 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:45.692 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 librados: starting msgr at 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 librados: starting objecter 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 Processor -- start 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 -- start start 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 librados: setting wanted keys 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 1 librados: calling monclient init 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.682+0000 7f9483196640 10 monclient: init 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f947c152550 0x7f947c1f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 10 monclient(hunting): picked mon.noname-a con 0x7f947c152550 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f947c152930 0x7f947c15e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 10 monclient(hunting): picked mon.noname-c con 0x7f947c152930 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f947c1f73a0 0x7f947c15af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 10 monclient(hunting): picked mon.noname-b con 0x7f947c1f73a0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:45.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f947c0cd5f0 con 0x7f947c152550 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f947c0cd310 con 0x7f947c1f73a0 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f947c06ded0 con 0x7f947c152930 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9483196640 10 monclient(hunting): authenticate will time out at 1058.252262s 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9473fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f947c152930 0x7f947c15e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f948170c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f947c1f73a0 0x7f947c15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9473fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f947c152930 0x7f947c15e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56760/0 (socket says 10.20.193.16:56760) 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f948170c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f947c1f73a0 0x7f947c15af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:49954/0 (socket says 10.20.193.16:49954) 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9473fff640 1 -- 10.20.193.16:0/3385820886 learned_addr learned my addr 10.20.193.16:0/3385820886 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9473fff640 10 monclient(hunting): get_auth_request con 0x7f947c152930 auth_method 0 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9473fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:45.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9473fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f9473fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f948170c640 10 monclient(hunting): get_auth_request con 0x7f947c1f73a0 auth_method 0 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f948170c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f948170c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.683+0000 7f948170c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9480f0b640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f947c152550 0x7f947c1f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f948170c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f948170c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f948170c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9480f0b640 10 monclient(hunting): get_auth_request con 0x7f947c152550 auth_method 0 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9480f0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9480f0b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9480f0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9473fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9473fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:45.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9473fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9480f0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9480f0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.684+0000 7f9480f0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.685+0000 7f948170c640 10 monclient(hunting): handle_auth_done global_id 4396 payload 824 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.685+0000 7f948170c640 1 -- 10.20.193.16:0/3385820886 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f947c152930 msgr2=0x7f947c15e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.685+0000 7f948170c640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f947c152930 0x7f947c15e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.685+0000 7f948170c640 1 -- 10.20.193.16:0/3385820886 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f947c152550 msgr2=0x7f947c1f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.685+0000 7f948170c640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f947c152550 0x7f947c1f32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.685+0000 7f948170c640 10 monclient: _finish_hunting 0 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.685+0000 7f948170c640 1 monclient: found mon.noname-b 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.685+0000 7f948170c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.685+0000 7f948170c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.685+0000 7f948170c640 1 -- 10.20.193.16:0/3385820886 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947c1f3a30 con 0x7f947c1f73a0 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.687+0000 7f948170c640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f947c1f73a0 0x7f947c15af90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f947c0cdae0 tx=0x7f9478004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:45.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.687+0000 7f94717fa640 10 client.?.objecter ms_handle_connect 0x7f947c1f73a0 2026-02-17T05:56:45.697 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.687+0000 7f94717fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:45.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 1 -- 10.20.193.16:0/3385820886 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947803c050 con 0x7f947c1f73a0 2026-02-17T05:56:45.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:45.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:45.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 10 monclient: dump: 2026-02-17T05:56:45.698 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:45.698 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:45.698 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:45.698 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 10 monclient: _finish_auth 0 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 10 monclient: _check_auth_tickets 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 1 -- 10.20.193.16:0/3385820886 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9478048070 con 0x7f947c1f73a0 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 5 monclient: authenticate success, global_id 4396 2026-02-17T05:56:45.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 10 monclient: _renew_subs 2026-02-17T05:56:45.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:45.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 1 -- 10.20.193.16:0/3385820886 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947c1f3d10 con 0x7f947c1f73a0 2026-02-17T05:56:45.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:45.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 1 -- 10.20.193.16:0/3385820886 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478009450 con 0x7f947c1f73a0 2026-02-17T05:56:45.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:45.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:45.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f94717fa640 10 monclient: dump: 2026-02-17T05:56:45.700 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:45.700 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9472ffd640 4 set_mon_vals no callback set 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 10 client.4396.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 10 monclient: _renew_subs 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 1 -- 10.20.193.16:0/3385820886 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947c14ca20 con 0x7f947c1f73a0 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 1 librados: init done 2026-02-17T05:56:45.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:45.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:45.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:45.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.690+0000 7f9483196640 1 -- 10.20.193.16:0/3385820886 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947c06e260 con 0x7f947c1f73a0 2026-02-17T05:56:45.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.701+0000 7f94717fa640 1 -- 10.20.193.16:0/3385820886 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f9478004280 con 0x7f947c1f73a0 2026-02-17T05:56:45.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.702+0000 7f94717fa640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f944c079190 0x7f944c07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:45.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.702+0000 7f9480f0b640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f944c079190 0x7f944c07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:45.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.702+0000 7f9480f0b640 10 monclient: get_auth_request con 0x7f944c079190 auth_method 0 2026-02-17T05:56:45.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.702+0000 7f94717fa640 10 client.4396.objecter ms_handle_connect 0x7f944c079190 2026-02-17T05:56:45.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.702+0000 7f9480f0b640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f944c079190 0x7f944c07b650 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f946c009c40 tx=0x7f946c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:45.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.703+0000 7f94717fa640 1 -- 10.20.193.16:0/3385820886 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f94780841a0 con 0x7f947c1f73a0 2026-02-17T05:56:45.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.703+0000 7f94717fa640 10 client.4396.objecter ms_dispatch2 0x7f947c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:45.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.703+0000 7f94717fa640 3 client.4396.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:45.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.703+0000 7f94717fa640 3 client.4396.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:45.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.703+0000 7f94717fa640 20 client.4396.objecter dump_active .. 0 homeless 2026-02-17T05:56:45.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.803+0000 7f94717fa640 1 -- 10.20.193.16:0/3385820886 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9478088160 con 0x7f947c1f73a0 2026-02-17T05:56:45.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.803+0000 7f94717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:45.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.803+0000 7f94717fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:45.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.907+0000 7f9483196640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2026-02-17T05:56:45.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.907+0000 7f9483196640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-17T05:56:45.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.907+0000 7f9483196640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:45.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.907+0000 7f9483196640 1 -- 10.20.193.16:0/3385820886 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f947c073ac0 con 0x7f947c1f73a0 2026-02-17T05:56:45.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.971+0000 7f94717fa640 1 -- 10.20.193.16:0/3385820886 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+192262 (secure 0 0 0) 0x7f947808c460 con 0x7f947c1f73a0 2026-02-17T05:56:45.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.971+0000 7f94717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-17T05:56:45.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.971+0000 7f94717fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:45.973 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 10 librados: watch_flush enter 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 10 librados: watch_flush exit 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 20 client.4396.objecter shutdown clearing up homeless session... 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 10 client.4396.objecter successfully canceled tick 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 -- 10.20.193.16:0/3385820886 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f944c079190 msgr2=0x7f944c07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f944c079190 0x7f944c07b650 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f946c009c40 tx=0x7f946c009290 comp rx=0 tx=0).stop 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 10 monclient: shutdown 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 -- 10.20.193.16:0/3385820886 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f947c1f73a0 msgr2=0x7f947c15af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f947c1f73a0 0x7f947c15af90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f947c0cdae0 tx=0x7f9478004b40 comp rx=0 tx=0).stop 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 -- 10.20.193.16:0/3385820886 shutdown_connections 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f944c079190 0x7f944c07b650 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f947c1f73a0 0x7f947c15af90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f947c152930 0x7f947c15e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 --2- 10.20.193.16:0/3385820886 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f947c152550 0x7f947c1f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 -- 10.20.193.16:0/3385820886 >> 10.20.193.16:0/3385820886 conn(0x7f947c0d1c50 msgr2=0x7f947c15f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 -- 10.20.193.16:0/3385820886 shutdown_connections 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 -- 10.20.193.16:0/3385820886 wait complete. 2026-02-17T05:56:45.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:45.972+0000 7f9483196640 1 librados: shutdown 2026-02-17T05:56:45.981 INFO:teuthology.orchestra.run.trial016.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6800","nonce":3354619179},{"type":"v1","addr":"10.20.193.16:6801","nonce":3354619179}]},"active_addr":"10.20.193.16:6801/3354619179","active_change":"2026-02-17T05:54:51.769378+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.16:0","nonce":3235060662}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.16:0","nonce":4291981703}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.16:0","nonce":3324508242}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.16:0","nonce":164574080}]}]} 2026-02-17T05:56:45.984 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-17T05:56:45.984 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-17T05:56:45.984 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.085+0000 7fea378a9640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.085+0000 7fea378a9640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.085+0000 7fea378a9640 10 monclient: monmap: 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.087889+0000 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.087889+0000 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.087 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.086+0000 7fea378a9640 1 Processor -- start 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.086+0000 7fea378a9640 1 -- start start 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.086+0000 7fea378a9640 10 monclient: init 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 10 monclient: _add_conns ranks=[3,0,2,1,4,5] 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fea30155e00 0x7fea300ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 10 monclient(hunting): picked mon.noname-a con 0x7fea30155e00 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fea301561e0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fea30074dc0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea300c96d0 con 0x7fea301561e0 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea30074860 con 0x7fea30074dc0 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fea300c9940 con 0x7fea30155e00 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea378a9640 10 monclient(hunting): authenticate will time out at 1058.657106s 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea3561e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fea30155e00 0x7fea300ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea35e1f640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fea30074dc0 0x7fea3015f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46394/0 (socket says 10.20.193.16:46394) 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea35e1f640 1 -- 10.20.193.16:0/702436282 learned_addr learned my addr 10.20.193.16:0/702436282 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea3561e640 10 monclient(hunting): get_auth_request con 0x7fea30155e00 auth_method 0 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea3561e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea3561e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.087+0000 7fea3561e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.088+0000 7fea3561e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.088+0000 7fea3561e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.088+0000 7fea3561e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea1ffff640 1 -- 10.20.193.16:0/702436282 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3398777058 0 0) 0x7fea2c002ca0 con 0x7fea30074dc0 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea1ffff640 10 monclient(hunting): _negotiate 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea1ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea1ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea1ffff640 10 monclient(hunting): my global_id is 4427 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea1ffff640 1 -- 10.20.193.16:0/702436282 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea18003620 con 0x7fea30074dc0 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea3561e640 10 monclient(hunting): handle_auth_done global_id 4362 payload 274 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea3561e640 1 -- 10.20.193.16:0/702436282 >> v1:10.20.193.148:6789/0 conn(0x7fea30074dc0 legacy=0x7fea3015f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea3561e640 1 -- 10.20.193.16:0/702436282 >> v1:10.20.193.16:6789/0 conn(0x7fea301561e0 legacy=0x7fea30074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea3561e640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea3561e640 1 monclient: found mon.noname-a 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea3561e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea3561e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.089+0000 7fea3561e640 1 -- 10.20.193.16:0/702436282 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea300c9c20 con 0x7fea30155e00 2026-02-17T05:56:46.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.091+0000 7fea3561e640 1 --2- 10.20.193.16:0/702436282 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30155e00 0x7fea300ca700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea200098a0 tx=0x7fea2002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7321bde78c1a282 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 1 -- 10.20.193.16:0/702436282 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2003c070 con 0x7fea30155e00 2026-02-17T05:56:46.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:46.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 10 monclient: dump: 2026-02-17T05:56:46.099 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.099 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.099 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.099 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.100 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.100 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.100 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.100 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.100 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.100 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 1 -- 10.20.193.16:0/702436282 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea2002fb70 con 0x7fea30155e00 2026-02-17T05:56:46.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 5 monclient: authenticate success, global_id 4362 2026-02-17T05:56:46.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 1 -- 10.20.193.16:0/702436282 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20037440 con 0x7fea30155e00 2026-02-17T05:56:46.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 4 set_mon_vals no callback set 2026-02-17T05:56:46.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:46.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea1ffff640 10 monclient: dump: 2026-02-17T05:56:46.102 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.102 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.102 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.102 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.102 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.102 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.102 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.102 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.102 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.102 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 10 monclient: shutdown 2026-02-17T05:56:46.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 1 -- 10.20.193.16:0/702436282 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30155e00 msgr2=0x7fea300ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 1 --2- 10.20.193.16:0/702436282 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30155e00 0x7fea300ca700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea200098a0 tx=0x7fea2002ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:46.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 1 -- 10.20.193.16:0/702436282 shutdown_connections 2026-02-17T05:56:46.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 1 --2- 10.20.193.16:0/702436282 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30155e00 0x7fea300ca700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 1 -- 10.20.193.16:0/702436282 >> 10.20.193.16:0/702436282 conn(0x7fea300ce6c0 msgr2=0x7fea300cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 1 -- 10.20.193.16:0/702436282 shutdown_connections 2026-02-17T05:56:46.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.093+0000 7fea378a9640 1 -- 10.20.193.16:0/702436282 wait complete. 2026-02-17T05:56:46.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient: monmap: 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.104 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 librados: starting msgr at 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 librados: starting objecter 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 Processor -- start 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 -- start start 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 librados: setting wanted keys 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 librados: calling monclient init 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient: init 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30074dc0 0x7fea3015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient(hunting): picked mon.noname-a con 0x7fea30074dc0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea30155e00 0x7fea3014c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient(hunting): picked mon.noname-c con 0x7fea30155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea301561e0 0x7fea301f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient(hunting): picked mon.noname-b con 0x7fea301561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea300c9360 con 0x7fea30074dc0 2026-02-17T05:56:46.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fea300ca1e0 con 0x7fea301561e0 2026-02-17T05:56:46.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea3015fa10 con 0x7fea30155e00 2026-02-17T05:56:46.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.094+0000 7fea378a9640 10 monclient(hunting): authenticate will time out at 1058.664793s 2026-02-17T05:56:46.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea3561e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30074dc0 0x7fea3015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea34e1d640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea30155e00 0x7fea3014c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea3561e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30074dc0 0x7fea3015ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59492/0 (socket says 10.20.193.16:59492) 2026-02-17T05:56:46.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea3561e640 1 -- 10.20.193.16:0/3588188717 learned_addr learned my addr 10.20.193.16:0/3588188717 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea3561e640 10 monclient(hunting): get_auth_request con 0x7fea30074dc0 auth_method 0 2026-02-17T05:56:46.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea3561e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea3561e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea3561e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea35e1f640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea301561e0 0x7fea301f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea35e1f640 10 monclient(hunting): get_auth_request con 0x7fea301561e0 auth_method 0 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea35e1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea35e1f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea35e1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea34e1d640 10 monclient(hunting): get_auth_request con 0x7fea30155e00 auth_method 0 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea34e1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea34e1d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea34e1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea3561e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea3561e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea3561e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea35e1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea35e1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea35e1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea34e1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea34e1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.095+0000 7fea34e1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.096+0000 7fea3561e640 10 monclient(hunting): handle_auth_done global_id 4368 payload 824 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.096+0000 7fea3561e640 1 -- 10.20.193.16:0/3588188717 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea30155e00 msgr2=0x7fea3014c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.096+0000 7fea3561e640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea30155e00 0x7fea3014c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.096+0000 7fea3561e640 1 -- 10.20.193.16:0/3588188717 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea301561e0 msgr2=0x7fea301f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.096+0000 7fea3561e640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea301561e0 0x7fea301f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.096+0000 7fea3561e640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.096+0000 7fea3561e640 1 monclient: found mon.noname-a 2026-02-17T05:56:46.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.096+0000 7fea3561e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.096+0000 7fea3561e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.096+0000 7fea3561e640 1 -- 10.20.193.16:0/3588188717 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea301f8210 con 0x7fea30074dc0 2026-02-17T05:56:46.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.097+0000 7fea3561e640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30074dc0 0x7fea3015ec00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fea20009130 tx=0x7fea200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.097+0000 7fea1dffb640 10 client.?.objecter ms_handle_connect 0x7fea30074dc0 2026-02-17T05:56:46.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.097+0000 7fea1dffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 1 -- 10.20.193.16:0/3588188717 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2003c040 con 0x7fea30074dc0 2026-02-17T05:56:46.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:46.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 10 monclient: dump: 2026-02-17T05:56:46.112 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.112 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.112 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.112 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.112 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 5 monclient: authenticate success, global_id 4368 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 1 -- 10.20.193.16:0/3588188717 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea20042630 con 0x7fea30074dc0 2026-02-17T05:56:46.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 10 monclient: _renew_subs 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 1 -- 10.20.193.16:0/3588188717 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea300d0210 con 0x7fea30074dc0 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 1 -- 10.20.193.16:0/3588188717 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20042a20 con 0x7fea30074dc0 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1dffb640 10 monclient: dump: 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.114 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea1f7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 10 client.4368.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 10 monclient: _renew_subs 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 1 -- 10.20.193.16:0/3588188717 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea301f7730 con 0x7fea30074dc0 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 1 librados: init done 2026-02-17T05:56:46.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.099+0000 7fea378a9640 1 -- 10.20.193.16:0/3588188717 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea30160450 con 0x7fea30074dc0 2026-02-17T05:56:46.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.112+0000 7fea1dffb640 1 -- 10.20.193.16:0/3588188717 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fea200413b0 con 0x7fea30074dc0 2026-02-17T05:56:46.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.112+0000 7fea1dffb640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea000791c0 0x7fea0007b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.112+0000 7fea34e1d640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea000791c0 0x7fea0007b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.112+0000 7fea34e1d640 10 monclient: get_auth_request con 0x7fea000791c0 auth_method 0 2026-02-17T05:56:46.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.112+0000 7fea34e1d640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea000791c0 0x7fea0007b680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fea301f4ee0 tx=0x7fea24007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.112+0000 7fea1dffb640 10 client.4368.objecter ms_handle_connect 0x7fea000791c0 2026-02-17T05:56:46.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.113+0000 7fea1dffb640 1 -- 10.20.193.16:0/3588188717 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fea2002fab0 con 0x7fea30074dc0 2026-02-17T05:56:46.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.113+0000 7fea1dffb640 10 client.4368.objecter ms_dispatch2 0x7fea30000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.113+0000 7fea1dffb640 3 client.4368.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.113+0000 7fea1dffb640 3 client.4368.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.113+0000 7fea1dffb640 20 client.4368.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.215+0000 7fea1dffb640 1 -- 10.20.193.16:0/3588188717 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fea30160450 con 0x7fea30074dc0 2026-02-17T05:56:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.215+0000 7fea1dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.215+0000 7fea1dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:46.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.298+0000 7fea378a9640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:46.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.298+0000 7fea378a9640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:46.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.298+0000 7fea378a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.298+0000 7fea378a9640 1 -- 10.20.193.16:0/3588188717 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea300ca700 con 0x7fea30074dc0 2026-02-17T05:56:46.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.348+0000 7fea1dffb640 1 -- 10.20.193.16:0/3588188717 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+24767 (secure 0 0 0) 0x7fea200871f0 con 0x7fea30074dc0 2026-02-17T05:56:46.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.348+0000 7fea1dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:46.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.348+0000 7fea1dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:46.350 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:56:46.350 INFO:teuthology.orchestra.run.trial016.stdout:{"epoch":33,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:56:45.354534+0000","last_up_change":"2026-02-17T05:56:37.615172+0000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:56:30.047883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-17T05:56:41.588944+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":33,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0100002288818359,"score_stable":4.0100002288818359,"optimal_score":0.81000000238418579,"raw_score_acting":3.25,"raw_score_stable":3.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6826","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6827","nonce":821011033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6828","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6829","nonce":821011033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6832","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6833","nonce":821011033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6830","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6831","nonce":821011033}]},"public_addr":"10.20.193.16:6827/821011033","cluster_addr":"10.20.193.16:6829/821011033","heartbeat_back_addr":"10.20.193.16:6833/821011033","heartbeat_front_addr":"10.20.193.16:6831/821011033","state":["exists","up"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6807","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6809","nonce":1618049789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6810","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6811","nonce":1618049789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6814","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6815","nonce":1618049789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6812","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6813","nonce":1618049789}]},"public_addr":"10.20.193.128:6809/1618049789","cluster_addr":"10.20.193.128:6811/1618049789","heartbeat_back_addr":"10.20.193.128:6815/1618049789","heartbeat_front_addr":"10.20.193.128:6813/1618049789","state":["exists","up"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6816","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6817","nonce":1588035471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6818","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6819","nonce":1588035471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6822","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6823","nonce":1588035471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6820","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6821","nonce":1588035471}]},"public_addr":"10.20.193.148:6817/1588035471","cluster_addr":"10.20.193.148:6819/1588035471","heartbeat_back_addr":"10.20.193.148:6823/1588035471","heartbeat_front_addr":"10.20.193.148:6821/1588035471","state":["exists","up"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6824","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6825","nonce":3772404125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6826","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6827","nonce":3772404125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6830","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6831","nonce":3772404125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6828","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6829","nonce":3772404125}]},"public_addr":"10.20.193.164:6825/3772404125","cluster_addr":"10.20.193.164:6827/3772404125","heartbeat_back_addr":"10.20.193.164:6831/3772404125","heartbeat_front_addr":"10.20.193.164:6829/3772404125","state":["exists","up"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6818","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6819","nonce":1157611076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6820","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6821","nonce":1157611076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6824","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6825","nonce":1157611076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6822","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6823","nonce":1157611076}]},"public_addr":"10.20.193.16:6819/1157611076","cluster_addr":"10.20.193.16:6821/1157611076","heartbeat_back_addr":"10.20.193.16:6825/1157611076","heartbeat_front_addr":"10.20.193.16:6823/1157611076","state":["exists","up"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6824","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6825","nonce":1593443319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6826","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6827","nonce":1593443319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6830","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6831","nonce":1593443319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6828","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6829","nonce":1593443319}]},"public_addr":"10.20.193.128:6825/1593443319","cluster_addr":"10.20.193.128:6827/1593443319","heartbeat_back_addr":"10.20.193.128:6831/1593443319","heartbeat_front_addr":"10.20.193.128:6829/1593443319","state":["exists","up"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6824","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6825","nonce":2491860138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6826","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6827","nonce":2491860138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6830","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6831","nonce":2491860138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6828","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6829","nonce":2491860138}]},"public_addr":"10.20.193.148:6825/2491860138","cluster_addr":"10.20.193.148:6827/2491860138","heartbeat_back_addr":"10.20.193.148:6831/2491860138","heartbeat_front_addr":"10.20.193.148:6829/2491860138","state":["exists","up"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6816","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6817","nonce":3942637745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6818","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6819","nonce":3942637745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6822","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6823","nonce":3942637745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6820","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6821","nonce":3942637745}]},"public_addr":"10.20.193.164:6817/3942637745","cluster_addr":"10.20.193.164:6819/3942637745","heartbeat_back_addr":"10.20.193.164:6823/3942637745","heartbeat_front_addr":"10.20.193.164:6821/3942637745","state":["exists","up"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6802","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6803","nonce":15364298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6804","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6805","nonce":15364298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6808","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6809","nonce":15364298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6806","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6807","nonce":15364298}]},"public_addr":"10.20.193.16:6803/15364298","cluster_addr":"10.20.193.16:6805/15364298","heartbeat_back_addr":"10.20.193.16:6809/15364298","heartbeat_front_addr":"10.20.193.16:6807/15364298","state":["exists","up"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6800","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6801","nonce":1813385411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6802","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6803","nonce":1813385411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6806","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6808","nonce":1813385411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6804","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6805","nonce":1813385411}]},"public_addr":"10.20.193.128:6801/1813385411","cluster_addr":"10.20.193.128:6803/1813385411","heartbeat_back_addr":"10.20.193.128:6808/1813385411","heartbeat_front_addr":"10.20.193.128:6805/1813385411","state":["exists","up"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6808","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6809","nonce":2531105113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6810","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6811","nonce":2531105113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6814","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6815","nonce":2531105113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6812","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6813","nonce":2531105113}]},"public_addr":"10.20.193.148:6809/2531105113","cluster_addr":"10.20.193.148:6811/2531105113","heartbeat_back_addr":"10.20.193.148:6815/2531105113","heartbeat_front_addr":"10.20.193.148:6813/2531105113","state":["exists","up"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6808","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6809","nonce":188912883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6810","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6811","nonce":188912883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6814","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6815","nonce":188912883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6812","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6813","nonce":188912883}]},"public_addr":"10.20.193.164:6809/188912883","cluster_addr":"10.20.193.164:6811/188912883","heartbeat_back_addr":"10.20.193.164:6815/188912883","heartbeat_front_addr":"10.20.193.164:6813/188912883","state":["exists","up"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6810","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6811","nonce":3960967243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6812","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6813","nonce":3960967243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6816","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6817","nonce":3960967243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6814","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6815","nonce":3960967243}]},"public_addr":"10.20.193.16:6811/3960967243","cluster_addr":"10.20.193.16:6813/3960967243","heartbeat_back_addr":"10.20.193.16:6817/3960967243","heartbeat_front_addr":"10.20.193.16:6815/3960967243","state":["exists","up"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6816","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6817","nonce":1434515412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6818","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6819","nonce":1434515412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6822","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6823","nonce":1434515412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6820","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6821","nonce":1434515412}]},"public_addr":"10.20.193.128:6817/1434515412","cluster_addr":"10.20.193.128:6819/1434515412","heartbeat_back_addr":"10.20.193.128:6823/1434515412","heartbeat_front_addr":"10.20.193.128:6821/1434515412","state":["exists","up"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6800","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6801","nonce":1855767579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6802","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6803","nonce":1855767579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6806","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6807","nonce":1855767579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6804","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6805","nonce":1855767579}]},"public_addr":"10.20.193.148:6801/1855767579","cluster_addr":"10.20.193.148:6803/1855767579","heartbeat_back_addr":"10.20.193.148:6807/1855767579","heartbeat_front_addr":"10.20.193.148:6805/1855767579","state":["exists","up"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6800","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6801","nonce":1881720827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6802","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6803","nonce":1881720827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6806","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6807","nonce":1881720827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6804","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6805","nonce":1881720827}]},"public_addr":"10.20.193.164:6801/1881720827","cluster_addr":"10.20.193.164:6803/1881720827","heartbeat_back_addr":"10.20.193.164:6807/1881720827","heartbeat_front_addr":"10.20.193.164:6805/1881720827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.650447+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.248604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:27.916170+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.151671+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.881661+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.409956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:28.238512+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.558291+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.920088+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.224894+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:27.660706+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.933443+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.293415+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.603293+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:28.448673+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.312258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:56:46.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 10 librados: watch_flush enter 2026-02-17T05:56:46.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 10 librados: watch_flush exit 2026-02-17T05:56:46.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 20 client.4368.objecter shutdown clearing up homeless session... 2026-02-17T05:56:46.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 10 client.4368.objecter successfully canceled tick 2026-02-17T05:56:46.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 -- 10.20.193.16:0/3588188717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea000791c0 msgr2=0x7fea0007b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea000791c0 0x7fea0007b680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fea301f4ee0 tx=0x7fea24007ef0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 10 monclient: shutdown 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 -- 10.20.193.16:0/3588188717 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30074dc0 msgr2=0x7fea3015ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30074dc0 0x7fea3015ec00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fea20009130 tx=0x7fea200047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 -- 10.20.193.16:0/3588188717 shutdown_connections 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea000791c0 0x7fea0007b680 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea301561e0 0x7fea301f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea30155e00 0x7fea3014c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 --2- 10.20.193.16:0/3588188717 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea30074dc0 0x7fea3015ec00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 -- 10.20.193.16:0/3588188717 >> 10.20.193.16:0/3588188717 conn(0x7fea300ce6c0 msgr2=0x7fea300cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 -- 10.20.193.16:0/3588188717 shutdown_connections 2026-02-17T05:56:46.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.349+0000 7fea378a9640 1 -- 10.20.193.16:0/3588188717 wait complete. 2026-02-17T05:56:46.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.350+0000 7fea378a9640 1 librados: shutdown 2026-02-17T05:56:46.359 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-17T05:56:46.359 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T05:56:46.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.429+0000 7f6e3377e640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.429+0000 7f6e3377e640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.429+0000 7f6e3377e640 10 monclient: monmap: 2026-02-17T05:56:46.430 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.430 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.430 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.430072+0000 2026-02-17T05:56:46.430 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.430072+0000 2026-02-17T05:56:46.430 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.430 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.431 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.431 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.431 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.431 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.431 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.431 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.431 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.429+0000 7f6e3377e640 1 Processor -- start 2026-02-17T05:56:46.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.429+0000 7f6e3377e640 1 -- start start 2026-02-17T05:56:46.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.429+0000 7f6e3377e640 10 monclient: init 2026-02-17T05:56:46.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.429+0000 7f6e3377e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.429+0000 7f6e3377e640 10 monclient: _add_conns ranks=[1,2,4,0,5,3] 2026-02-17T05:56:46.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.429+0000 7f6e3377e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f6e2c14f020 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:46.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.429+0000 7f6e3377e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6e2c14f400 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.430+0000 7f6e3377e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f6e2c15caf0 0x7f6e2c0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.430+0000 7f6e3377e640 10 monclient(hunting): picked mon.noname-b con 0x7f6e2c15caf0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:46.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.430+0000 7f6e3377e640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2c0c5b10 con 0x7f6e2c14f020 2026-02-17T05:56:46.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.430+0000 7f6e3377e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2c06e620 con 0x7f6e2c14f400 2026-02-17T05:56:46.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.430+0000 7f6e3377e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.430+0000 7f6e3377e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f6e2c06e900 con 0x7f6e2c15caf0 2026-02-17T05:56:46.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.430+0000 7f6e3377e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.430+0000 7f6e3377e640 10 monclient(hunting): authenticate will time out at 1058.999023s 2026-02-17T05:56:46.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.446+0000 7f6e30cf2640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f6e2c14f400 0x7f6e2c15c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46398/0 (socket says 10.20.193.16:46398) 2026-02-17T05:56:46.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.446+0000 7f6e30cf2640 1 -- 10.20.193.16:0/950169827 learned_addr learned my addr 10.20.193.16:0/950169827 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:46.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.446+0000 7f6e31cf4640 1 --2- 10.20.193.16:0/950169827 >> v2:10.20.193.128:3300/0 conn(0x7f6e2c15caf0 0x7f6e2c0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.447+0000 7f6e31cf4640 10 monclient(hunting): get_auth_request con 0x7f6e2c15caf0 auth_method 0 2026-02-17T05:56:46.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.447+0000 7f6e31cf4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.447+0000 7f6e31cf4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.447+0000 7f6e31cf4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.447+0000 7f6e31cf4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.447+0000 7f6e31cf4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.447+0000 7f6e31cf4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.448+0000 7f6e31cf4640 10 monclient(hunting): handle_auth_done global_id 4402 payload 274 2026-02-17T05:56:46.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.448+0000 7f6e31cf4640 1 -- 10.20.193.16:0/950169827 >> v1:10.20.193.148:6789/0 conn(0x7f6e2c14f400 legacy=0x7f6e2c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.448+0000 7f6e31cf4640 1 -- 10.20.193.16:0/950169827 >> v1:10.20.193.128:6789/0 conn(0x7f6e2c14f020 legacy=0x7f6e2c06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.448+0000 7f6e31cf4640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.448+0000 7f6e31cf4640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.448+0000 7f6e31cf4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.448+0000 7f6e31cf4640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.448+0000 7f6e31cf4640 1 -- 10.20.193.16:0/950169827 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c06ebe0 con 0x7f6e2c15caf0 2026-02-17T05:56:46.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.450+0000 7f6e31cf4640 1 --2- 10.20.193.16:0/950169827 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e2c15caf0 0x7f6e2c0c6010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009d90 tx=0x7f6e2802f2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947f64739d3c87c0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.453+0000 7f6e1bfff640 1 -- 10.20.193.16:0/950169827 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2803c070 con 0x7f6e2c15caf0 2026-02-17T05:56:46.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.453+0000 7f6e1bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.453+0000 7f6e1bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.453+0000 7f6e1bfff640 10 monclient: dump: 2026-02-17T05:56:46.453 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.453 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.453+0000 7f6e1bfff640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.453+0000 7f6e1bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.453+0000 7f6e1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.453+0000 7f6e3377e640 5 monclient: authenticate success, global_id 4402 2026-02-17T05:56:46.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.453+0000 7f6e3377e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:46.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.454+0000 7f6e1bfff640 1 -- 10.20.193.16:0/950169827 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e28037440 con 0x7f6e2c15caf0 2026-02-17T05:56:46.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.454+0000 7f6e1bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.454+0000 7f6e1bfff640 1 -- 10.20.193.16:0/950169827 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28037810 con 0x7f6e2c15caf0 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.454+0000 7f6e1bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.454+0000 7f6e1bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.454+0000 7f6e1bfff640 10 monclient: dump: 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.457 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.454+0000 7f6e3377e640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.454+0000 7f6e3377e640 4 set_mon_vals no callback set 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 10 monclient: shutdown 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 1 -- 10.20.193.16:0/950169827 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e2c15caf0 msgr2=0x7f6e2c0c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 1 --2- 10.20.193.16:0/950169827 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e2c15caf0 0x7f6e2c0c6010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009d90 tx=0x7f6e2802f2c0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 1 -- 10.20.193.16:0/950169827 shutdown_connections 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 1 --2- 10.20.193.16:0/950169827 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e2c15caf0 0x7f6e2c0c6010 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 1 -- 10.20.193.16:0/950169827 >> 10.20.193.16:0/950169827 conn(0x7f6e2c0cd770 msgr2=0x7f6e2c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 1 -- 10.20.193.16:0/950169827 shutdown_connections 2026-02-17T05:56:46.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 1 -- 10.20.193.16:0/950169827 wait complete. 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 10 monclient: monmap: 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.459 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 1 librados: starting msgr at 2026-02-17T05:56:46.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.455+0000 7f6e3377e640 1 librados: starting objecter 2026-02-17T05:56:46.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 1 Processor -- start 2026-02-17T05:56:46.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 1 -- start start 2026-02-17T05:56:46.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 1 librados: setting wanted keys 2026-02-17T05:56:46.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 1 librados: calling monclient init 2026-02-17T05:56:46.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient: init 2026-02-17T05:56:46.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e2c14f020 0x7f6e2c1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient(hunting): picked mon.noname-b con 0x7f6e2c14f020 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e2c14f400 0x7f6e2c15b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient(hunting): picked mon.noname-c con 0x7f6e2c14f400 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e2c15caf0 0x7f6e2c15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient(hunting): picked mon.noname-a con 0x7f6e2c15caf0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c06e2f0 con 0x7f6e2c15caf0 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c06f1a0 con 0x7f6e2c14f020 2026-02-17T05:56:46.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c0c6550 con 0x7f6e2c14f400 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.456+0000 7f6e3377e640 10 monclient(hunting): authenticate will time out at 1059.025737s 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e314f3640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e2c14f020 0x7f6e2c1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e30cf2640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e2c14f400 0x7f6e2c15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e314f3640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e2c14f020 0x7f6e2c1f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:49992/0 (socket says 10.20.193.16:49992) 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e314f3640 1 -- 10.20.193.16:0/580725795 learned_addr learned my addr 10.20.193.16:0/580725795 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e314f3640 10 monclient(hunting): get_auth_request con 0x7f6e2c14f020 auth_method 0 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e314f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e314f3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e314f3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e30cf2640 10 monclient(hunting): get_auth_request con 0x7f6e2c14f400 auth_method 0 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e30cf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e30cf2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e30cf2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e30cf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e30cf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e30cf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e314f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e314f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.457+0000 7f6e314f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.458+0000 7f6e30cf2640 10 monclient(hunting): handle_auth_done global_id 4433 payload 824 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.458+0000 7f6e30cf2640 1 -- 10.20.193.16:0/580725795 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e2c14f020 msgr2=0x7f6e2c1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.458+0000 7f6e30cf2640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e2c14f020 0x7f6e2c1f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.458+0000 7f6e30cf2640 1 -- 10.20.193.16:0/580725795 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e2c15caf0 msgr2=0x7f6e2c15bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.458+0000 7f6e30cf2640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e2c15caf0 0x7f6e2c15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.458+0000 7f6e30cf2640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.458+0000 7f6e30cf2640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.458+0000 7f6e30cf2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.458+0000 7f6e30cf2640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.458+0000 7f6e30cf2640 1 -- 10.20.193.16:0/580725795 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c15a6d0 con 0x7f6e2c14f400 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.459+0000 7f6e30cf2640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e2c14f400 0x7f6e2c15b770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000c530 tx=0x7f6e2000ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.460+0000 7f6e19ffb640 10 client.?.objecter ms_handle_connect 0x7f6e2c14f400 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.460+0000 7f6e19ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 1 -- 10.20.193.16:0/580725795 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20017070 con 0x7f6e2c14f400 2026-02-17T05:56:46.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 10 monclient: dump: 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 1 -- 10.20.193.16:0/580725795 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e20005de0 con 0x7f6e2c14f400 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e3377e640 5 monclient: authenticate success, global_id 4433 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e3377e640 10 monclient: _renew_subs 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e3377e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e3377e640 1 -- 10.20.193.16:0/580725795 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c15a9b0 con 0x7f6e2c14f400 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 1 -- 10.20.193.16:0/580725795 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200072b0 con 0x7f6e2c14f400 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e19ffb640 10 monclient: dump: 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e1b7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:46.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e3377e640 10 client.4433.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e3377e640 10 monclient: _renew_subs 2026-02-17T05:56:46.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e3377e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e3377e640 1 -- 10.20.193.16:0/580725795 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c15af70 con 0x7f6e2c14f400 2026-02-17T05:56:46.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.461+0000 7f6e3377e640 1 librados: init done 2026-02-17T05:56:46.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.462+0000 7f6e3377e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.462+0000 7f6e3377e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.462+0000 7f6e3377e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.462+0000 7f6e3377e640 1 -- 10.20.193.16:0/580725795 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c0c70f0 con 0x7f6e2c14f400 2026-02-17T05:56:46.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.474+0000 7f6e19ffb640 1 -- 10.20.193.16:0/580725795 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f6e20007450 con 0x7f6e2c14f400 2026-02-17T05:56:46.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.475+0000 7f6e19ffb640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dfc079140 0x7f6dfc07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.475+0000 7f6e314f3640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dfc079140 0x7f6dfc07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.475+0000 7f6e314f3640 10 monclient: get_auth_request con 0x7f6dfc079140 auth_method 0 2026-02-17T05:56:46.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.475+0000 7f6e314f3640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dfc079140 0x7f6dfc07b600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0055f0 tx=0x7f6e1c002990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.475+0000 7f6e19ffb640 10 client.4433.objecter ms_handle_connect 0x7f6dfc079140 2026-02-17T05:56:46.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.475+0000 7f6e19ffb640 1 -- 10.20.193.16:0/580725795 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f6e20007770 con 0x7f6e2c14f400 2026-02-17T05:56:46.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.475+0000 7f6e19ffb640 10 client.4433.objecter ms_dispatch2 0x7f6e2c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.475+0000 7f6e19ffb640 3 client.4433.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.475+0000 7f6e19ffb640 3 client.4433.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.475+0000 7f6e19ffb640 20 client.4433.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.577+0000 7f6e19ffb640 1 -- 10.20.193.16:0/580725795 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6e20021020 con 0x7f6e2c14f400 2026-02-17T05:56:46.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.577+0000 7f6e19ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.577+0000 7f6e19ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:46.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.666+0000 7f6e3377e640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:46.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.666+0000 7f6e3377e640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:46.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.666+0000 7f6e3377e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.666+0000 7f6e3377e640 1 -- 10.20.193.16:0/580725795 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e2c14d9b0 con 0x7f6e2c14f400 2026-02-17T05:56:46.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.689+0000 7f6e19ffb640 1 -- 10.20.193.16:0/580725795 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+24767 (secure 0 0 0) 0x7f6e2007be20 con 0x7f6e2c14f400 2026-02-17T05:56:46.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.689+0000 7f6e19ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:56:46.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.689+0000 7f6e19ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:46.690 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:56:46.691 INFO:teuthology.orchestra.run.trial016.stdout:{"epoch":33,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:56:45.354534+0000","last_up_change":"2026-02-17T05:56:37.615172+0000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:56:30.047883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-17T05:56:41.588944+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":33,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0100002288818359,"score_stable":4.0100002288818359,"optimal_score":0.81000000238418579,"raw_score_acting":3.25,"raw_score_stable":3.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6826","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6827","nonce":821011033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6828","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6829","nonce":821011033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6832","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6833","nonce":821011033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6830","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6831","nonce":821011033}]},"public_addr":"10.20.193.16:6827/821011033","cluster_addr":"10.20.193.16:6829/821011033","heartbeat_back_addr":"10.20.193.16:6833/821011033","heartbeat_front_addr":"10.20.193.16:6831/821011033","state":["exists","up"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6807","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6809","nonce":1618049789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6810","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6811","nonce":1618049789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6814","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6815","nonce":1618049789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6812","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6813","nonce":1618049789}]},"public_addr":"10.20.193.128:6809/1618049789","cluster_addr":"10.20.193.128:6811/1618049789","heartbeat_back_addr":"10.20.193.128:6815/1618049789","heartbeat_front_addr":"10.20.193.128:6813/1618049789","state":["exists","up"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6816","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6817","nonce":1588035471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6818","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6819","nonce":1588035471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6822","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6823","nonce":1588035471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6820","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6821","nonce":1588035471}]},"public_addr":"10.20.193.148:6817/1588035471","cluster_addr":"10.20.193.148:6819/1588035471","heartbeat_back_addr":"10.20.193.148:6823/1588035471","heartbeat_front_addr":"10.20.193.148:6821/1588035471","state":["exists","up"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6824","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6825","nonce":3772404125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6826","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6827","nonce":3772404125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6830","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6831","nonce":3772404125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6828","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6829","nonce":3772404125}]},"public_addr":"10.20.193.164:6825/3772404125","cluster_addr":"10.20.193.164:6827/3772404125","heartbeat_back_addr":"10.20.193.164:6831/3772404125","heartbeat_front_addr":"10.20.193.164:6829/3772404125","state":["exists","up"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6818","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6819","nonce":1157611076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6820","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6821","nonce":1157611076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6824","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6825","nonce":1157611076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6822","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6823","nonce":1157611076}]},"public_addr":"10.20.193.16:6819/1157611076","cluster_addr":"10.20.193.16:6821/1157611076","heartbeat_back_addr":"10.20.193.16:6825/1157611076","heartbeat_front_addr":"10.20.193.16:6823/1157611076","state":["exists","up"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6824","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6825","nonce":1593443319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6826","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6827","nonce":1593443319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6830","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6831","nonce":1593443319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6828","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6829","nonce":1593443319}]},"public_addr":"10.20.193.128:6825/1593443319","cluster_addr":"10.20.193.128:6827/1593443319","heartbeat_back_addr":"10.20.193.128:6831/1593443319","heartbeat_front_addr":"10.20.193.128:6829/1593443319","state":["exists","up"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6824","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6825","nonce":2491860138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6826","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6827","nonce":2491860138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6830","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6831","nonce":2491860138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6828","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6829","nonce":2491860138}]},"public_addr":"10.20.193.148:6825/2491860138","cluster_addr":"10.20.193.148:6827/2491860138","heartbeat_back_addr":"10.20.193.148:6831/2491860138","heartbeat_front_addr":"10.20.193.148:6829/2491860138","state":["exists","up"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6816","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6817","nonce":3942637745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6818","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6819","nonce":3942637745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6822","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6823","nonce":3942637745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6820","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6821","nonce":3942637745}]},"public_addr":"10.20.193.164:6817/3942637745","cluster_addr":"10.20.193.164:6819/3942637745","heartbeat_back_addr":"10.20.193.164:6823/3942637745","heartbeat_front_addr":"10.20.193.164:6821/3942637745","state":["exists","up"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6802","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6803","nonce":15364298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6804","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6805","nonce":15364298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6808","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6809","nonce":15364298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6806","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6807","nonce":15364298}]},"public_addr":"10.20.193.16:6803/15364298","cluster_addr":"10.20.193.16:6805/15364298","heartbeat_back_addr":"10.20.193.16:6809/15364298","heartbeat_front_addr":"10.20.193.16:6807/15364298","state":["exists","up"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6800","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6801","nonce":1813385411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6802","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6803","nonce":1813385411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6806","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6808","nonce":1813385411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6804","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6805","nonce":1813385411}]},"public_addr":"10.20.193.128:6801/1813385411","cluster_addr":"10.20.193.128:6803/1813385411","heartbeat_back_addr":"10.20.193.128:6808/1813385411","heartbeat_front_addr":"10.20.193.128:6805/1813385411","state":["exists","up"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6808","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6809","nonce":2531105113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6810","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6811","nonce":2531105113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6814","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6815","nonce":2531105113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6812","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6813","nonce":2531105113}]},"public_addr":"10.20.193.148:6809/2531105113","cluster_addr":"10.20.193.148:6811/2531105113","heartbeat_back_addr":"10.20.193.148:6815/2531105113","heartbeat_front_addr":"10.20.193.148:6813/2531105113","state":["exists","up"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6808","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6809","nonce":188912883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6810","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6811","nonce":188912883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6814","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6815","nonce":188912883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6812","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6813","nonce":188912883}]},"public_addr":"10.20.193.164:6809/188912883","cluster_addr":"10.20.193.164:6811/188912883","heartbeat_back_addr":"10.20.193.164:6815/188912883","heartbeat_front_addr":"10.20.193.164:6813/188912883","state":["exists","up"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6810","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6811","nonce":3960967243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6812","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6813","nonce":3960967243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6816","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6817","nonce":3960967243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6814","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6815","nonce":3960967243}]},"public_addr":"10.20.193.16:6811/3960967243","cluster_addr":"10.20.193.16:6813/3960967243","heartbeat_back_addr":"10.20.193.16:6817/3960967243","heartbeat_front_addr":"10.20.193.16:6815/3960967243","state":["exists","up"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6816","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6817","nonce":1434515412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6818","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6819","nonce":1434515412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6822","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6823","nonce":1434515412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6820","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6821","nonce":1434515412}]},"public_addr":"10.20.193.128:6817/1434515412","cluster_addr":"10.20.193.128:6819/1434515412","heartbeat_back_addr":"10.20.193.128:6823/1434515412","heartbeat_front_addr":"10.20.193.128:6821/1434515412","state":["exists","up"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6800","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6801","nonce":1855767579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6802","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6803","nonce":1855767579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6806","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6807","nonce":1855767579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6804","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6805","nonce":1855767579}]},"public_addr":"10.20.193.148:6801/1855767579","cluster_addr":"10.20.193.148:6803/1855767579","heartbeat_back_addr":"10.20.193.148:6807/1855767579","heartbeat_front_addr":"10.20.193.148:6805/1855767579","state":["exists","up"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6800","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6801","nonce":1881720827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6802","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6803","nonce":1881720827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6806","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6807","nonce":1881720827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6804","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6805","nonce":1881720827}]},"public_addr":"10.20.193.164:6801/1881720827","cluster_addr":"10.20.193.164:6803/1881720827","heartbeat_back_addr":"10.20.193.164:6807/1881720827","heartbeat_front_addr":"10.20.193.164:6805/1881720827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.650447+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.248604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:27.916170+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.151671+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.881661+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.409956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:28.238512+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.558291+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.920088+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.224894+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:27.660706+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.933443+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.293415+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.603293+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:28.448673+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.312258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:56:46.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 10 librados: watch_flush enter 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 10 librados: watch_flush exit 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 20 client.4433.objecter shutdown clearing up homeless session... 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 10 client.4433.objecter successfully canceled tick 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 -- 10.20.193.16:0/580725795 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dfc079140 msgr2=0x7f6dfc07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dfc079140 0x7f6dfc07b600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0055f0 tx=0x7f6e1c002990 comp rx=0 tx=0).stop 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 10 monclient: shutdown 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 -- 10.20.193.16:0/580725795 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e2c14f400 msgr2=0x7f6e2c15b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e2c14f400 0x7f6e2c15b770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000c530 tx=0x7f6e2000ca00 comp rx=0 tx=0).stop 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 -- 10.20.193.16:0/580725795 shutdown_connections 2026-02-17T05:56:46.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dfc079140 0x7f6dfc07b600 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e2c15caf0 0x7f6e2c15bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e2c14f400 0x7f6e2c15b770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 --2- 10.20.193.16:0/580725795 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e2c14f020 0x7f6e2c1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 -- 10.20.193.16:0/580725795 >> 10.20.193.16:0/580725795 conn(0x7f6e2c0cd770 msgr2=0x7f6e2c14ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 -- 10.20.193.16:0/580725795 shutdown_connections 2026-02-17T05:56:46.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 -- 10.20.193.16:0/580725795 wait complete. 2026-02-17T05:56:46.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.691+0000 7f6e3377e640 1 librados: shutdown 2026-02-17T05:56:46.702 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-17T05:56:46.703 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-17T05:56:46.703 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-17T05:56:46.703 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-17T05:56:46.703 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2026-02-17T05:56:46.703 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2026-02-17T05:56:46.704 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2026-02-17T05:56:46.704 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2026-02-17T05:56:46.704 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2026-02-17T05:56:46.704 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2026-02-17T05:56:46.704 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2026-02-17T05:56:46.704 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2026-02-17T05:56:46.704 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2026-02-17T05:56:46.705 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2026-02-17T05:56:46.705 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2026-02-17T05:56:46.705 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2026-02-17T05:56:46.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f20a8d85640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f20a8d85640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f20a8d85640 10 monclient: monmap: 2026-02-17T05:56:46.788 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.788 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.788 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.788162+0000 2026-02-17T05:56:46.788 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.788162+0000 2026-02-17T05:56:46.788 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.788 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.788 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.788 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.789 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.789 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.789 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.789 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.789 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f1f1f424640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f1f1f424640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f1f1f424640 10 monclient: monmap: 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.788495+0000 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.788495+0000 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.790 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.791 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.791 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.791 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f20a8d85640 1 Processor -- start 2026-02-17T05:56:46.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f20a8d85640 1 -- start start 2026-02-17T05:56:46.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f20a8d85640 10 monclient: init 2026-02-17T05:56:46.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f804332c640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f804332c640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.787+0000 7f804332c640 10 monclient: monmap: 2026-02-17T05:56:46.792 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.792 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.792 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.788886+0000 2026-02-17T05:56:46.792 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.788886+0000 2026-02-17T05:56:46.793 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.793 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.793 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.793 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.793 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.793 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.793 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.793 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.793 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f1f1f424640 1 Processor -- start 2026-02-17T05:56:46.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f1f1f424640 1 -- start start 2026-02-17T05:56:46.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f1f1f424640 10 monclient: init 2026-02-17T05:56:46.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 10 monclient: _add_conns ranks=[1,4,0,3,2,5] 2026-02-17T05:56:46.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f20a414e830 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:46.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f20a40c7d00 0x7f20a406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 10 monclient(hunting): picked mon.noname-b con 0x7f20a40c7d00 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:46.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f20a414ec10 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:46.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a40c81f0 con 0x7f20a414ec10 2026-02-17T05:56:46.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a40c84d0 con 0x7f20a414e830 2026-02-17T05:56:46.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f20a4159c30 con 0x7f20a40c7d00 2026-02-17T05:56:46.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f20a8d85640 10 monclient(hunting): authenticate will time out at 1059.357565s 2026-02-17T05:56:46.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.788+0000 7f804332c640 1 Processor -- start 2026-02-17T05:56:46.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 1 -- start start 2026-02-17T05:56:46.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 10 monclient: init 2026-02-17T05:56:46.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 10 monclient: _add_conns ranks=[3,0,2,4,1,5] 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1f18153a10 0x7f1f1815db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 10 monclient(hunting): picked mon.noname-a con 0x7f1f18153a10 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1f18070df0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1f18070390 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f18159de0 con 0x7f1f18070df0 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1815a0c0 con 0x7f1f18070390 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1f1806f7f0 con 0x7f1f18153a10 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1f424640 10 monclient(hunting): authenticate will time out at 1059.358190s 2026-02-17T05:56:46.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f20a2575640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f20a414e830 0x7f20a4159520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:53726/0 (socket says 10.20.193.16:53726) 2026-02-17T05:56:46.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f20a1d74640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f20a40c7d00 0x7f20a406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f20a2575640 1 -- 10.20.193.16:0/2421196733 learned_addr learned my addr 10.20.193.16:0/2421196733 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:46.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f20a1d74640 10 monclient(hunting): get_auth_request con 0x7f20a40c7d00 auth_method 0 2026-02-17T05:56:46.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f20a1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f20a1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f20a1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1d99a640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f1f18070390 0x7f1f180c8d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46402/0 (socket says 10.20.193.16:46402) 2026-02-17T05:56:46.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f1f1d99a640 1 -- 10.20.193.16:0/2389521453 learned_addr learned my addr 10.20.193.16:0/2389521453 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:46.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 10 monclient: _add_conns ranks=[5,0,1,3,4,2] 2026-02-17T05:56:46.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f803c155fa0 0x7f803c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 10 monclient(hunting): picked mon.noname-c con 0x7f803c155fa0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:46.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f803c156380 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:46.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f20a1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f20a1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f803c15d360 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:46.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c0ccae0 con 0x7f803c156380 2026-02-17T05:56:46.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c0ccdc0 con 0x7f803c15d360 2026-02-17T05:56:46.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f803c0cd9d0 con 0x7f803c155fa0 2026-02-17T05:56:46.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f804332c640 10 monclient(hunting): authenticate will time out at 1059.358945s 2026-02-17T05:56:46.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.789+0000 7f20a1d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.790+0000 7f1f1d199640 1 --2- 10.20.193.16:0/2389521453 >> v2:10.20.193.16:3300/0 conn(0x7f1f18153a10 0x7f1f1815db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.790+0000 7f80410a1640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f803c155fa0 0x7f803c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.790+0000 7f1f1d199640 10 monclient(hunting): get_auth_request con 0x7f1f18153a10 auth_method 0 2026-02-17T05:56:46.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.790+0000 7f1f1d199640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.790+0000 7f1f1d199640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.790+0000 7f1f1d199640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f80410a1640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f803c155fa0 0x7f803c0cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56784/0 (socket says 10.20.193.16:56784) 2026-02-17T05:56:46.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f80410a1640 1 -- 10.20.193.16:0/2866060630 learned_addr learned my addr 10.20.193.16:0/2866060630 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f80410a1640 10 monclient(hunting): get_auth_request con 0x7f803c155fa0 auth_method 0 2026-02-17T05:56:46.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f80410a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f80410a1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f80410a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f20a1d74640 10 monclient(hunting): handle_auth_done global_id 4408 payload 274 2026-02-17T05:56:46.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f20a1d74640 1 -- 10.20.193.16:0/2421196733 >> v1:10.20.193.128:6789/0 conn(0x7f20a414e830 legacy=0x7f20a4159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f20a1d74640 1 -- 10.20.193.16:0/2421196733 >> v1:10.20.193.16:6789/0 conn(0x7f20a414ec10 legacy=0x7f20a406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4096750844 0 0) 0x7f1f1815a0c0 con 0x7f1f18070390 2026-02-17T05:56:46.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f1f07fff640 10 monclient(hunting): _negotiate 2026-02-17T05:56:46.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f1f07fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f1f07fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f1f07fff640 10 monclient(hunting): my global_id is 4436 2026-02-17T05:56:46.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f20a1d74640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f20a1d74640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f00003620 con 0x7f1f18070390 2026-02-17T05:56:46.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f20a1d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f20a1d74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f20a1d74640 1 -- 10.20.193.16:0/2421196733 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a40c87b0 con 0x7f20a40c7d00 2026-02-17T05:56:46.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f1f1d199640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f1f1d199640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.791+0000 7f1f1d199640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.792+0000 7f80410a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.792+0000 7f80410a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.792+0000 7f80410a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.792+0000 7f7ebf67c640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.792+0000 7f7ebf67c640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.792+0000 7f7ebf67c640 10 monclient: monmap: 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.793754+0000 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.793754+0000 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.809 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.810 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.810 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.810 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.810 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.810 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2931645519 0 0) 0x7f1f18159de0 con 0x7f1f18070df0 2026-02-17T05:56:46.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f1f07fff640 10 monclient(hunting): _negotiate 2026-02-17T05:56:46.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f1f07fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f7ebf67c640 1 Processor -- start 2026-02-17T05:56:46.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f1f07fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f1f07fff640 10 monclient(hunting): my global_id is 4374 2026-02-17T05:56:46.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f1815a0c0 con 0x7f1f18070df0 2026-02-17T05:56:46.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f20a1d74640 1 --2- 10.20.193.16:0/2421196733 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20a40c7d00 0x7f20a406d6e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f208c0098a0 tx=0x7f208c02eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b10bad9428861f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f0e06bb6640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f0e06bb6640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f0e06bb6640 10 monclient: monmap: 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.794729+0000 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.794729+0000 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.813 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.814 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.814 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f7ebf67c640 1 -- start start 2026-02-17T05:56:46.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.793+0000 7f7ebf67c640 10 monclient: init 2026-02-17T05:56:46.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f80410a1640 10 monclient(hunting): handle_auth_done global_id 4439 payload 274 2026-02-17T05:56:46.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f80410a1640 1 -- 10.20.193.16:0/2866060630 >> v1:10.20.193.128:6789/0 conn(0x7f803c15d360 legacy=0x7f803c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f80410a1640 1 -- 10.20.193.16:0/2866060630 >> v1:10.20.193.16:6789/0 conn(0x7f803c156380 legacy=0x7f803c0740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f80410a1640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f80410a1640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f80410a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f80410a1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f80410a1640 1 -- 10.20.193.16:0/2866060630 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c0cd0a0 con 0x7f803c155fa0 2026-02-17T05:56:46.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f0e06bb6640 1 Processor -- start 2026-02-17T05:56:46.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f0e06bb6640 1 -- start start 2026-02-17T05:56:46.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f0e06bb6640 10 monclient: init 2026-02-17T05:56:46.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7fdd4fa8f640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7fdd4fa8f640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7fdd4fa8f640 10 monclient: monmap: 2026-02-17T05:56:46.816 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.816 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.816 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.795409+0000 2026-02-17T05:56:46.816 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.795409+0000 2026-02-17T05:56:46.816 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.816 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 <== mon.2 v1:10.20.193.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2061995790 0 0) 0x7f1f00003620 con 0x7f1f18070390 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 >> v2:10.20.193.16:3300/0 conn(0x7f1f18153a10 msgr2=0x7f1f1815db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 --2- 10.20.193.16:0/2389521453 >> v2:10.20.193.16:3300/0 conn(0x7f1f18153a10 0x7f1f1815db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 >> v1:10.20.193.16:6789/0 conn(0x7f1f18070df0 legacy=0x7f1f1806fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.148:6789/0 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 --> v1:10.20.193.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1815a3a0 con 0x7f1f18070390 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 <== mon.2 v1:10.20.193.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f1f14003280 con 0x7f1f18070390 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 10 monclient: dump: 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.818 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] but i'm connected to v1:10.20.193.148:6789/0, reconnecting 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 >> v1:10.20.193.148:6789/0 conn(0x7f1f18070390 legacy=0x7f1f180c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 10 monclient: _add_conns ranks=[3,0,4,1,2,5] 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f7eb8155e00 0x7f7eb80ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 10 monclient(hunting): picked mon.noname-a con 0x7f7eb8155e00 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7eb81561e0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f7eb8074dc0 0x7f7eb80cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 10 monclient(hunting): picked mon.noname-b con 0x7f7eb8074dc0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb80c96d0 con 0x7f7eb81561e0 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb8074860 con 0x7f7eb8155e00 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb80c9940 con 0x7f7eb8074dc0 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f7ebf67c640 10 monclient(hunting): authenticate will time out at 1059.363827s 2026-02-17T05:56:46.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:46.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 --2- 10.20.193.16:0/2389521453 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f00003dd0 0x7f1f000041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 10 monclient(hunting): picked mon.c con 0x7f1f00003dd0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.794+0000 7f1f07fff640 1 --2- 10.20.193.16:0/2389521453 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f00004900 0x7f1f00004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 10 monclient(hunting): picked mon.b con 0x7f1f00004900 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 1 --2- 10.20.193.16:0/2389521453 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f00005380 0x7f1f0000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 10 monclient(hunting): picked mon.a con 0x7f1f00005380 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1806f7f0 con 0x7f1f00005380 2026-02-17T05:56:46.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0000c2c0 con 0x7f1f00004900 2026-02-17T05:56:46.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0000c4c0 con 0x7f1f00003dd0 2026-02-17T05:56:46.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f1f07fff640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:46.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 10 monclient: _add_conns ranks=[5,4,1,3,2,0] 2026-02-17T05:56:46.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f0e00155fa0 0x7f0e000cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 10 monclient(hunting): picked mon.noname-c con 0x7f0e00155fa0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:46.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f0e00156380 0x7f0e000740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7fdd4fa8f640 1 Processor -- start 2026-02-17T05:56:46.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 10 monclient(hunting): picked mon.noname-b con 0x7f0e00156380 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:46.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0e000745e0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:46.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e000ccae0 con 0x7f0e000745e0 2026-02-17T05:56:46.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f0e000cd9d0 con 0x7f0e00156380 2026-02-17T05:56:46.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f0e000ccdc0 con 0x7f0e00155fa0 2026-02-17T05:56:46.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7f0e06bb6640 10 monclient(hunting): authenticate will time out at 1059.364215s 2026-02-17T05:56:46.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7fdd4fa8f640 1 -- start start 2026-02-17T05:56:46.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.795+0000 7fdd4fa8f640 10 monclient: init 2026-02-17T05:56:46.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 1 -- 10.20.193.16:0/2421196733 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c03c070 con 0x7f20a40c7d00 2026-02-17T05:56:46.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 10 monclient: dump: 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 1 -- 10.20.193.16:0/2421196733 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f208c02fb70 con 0x7f20a40c7d00 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 1 -- 10.20.193.16:0/2421196733 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c037440 con 0x7f20a40c7d00 2026-02-17T05:56:46.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a1573640 10 monclient: dump: 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a8d85640 5 monclient: authenticate success, global_id 4408 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a8d85640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a8d85640 4 set_mon_vals no callback set 2026-02-17T05:56:46.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f1f1d199640 1 --2- 10.20.193.16:0/2389521453 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f00003dd0 0x7f1f000041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f1f1d199640 10 monclient(hunting): get_auth_request con 0x7f1f00003dd0 auth_method 0 2026-02-17T05:56:46.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f1f1d199640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f1f1d199640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f1f1d199640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:46.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 10 monclient: _add_conns ranks=[2,3,4,0,1,5] 2026-02-17T05:56:46.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdd48155fa0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fdd48156380 0x7fdd480740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 10 monclient(hunting): picked mon.noname-a con 0x7fdd48156380 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fdd480745e0 0x7fdd4815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 10 monclient(hunting): picked mon.noname-b con 0x7fdd480745e0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd480ccae0 con 0x7fdd48155fa0 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fdd480cc8b0 con 0x7fdd48156380 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fdd480cd9d0 con 0x7fdd480745e0 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7fdd4fa8f640 10 monclient(hunting): authenticate will time out at 1059.365792s 2026-02-17T05:56:46.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f0e0492b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f0e00155fa0 0x7f0e000cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a8d85640 10 monclient: shutdown 2026-02-17T05:56:46.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a8d85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a8d85640 1 -- 10.20.193.16:0/2421196733 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20a40c7d00 msgr2=0x7f20a406d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a8d85640 1 --2- 10.20.193.16:0/2421196733 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20a40c7d00 0x7f20a406d6e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f208c0098a0 tx=0x7f208c02eeb0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a8d85640 1 -- 10.20.193.16:0/2421196733 shutdown_connections 2026-02-17T05:56:46.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.796+0000 7f20a8d85640 1 --2- 10.20.193.16:0/2421196733 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20a40c7d00 0x7f20a406d6e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f20a8d85640 1 -- 10.20.193.16:0/2421196733 >> 10.20.193.16:0/2421196733 conn(0x7f20a40cd770 msgr2=0x7f20a40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f20a8d85640 1 -- 10.20.193.16:0/2421196733 shutdown_connections 2026-02-17T05:56:46.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f20a8d85640 1 -- 10.20.193.16:0/2421196733 wait complete. 2026-02-17T05:56:46.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f80410a1640 1 --2- 10.20.193.16:0/2866060630 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c155fa0 0x7f803c0cc1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f803c0cdb50 tx=0x7f802c02eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f6690d90fcf0a4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f1f1d199640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f1f1d199640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f1f1d199640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:46.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f0e0492b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f0e00155fa0 0x7f0e000cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56802/0 (socket says 10.20.193.16:56802) 2026-02-17T05:56:46.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f7ebdbf2640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f7eb8074dc0 0x7f7eb80cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f0e0492b640 1 -- 10.20.193.16:0/1311833023 learned_addr learned my addr 10.20.193.16:0/1311833023 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f0e0492b640 10 monclient(hunting): get_auth_request con 0x7f0e00155fa0 auth_method 0 2026-02-17T05:56:46.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f1f1c998640 1 --2- 10.20.193.16:0/2389521453 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f00004900 0x7f1f00004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f1f1c998640 10 monclient(hunting): get_auth_request con 0x7f1f00004900 auth_method 0 2026-02-17T05:56:46.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f1f1c998640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f1f1c998640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f1f1c998640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:46.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f0e0492b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f0e0492b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f0e0492b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f7ebdbf2640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f7eb8074dc0 0x7f7eb80cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50024/0 (socket says 10.20.193.16:50024) 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f7ebdbf2640 1 -- 10.20.193.16:0/2467910961 learned_addr learned my addr 10.20.193.16:0/2467910961 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f7ebdbf2640 10 monclient(hunting): get_auth_request con 0x7f7eb8074dc0 auth_method 0 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f7ebdbf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f7ebdbf2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f7ebdbf2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f20a8d85640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7fa1ea8f2640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7fa1ea8f2640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7fa1ea8f2640 10 monclient: monmap: 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.798770+0000 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.798770+0000 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f20a8d85640 10 monclient: monmap: 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7fdd4e005640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fdd480745e0 0x7fdd4815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f0e0492b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f0e0492b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.797+0000 7f0e0492b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fdd4e005640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fdd480745e0 0x7fdd4815d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50042/0 (socket says 10.20.193.16:50042) 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fdd4e005640 1 -- 10.20.193.16:0/3021793843 learned_addr learned my addr 10.20.193.16:0/3021793843 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fdd4e005640 10 monclient(hunting): get_auth_request con 0x7fdd480745e0 auth_method 0 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fdd4e005640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fdd4e005640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fdd4e005640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f7ebdbf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f7ebdbf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f7ebdbf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f20a8d85640 1 librados: starting msgr at 2026-02-17T05:56:46.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f20a8d85640 1 librados: starting objecter 2026-02-17T05:56:46.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fa1ea8f2640 1 Processor -- start 2026-02-17T05:56:46.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f0df7fff640 1 --2- 10.20.193.16:0/1311833023 >> v2:10.20.193.128:3300/0 conn(0x7f0e00156380 0x7f0e000740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fa1ea8f2640 1 -- start start 2026-02-17T05:56:46.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fa1ea8f2640 10 monclient: init 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f20a8d85640 1 Processor -- start 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f20a8d85640 1 -- start start 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f20a8d85640 1 librados: setting wanted keys 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f20a8d85640 1 librados: calling monclient init 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f20a8d85640 10 monclient: init 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7ff9493d3640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7ff9493d3640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7ff9493d3640 10 monclient: monmap: 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.799655+0000 2026-02-17T05:56:46.838 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.799655+0000 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 10 monclient(hunting): handle_auth_done global_id 4436 payload 293 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 1 -- 10.20.193.16:0/2389521453 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f00004900 msgr2=0x7f1f00004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 1 --2- 10.20.193.16:0/2389521453 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f00004900 0x7f1f00004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 1 -- 10.20.193.16:0/2389521453 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f00005380 msgr2=0x7f1f0000bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 1 --2- 10.20.193.16:0/2389521453 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f00005380 0x7f1f0000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 1 monclient: found mon.c 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d99a640 1 -- 10.20.193.16:0/2389521453 reap_dead start 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 1 -- 10.20.193.16:0/2389521453 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f00003620 con 0x7f1f00003dd0 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1d199640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1f424640 5 monclient: authenticate success, global_id 4436 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f1f1f424640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fdd4e005640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fdd4e005640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7fdd4e005640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.798+0000 7f20a8d85640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:46.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f0df7fff640 10 monclient(hunting): get_auth_request con 0x7f0e00156380 auth_method 0 2026-02-17T05:56:46.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f0df7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f0df7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f0df7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20a40c7d00 0x7f20a4143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 10 monclient: _add_conns ranks=[0,5,4,3,2,1] 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa1e4155e00 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fa1e41561e0 0x7fa1e40ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 10 monclient(hunting): picked mon.noname-c con 0x7fa1e41561e0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fa1e4074dc0 0x7fa1e40cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 10 monclient(hunting): picked mon.noname-b con 0x7fa1e4074dc0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e40c96d0 con 0x7fa1e4155e00 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 1 Processor -- start 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 10 monclient(hunting): picked mon.noname-a con 0x7f20a40c7d00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e4074860 con 0x7fa1e4074dc0 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e40c9940 con 0x7fa1e41561e0 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7fa1ea8f2640 10 monclient(hunting): authenticate will time out at 1059.368247s 2026-02-17T05:56:46.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 1 -- start start 2026-02-17T05:56:46.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 10 monclient: init 2026-02-17T05:56:46.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20a414e830 0x7f20a40c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 10 monclient(hunting): picked mon.noname-c con 0x7f20a414e830 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20a414ec10 0x7f20a40c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 10 monclient(hunting): picked mon.noname-b con 0x7f20a414ec10 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20a40c9050 con 0x7f20a40c7d00 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f20a40c8d70 con 0x7f20a414ec10 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20a406ee80 con 0x7f20a414e830 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f20a8d85640 10 monclient(hunting): authenticate will time out at 1059.368058s 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 10 monclient: _add_conns ranks=[4,3,2,0,5,1] 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7ff944155e00 0x7ff9440ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 10 monclient(hunting): picked mon.noname-b con 0x7ff944155e00 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7ff9441561e0 0x7ff9440cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 10 monclient(hunting): picked mon.noname-a con 0x7ff9441561e0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff944074dc0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9440c96d0 con 0x7ff944074dc0 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff944074860 con 0x7ff9441561e0 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7ff9440c9940 con 0x7ff944155e00 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7ff9493d3640 10 monclient(hunting): authenticate will time out at 1059.368910s 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f7ebdbf2640 10 monclient(hunting): handle_auth_done global_id 4417 payload 274 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f7ebdbf2640 1 -- 10.20.193.16:0/2467910961 >> v2:10.20.193.16:3300/0 conn(0x7f7eb8155e00 msgr2=0x7f7eb80ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f7ebdbf2640 1 --2- 10.20.193.16:0/2467910961 >> v2:10.20.193.16:3300/0 conn(0x7f7eb8155e00 0x7f7eb80ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f7ebdbf2640 1 -- 10.20.193.16:0/2467910961 >> v1:10.20.193.16:6789/0 conn(0x7f7eb81561e0 legacy=0x7f7eb8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f7ebdbf2640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f7ebdbf2640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f7ebdbf2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f7ebdbf2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.799+0000 7f7ebdbf2640 1 -- 10.20.193.16:0/2467910961 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb80c9c20 con 0x7f7eb8074dc0 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f0e0492b640 10 monclient(hunting): handle_auth_done global_id 4442 payload 274 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f0e0492b640 1 -- 10.20.193.16:0/1311833023 >> v2:10.20.193.128:3300/0 conn(0x7f0e00156380 msgr2=0x7f0e000740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f0e0492b640 1 --2- 10.20.193.16:0/1311833023 >> v2:10.20.193.128:3300/0 conn(0x7f0e00156380 0x7f0e000740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f0e0492b640 1 -- 10.20.193.16:0/1311833023 >> v1:10.20.193.128:6789/0 conn(0x7f0e000745e0 legacy=0x7f0e0015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f0e0492b640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f0e0492b640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f0e0492b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f0e0492b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f0e0492b640 1 -- 10.20.193.16:0/1311833023 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e000cd0a0 con 0x7f0e00155fa0 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f3bbe10a640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f3bbe10a640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f3bbe10a640 10 monclient: monmap: 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.846 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.801446+0000 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.801446+0000 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f6e8695b640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f6e8695b640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f6e8695b640 10 monclient: monmap: 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.801434+0000 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.801434+0000 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.847 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7f3bbe10a640 1 Processor -- start 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7fdd4e005640 10 monclient(hunting): handle_auth_done global_id 4420 payload 274 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7fdd4e005640 1 -- 10.20.193.16:0/3021793843 >> v2:10.20.193.16:3300/0 conn(0x7fdd48156380 msgr2=0x7fdd480740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7fdd4e005640 1 --2- 10.20.193.16:0/3021793843 >> v2:10.20.193.16:3300/0 conn(0x7fdd48156380 0x7fdd480740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7fdd4e005640 1 -- 10.20.193.16:0/3021793843 >> v1:10.20.193.148:6789/0 conn(0x7fdd48155fa0 legacy=0x7fdd480cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7fdd4e005640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7fdd4e005640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7fdd4e005640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7fdd4e005640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.800+0000 7fdd4e005640 1 -- 10.20.193.16:0/3021793843 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd480ccdc0 con 0x7fdd480745e0 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 1 Processor -- start 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 1 -- start start 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 10 monclient: init 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f344f787640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f344f787640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f344f787640 10 monclient: monmap: 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.802131+0000 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.802131+0000 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 1 -- start start 2026-02-17T05:56:46.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 10 monclient: init 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 10 monclient: _add_conns ranks=[2,1,3,5,4,0] 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3bb815b8e0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3bb80cb250 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f3bb8073170 0x7f3bb815bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 10 monclient(hunting): picked mon.noname-a con 0x7f3bb8073170 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb80c85d0 con 0x7f3bb80cb250 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb80c88b0 con 0x7f3bb815b8e0 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb80c94c0 con 0x7f3bb8073170 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f3bbe10a640 10 monclient(hunting): authenticate will time out at 1059.370592s 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f344f787640 1 Processor -- start 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 10 monclient: _add_conns ranks=[2,3,0,4,5,1] 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6e80155e00 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f6e801561e0 0x7f6e800ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 10 monclient(hunting): picked mon.noname-a con 0x7f6e801561e0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6e80074dc0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:46.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e800c96d0 con 0x7f6e80074dc0 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e80074860 con 0x7f6e80155e00 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6e800c9940 con 0x7f6e801561e0 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f6e8695b640 10 monclient(hunting): authenticate will time out at 1059.370738s 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7fdb3e61a640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7fdb3e61a640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7fdb3e61a640 10 monclient: monmap: 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.802967+0000 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.802967+0000 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f344f787640 1 -- start start 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.801+0000 7f344f787640 10 monclient: init 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 10 monclient: _add_conns ranks=[3,5,2,0,1,4] 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f3448155e00 0x7f34480ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 10 monclient(hunting): picked mon.noname-a con 0x7f3448155e00 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f34481561e0 0x7f34480cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 10 monclient(hunting): picked mon.noname-c con 0x7f34481561e0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f1f1d199640 1 --2- 10.20.193.16:0/2389521453 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f00003dd0 0x7f1f000041d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009bc0 tx=0x7f1f0802f530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc5043eec1fa6290 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3448074dc0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34480c96d0 con 0x7f3448074dc0 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3448074860 con 0x7f3448155e00 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f34480c9940 con 0x7f34481561e0 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f344f787640 10 monclient(hunting): authenticate will time out at 1059.371330s 2026-02-17T05:56:46.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7f7ebdbf2640 1 --2- 10.20.193.16:0/2467910961 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7eb8074dc0 0x7f7eb80cc1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7eb806cbb0 tx=0x7f7eb402f270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6e9ab57b24eefd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7fdb3e61a640 1 Processor -- start 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7fdb3e61a640 1 -- start start 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.802+0000 7fdb3e61a640 10 monclient: init 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 10 monclient: _add_conns ranks=[0,5,2,4,3,1] 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f0e0492b640 1 --2- 10.20.193.16:0/1311833023 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e00155fa0 0x7f0e000cc1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e0008f340 tx=0x7f0df002ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a003dd9c3b9e258 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdb38155e00 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fdb381561e0 0x7fdb380ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 10 monclient(hunting): picked mon.noname-c con 0x7fdb381561e0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdb38074dc0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb380c96d0 con 0x7fdb38155e00 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb38074860 con 0x7fdb38074dc0 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fdb380c9940 con 0x7fdb381561e0 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdb3e61a640 10 monclient(hunting): authenticate will time out at 1059.372104s 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fdd4e005640 1 --2- 10.20.193.16:0/3021793843 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd480745e0 0x7fdd4815d360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdd4808e730 tx=0x7fdd4402f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2012c842e21c1a5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fd9f77ba640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fd9f77ba640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fd9f77ba640 10 monclient: monmap: 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.804143+0000 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.804143+0000 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f20a2575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20a40c7d00 0x7f20a4143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f3bb7fff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f3bb8073170 0x7f3bb815bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fd9f77ba640 1 Processor -- start 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f961eba0640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f961eba0640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f961eba0640 10 monclient: monmap: 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.804729+0000 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.804729+0000 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fd9f77ba640 1 -- start start 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7fd9f77ba640 10 monclient: init 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f20a2575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20a40c7d00 0x7f20a4143010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59560/0 (socket says 10.20.193.16:59560) 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f20a2575640 1 -- 10.20.193.16:0/578125960 learned_addr learned my addr 10.20.193.16:0/578125960 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f20a2575640 10 monclient(hunting): get_auth_request con 0x7f20a40c7d00 auth_method 0 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f3bb7fff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f3bb8073170 0x7f3bb815bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59574/0 (socket says 10.20.193.16:59574) 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f3bb7fff640 1 -- 10.20.193.16:0/492819730 learned_addr learned my addr 10.20.193.16:0/492819730 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f3bb7fff640 10 monclient(hunting): get_auth_request con 0x7f3bb8073170 auth_method 0 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f3bb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f3bb7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f3bb7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f20a2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f20a2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.803+0000 7f20a2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 1 Processor -- start 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f20a2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f20a2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f20a2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 1 -- start start 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 10 monclient: init 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 10 monclient: _add_conns ranks=[0,3,2,4,1,5] 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd9f0155e00 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fd9f01561e0 0x7fd9f00ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 10 monclient(hunting): picked mon.noname-a con 0x7fd9f01561e0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f3bb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f3bb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f3bb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd9f0074dc0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f00c96d0 con 0x7fd9f0155e00 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f0074860 con 0x7fd9f0074dc0 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f00c9940 con 0x7fd9f01561e0 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7fd9f77ba640 10 monclient(hunting): authenticate will time out at 1059.373499s 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 10 monclient: _add_conns ranks=[4,1,3,2,0,5] 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f9618155e00 0x7f96180ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 10 monclient(hunting): picked mon.noname-b con 0x7f9618155e00 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f96181561e0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f9618074dc0 0x7f96180cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 10 monclient(hunting): picked mon.noname-a con 0x7f9618074dc0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96180c96d0 con 0x7f96181561e0 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9618074860 con 0x7f9618074dc0 2026-02-17T05:56:46.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f96180c9940 con 0x7f9618155e00 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.804+0000 7f961eba0640 10 monclient(hunting): authenticate will time out at 1059.373836s 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.805+0000 7f3bb7fff640 10 monclient(hunting): handle_auth_done global_id 4380 payload 274 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.805+0000 7f3bb7fff640 1 -- 10.20.193.16:0/492819730 >> v1:10.20.193.148:6789/0 conn(0x7f3bb815b8e0 legacy=0x7f3bb8151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.805+0000 7f3bb7fff640 1 -- 10.20.193.16:0/492819730 >> v1:10.20.193.128:6789/0 conn(0x7f3bb80cb250 legacy=0x7f3bb806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.805+0000 7f3bb7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.805+0000 7f3bb7fff640 1 monclient: found mon.noname-a 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.805+0000 7f3bb7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.805+0000 7f3bb7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.805+0000 7f3bb7fff640 1 -- 10.20.193.16:0/492819730 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb80c8b90 con 0x7f3bb8073170 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.805+0000 7f20a1d74640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20a414e830 0x7f20a40c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0803b070 con 0x7f1f00003dd0 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f07fff640 10 monclient: dump: 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.859 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f08036400 con 0x7f1f00003dd0 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f07fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f07fff640 1 -- 10.20.193.16:0/2389521453 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08009790 con 0x7f1f00003dd0 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f07fff640 10 monclient: dump: 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 4 set_mon_vals no callback set 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f20a1d74640 10 monclient(hunting): get_auth_request con 0x7f20a414e830 auth_method 0 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f20a1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f20a1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f20a1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 10 monclient: shutdown 2026-02-17T05:56:46.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 1 -- 10.20.193.16:0/2389521453 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f00003dd0 msgr2=0x7f1f000041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 1 --2- 10.20.193.16:0/2389521453 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f00003dd0 0x7f1f000041d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009bc0 tx=0x7f1f0802f530 comp rx=0 tx=0).stop 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f6e7ffff640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f6e80155e00 0x7f6e80074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46426/0 (socket says 10.20.193.16:46426) 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f6e7ffff640 1 -- 10.20.193.16:0/2645955090 learned_addr learned my addr 10.20.193.16:0/2645955090 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f3bb7fff640 1 --2- 10.20.193.16:0/492819730 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3bb8073170 0x7f3bb815bcc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009e50 tx=0x7f3bac02f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2966ebb3a290b24 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 1 -- 10.20.193.16:0/2389521453 shutdown_connections 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 1 --2- 10.20.193.16:0/2389521453 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f00003dd0 0x7f1f000041d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 1 -- 10.20.193.16:0/2389521453 >> 10.20.193.16:0/2389521453 conn(0x7f1f180cf580 msgr2=0x7f1f180cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 1 -- 10.20.193.16:0/2389521453 shutdown_connections 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7f1f1f424640 1 -- 10.20.193.16:0/2389521453 wait complete. 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7ff9437fe640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7ff944074dc0 0x7ff944074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46424/0 (socket says 10.20.193.16:46424) 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.806+0000 7ff9437fe640 1 -- 10.20.193.16:0/203317304 learned_addr learned my addr 10.20.193.16:0/203317304 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:46.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f1f1f424640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f1f1f424640 10 monclient: monmap: 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f43cf2e8640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f43cf2e8640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f43cf2e8640 10 monclient: monmap: 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.808499+0000 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.808499+0000 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.862 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f20a2d76640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20a414ec10 0x7f20a40c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f1f1f424640 1 librados: starting msgr at 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f1f1f424640 1 librados: starting objecter 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 1 -- 10.20.193.16:0/2866060630 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c03c070 con 0x7f803c155fa0 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 10 monclient: dump: 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 1 -- 10.20.193.16:0/2866060630 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f802c02fbb0 con 0x7f803c155fa0 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 1 -- 10.20.193.16:0/2866060630 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c037440 con 0x7f803c155fa0 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f802bfff640 10 monclient: dump: 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f804332c640 5 monclient: authenticate success, global_id 4439 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f804332c640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f804332c640 4 set_mon_vals no callback set 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f1f1f424640 1 Processor -- start 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 1 -- 10.20.193.16:0/1311833023 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df003c070 con 0x7f0e00155fa0 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 10 monclient: dump: 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.864 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 1 -- 10.20.193.16:0/1311833023 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df002fbc0 con 0x7f0e00155fa0 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0e06bb6640 5 monclient: authenticate success, global_id 4442 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0e06bb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f1f1f424640 1 -- start start 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f1f1f424640 1 librados: setting wanted keys 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 1 -- 10.20.193.16:0/1311833023 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0037440 con 0x7f0e00155fa0 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0e06bb6640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0e06bb6640 4 set_mon_vals no callback set 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f0df77fe640 10 monclient: dump: 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.865 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f1f1f424640 1 librados: calling monclient init 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.807+0000 7f1f1f424640 10 monclient: init 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 1 Processor -- start 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 1 -- start start 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 10 monclient: init 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1902b75640 10 monclient: get_monmap_and_config 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1902b75640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1902b75640 10 monclient: monmap: 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:46.809133+0000 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:46.809133+0000 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:46.866 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f804332c640 10 monclient: shutdown 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f804332c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f804332c640 1 -- 10.20.193.16:0/2866060630 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c155fa0 msgr2=0x7f803c0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f804332c640 1 --2- 10.20.193.16:0/2866060630 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c155fa0 0x7f803c0cc1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f803c0cdb50 tx=0x7f802c02eef0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f804332c640 1 -- 10.20.193.16:0/2866060630 shutdown_connections 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f804332c640 1 --2- 10.20.193.16:0/2866060630 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c155fa0 0x7f803c0cc1a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f804332c640 1 -- 10.20.193.16:0/2866060630 >> 10.20.193.16:0/2866060630 conn(0x7f803c0ce6c0 msgr2=0x7f803c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f804332c640 1 -- 10.20.193.16:0/2866060630 shutdown_connections 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f0e06bb6640 10 monclient: shutdown 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f0e06bb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f0e06bb6640 1 -- 10.20.193.16:0/1311833023 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e00155fa0 msgr2=0x7f0e000cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f0e06bb6640 1 --2- 10.20.193.16:0/1311833023 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e00155fa0 0x7f0e000cc1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e0008f340 tx=0x7f0df002ef90 comp rx=0 tx=0).stop 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f804332c640 1 -- 10.20.193.16:0/2866060630 wait complete. 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7fa1e37fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fa1e41561e0 0x7fa1e40ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f00003dd0 0x7f1f180c3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f0e06bb6640 1 -- 10.20.193.16:0/1311833023 shutdown_connections 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f0e06bb6640 1 --2- 10.20.193.16:0/1311833023 >> v2:10.20.193.128:3300/0 conn(0x7f0e00156380 0x7f0e000740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 10 monclient(hunting): picked mon.noname-c con 0x7f1f00003dd0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f0e06bb6640 1 --2- 10.20.193.16:0/1311833023 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e00155fa0 0x7f0e000cc1a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f0e06bb6640 1 -- 10.20.193.16:0/1311833023 >> 10.20.193.16:0/1311833023 conn(0x7f0e000ce6c0 msgr2=0x7f0e000cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f180c4ef0 0x7f1f1813c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 10 monclient(hunting): picked mon.noname-a con 0x7f1f180c4ef0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f0e06bb6640 1 -- 10.20.193.16:0/1311833023 shutdown_connections 2026-02-17T05:56:46.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f180c5370 0x7f1f1813cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 10 monclient(hunting): picked mon.noname-b con 0x7f1f180c5370 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2d76640 10 monclient(hunting): get_auth_request con 0x7f20a414ec10 auth_method 0 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1806f480 con 0x7f1f180c4ef0 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1815a960 con 0x7f1f180c5370 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18159aa0 con 0x7f1f00003dd0 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1f1f424640 10 monclient(hunting): authenticate will time out at 1059.377597s 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f0e06bb6640 1 -- 10.20.193.16:0/1311833023 wait complete. 2026-02-17T05:56:46.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2575640 10 monclient(hunting): handle_auth_done global_id 4377 payload 824 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1902b75640 1 Processor -- start 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7ff942ffd640 1 --2- 10.20.193.16:0/203317304 >> v2:10.20.193.128:3300/0 conn(0x7ff944155e00 0x7ff9440ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2575640 1 -- 10.20.193.16:0/578125960 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20a414e830 msgr2=0x7f20a40c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2575640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f20a414e830 0x7f20a40c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2575640 1 -- 10.20.193.16:0/578125960 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20a414ec10 msgr2=0x7f20a40c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2575640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f20a414ec10 0x7f20a40c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2575640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2575640 1 monclient: found mon.noname-a 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f20a2575640 1 -- 10.20.193.16:0/578125960 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a4146f70 con 0x7f20a40c7d00 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f344dcfd640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f3448074dc0 0x7f3448074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46442/0 (socket says 10.20.193.16:46442) 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f344dcfd640 1 -- 10.20.193.16:0/2008726529 learned_addr learned my addr 10.20.193.16:0/2008726529 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1902b75640 1 -- start start 2026-02-17T05:56:46.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f1902b75640 10 monclient: init 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 10 monclient: _add_conns ranks=[0,2,3,5,4,1] 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f43c8155e00 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f43c81561e0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f43c815fa10 0x7f43c80ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 10 monclient(hunting): picked mon.noname-a con 0x7f43c815fa10 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c80c96d0 con 0x7f43c8155e00 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c8074860 con 0x7f43c81561e0 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f43c80c9940 con 0x7f43c815fa10 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f43cf2e8640 10 monclient(hunting): authenticate will time out at 1059.377974s 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7f804332c640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient: monmap: 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.871 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f6e7f7fe640 1 --2- 10.20.193.16:0/2645955090 >> v2:10.20.193.16:3300/0 conn(0x7f6e801561e0 0x7f6e800ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.808+0000 7fdb3cb90640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fdb38074dc0 0x7fdb3815f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46458/0 (socket says 10.20.193.16:46458) 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7fdb3cb90640 1 -- 10.20.193.16:0/3490808065 learned_addr learned my addr 10.20.193.16:0/3490808065 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7fa1e37fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fa1e41561e0 0x7fa1e40ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56828/0 (socket says 10.20.193.16:56828) 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7fa1e37fe640 1 -- 10.20.193.16:0/2413126280 learned_addr learned my addr 10.20.193.16:0/2413126280 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7fa1e37fe640 10 monclient(hunting): get_auth_request con 0x7fa1e41561e0 auth_method 0 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7fa1e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7fa1e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7fa1e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 librados: starting msgr at 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 librados: starting objecter 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 Processor -- start 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 1 -- 10.20.193.16:0/2467910961 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb402fcd0 con 0x7f7eb8074dc0 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 10 monclient: dump: 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.872 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 1 -- 10.20.193.16:0/2467910961 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb40373f0 con 0x7f7eb8074dc0 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 5 monclient: authenticate success, global_id 4417 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 -- start start 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 librados: setting wanted keys 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 librados: calling monclient init 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient: init 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 1 -- 10.20.193.16:0/2467910961 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb40377c0 con 0x7f7eb8074dc0 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 4 set_mon_vals no callback set 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ea7fff640 10 monclient: dump: 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.873 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f0e06bb6640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f0e06bb6640 10 monclient: monmap: 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7ff942ffd640 10 monclient(hunting): get_auth_request con 0x7ff944155e00 auth_method 0 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7ff942ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7ff942ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7ff942ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 10 monclient: _add_conns ranks=[2,5,3,0,1,4] 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f18fc155e00 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f18fc1561e0 0x7f18fc0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 10 monclient(hunting): picked mon.noname-c con 0x7f18fc1561e0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:46.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f18fc074dc0 0x7f18fc0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 10 monclient(hunting): picked mon.noname-a con 0x7f18fc074dc0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18fc0c96d0 con 0x7f18fc155e00 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 10 monclient: shutdown 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2467910961 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7eb8074dc0 msgr2=0x7f7eb80cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 1 --2- 10.20.193.16:0/2467910961 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7eb8074dc0 0x7f7eb80cc1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7eb806cbb0 tx=0x7f7eb402f270 comp rx=0 tx=0).stop 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f18fc074860 con 0x7f18fc074dc0 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f18fc0c9940 con 0x7f18fc1561e0 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f1902b75640 10 monclient(hunting): authenticate will time out at 1059.378589s 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c155fa0 0x7f803c1f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f0e06bb6640 1 librados: starting msgr at 2026-02-17T05:56:46.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f0e06bb6640 1 librados: starting objecter 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient(hunting): picked mon.noname-b con 0x7f803c155fa0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f803c156380 0x7f803c149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient(hunting): picked mon.noname-a con 0x7f803c156380 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c15d360 0x7f803c1f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2467910961 shutdown_connections 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 1 --2- 10.20.193.16:0/2467910961 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7eb8074dc0 0x7f7eb80cc1a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 1 --2- 10.20.193.16:0/2467910961 >> v2:10.20.193.16:3300/0 conn(0x7f7eb8155e00 0x7f7eb80ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2467910961 >> 10.20.193.16:0/2467910961 conn(0x7f7eb80ce6c0 msgr2=0x7f7eb80cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient(hunting): picked mon.noname-c con 0x7f803c15d360 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0747b0 con 0x7f803c156380 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0cd660 con 0x7f803c155fa0 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f803c074d50 con 0x7f803c15d360 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f804332c640 10 monclient(hunting): authenticate will time out at 1059.378914s 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f0e06bb6640 1 Processor -- start 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2467910961 shutdown_connections 2026-02-17T05:56:46.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.809+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2467910961 wait complete. 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7fa1e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7fa1e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7fa1e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 1 -- start start 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 1 librados: setting wanted keys 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7fa1e8e68640 1 --2- 10.20.193.16:0/2413126280 >> v2:10.20.193.128:3300/0 conn(0x7fa1e4074dc0 0x7fa1e40cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 1 librados: calling monclient init 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 10 monclient: init 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f7ebf67c640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f7ebf67c640 10 monclient: monmap: 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.877 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7ff942ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7ff942ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7ff942ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f7ebf67c640 1 librados: starting msgr at 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f7ebf67c640 1 librados: starting objecter 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f7ebf67c640 1 Processor -- start 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f7ebf67c640 1 -- start start 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f7ebf67c640 1 librados: setting wanted keys 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f7ebf67c640 1 librados: calling monclient init 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f7ebf67c640 10 monclient: init 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f6e7f7fe640 10 monclient(hunting): get_auth_request con 0x7f6e801561e0 auth_method 0 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f6e7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f6e7f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f6e7f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e000745e0 0x7f0e001f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7fa1e8e68640 10 monclient(hunting): get_auth_request con 0x7fa1e4074dc0 auth_method 0 2026-02-17T05:56:46.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7fa1e8e68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7fa1e8e68640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 10 monclient(hunting): picked mon.noname-b con 0x7f0e000745e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7fa1e8e68640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e00155fa0 0x7f0e001f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 10 monclient(hunting): picked mon.noname-c con 0x7f0e00155fa0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e00156380 0x7f0e001f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 10 monclient(hunting): picked mon.noname-a con 0x7f0e00156380 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e00074d50 con 0x7f0e00156380 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.810+0000 7f0e06bb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f0e06bb6640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f0e000cd660 con 0x7f0e000745e0 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f0e06bb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f0e06bb6640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e000cc6e0 con 0x7f0e00155fa0 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f0e06bb6640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f0e06bb6640 10 monclient(hunting): authenticate will time out at 1059.380057s 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7eb8074dc0 0x7f7eb8143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 10 monclient(hunting): picked mon.noname-a con 0x7f7eb8074dc0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7eb8155e00 0x7f7eb815b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 10 monclient(hunting): picked mon.noname-c con 0x7f7eb8155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7eb81561e0 0x7f7eb815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 10 monclient(hunting): picked mon.noname-b con 0x7f7eb81561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb80c9360 con 0x7f7eb8074dc0 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb80ca1e0 con 0x7f7eb81561e0 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb80c8c20 con 0x7f7eb8155e00 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f7ebf67c640 10 monclient(hunting): authenticate will time out at 1059.380193s 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f6e7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f6e7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f6e7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7fa1e37fe640 10 monclient(hunting): handle_auth_done global_id 4451 payload 274 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7fa1e37fe640 1 -- 10.20.193.16:0/2413126280 >> v2:10.20.193.128:3300/0 conn(0x7fa1e4074dc0 msgr2=0x7fa1e40cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7fa1e37fe640 1 --2- 10.20.193.16:0/2413126280 >> v2:10.20.193.128:3300/0 conn(0x7fa1e4074dc0 0x7fa1e40cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7fa1e37fe640 1 -- 10.20.193.16:0/2413126280 >> v1:10.20.193.16:6789/0 conn(0x7fa1e4155e00 legacy=0x7fa1e4074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7fa1e37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7fa1e37fe640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7fa1e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7fa1e37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7fa1e37fe640 1 -- 10.20.193.16:0/2413126280 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e40c9c20 con 0x7fa1e41561e0 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7f344ccfb640 1 --2- 10.20.193.16:0/2008726529 >> v2:10.20.193.148:3300/0 conn(0x7f34481561e0 0x7f34480cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7ff942ffd640 10 monclient(hunting): handle_auth_done global_id 4429 payload 274 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7ff942ffd640 1 -- 10.20.193.16:0/203317304 >> v2:10.20.193.16:3300/0 conn(0x7ff9441561e0 msgr2=0x7ff9440cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7ff942ffd640 1 --2- 10.20.193.16:0/203317304 >> v2:10.20.193.16:3300/0 conn(0x7ff9441561e0 0x7ff9440cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7ff942ffd640 1 -- 10.20.193.16:0/203317304 >> v1:10.20.193.148:6789/0 conn(0x7ff944074dc0 legacy=0x7ff944074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7ff942ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7ff942ffd640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7ff942ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7ff942ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.811+0000 7ff942ffd640 1 -- 10.20.193.16:0/203317304 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9440c9c20 con 0x7ff944155e00 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f6e7f7fe640 10 monclient(hunting): handle_auth_done global_id 4383 payload 274 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f6e7f7fe640 1 -- 10.20.193.16:0/2645955090 >> v1:10.20.193.148:6789/0 conn(0x7f6e80155e00 legacy=0x7f6e80074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f6e7f7fe640 1 -- 10.20.193.16:0/2645955090 >> v1:10.20.193.16:6789/0 conn(0x7f6e80074dc0 legacy=0x7f6e8015f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f6e7f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f6e7f7fe640 1 monclient: found mon.noname-a 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f6e7f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f6e7f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f6e7f7fe640 1 -- 10.20.193.16:0/2645955090 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e800c9c20 con 0x7f6e801561e0 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f344ccfb640 10 monclient(hunting): get_auth_request con 0x7f34481561e0 auth_method 0 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f344ccfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f344ccfb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f344ccfb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f20a2575640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20a40c7d00 0x7f20a4143010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f209800c760 tx=0x7f209800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f208affd640 10 client.?.objecter ms_handle_connect 0x7f20a40c7d00 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.812+0000 7f208affd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7ff942ffd640 1 --2- 10.20.193.16:0/203317304 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff944155e00 0x7ff9440ca700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff94406ce20 tx=0x7ff93802ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb139d9f9d5843ee server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f961c915640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f9618155e00 0x7f96180ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f961c915640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f9618155e00 0x7f96180ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50090/0 (socket says 10.20.193.16:50090) 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f961c915640 1 -- 10.20.193.16:0/1562582750 learned_addr learned my addr 10.20.193.16:0/1562582750 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f961c915640 10 monclient(hunting): get_auth_request con 0x7f9618155e00 auth_method 0 2026-02-17T05:56:46.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f961c915640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f961c915640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f961c915640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f344ccfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f344ccfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f344ccfb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7f6e7f7fe640 1 --2- 10.20.193.16:0/2645955090 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e801561e0 0x7f6e800ca700 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e700098a0 tx=0x7f6e7002ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b57ae8b65ab8d2d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.813+0000 7fa1e37fe640 1 --2- 10.20.193.16:0/2413126280 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa1e41561e0 0x7fa1e40ca700 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40693f0 tx=0x7fa1d402eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=934f888dc7e085b2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f961c915640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f961c915640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f961c915640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f344d4fc640 1 --2- 10.20.193.16:0/2008726529 >> v2:10.20.193.16:3300/0 conn(0x7f3448155e00 0x7f34480ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f344d4fc640 10 monclient(hunting): get_auth_request con 0x7f3448155e00 auth_method 0 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f344d4fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f344d4fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f344d4fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f344d4fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f344d4fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.814+0000 7f344d4fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f344ccfb640 10 monclient(hunting): handle_auth_done global_id 4457 payload 274 2026-02-17T05:56:46.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f344ccfb640 1 -- 10.20.193.16:0/2008726529 >> v2:10.20.193.16:3300/0 conn(0x7f3448155e00 msgr2=0x7f34480ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f344ccfb640 1 --2- 10.20.193.16:0/2008726529 >> v2:10.20.193.16:3300/0 conn(0x7f3448155e00 0x7f34480ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f344ccfb640 1 -- 10.20.193.16:0/2008726529 >> v1:10.20.193.148:6789/0 conn(0x7f3448074dc0 legacy=0x7f3448074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f344ccfb640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f344ccfb640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f344ccfb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f344ccfb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f344ccfb640 1 -- 10.20.193.16:0/2008726529 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34480c9c20 con 0x7f34481561e0 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 1 -- 10.20.193.16:0/3021793843 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4402fd70 con 0x7fdd480745e0 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 10 monclient: dump: 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 1 -- 10.20.193.16:0/3021793843 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd440373f0 con 0x7fdd480745e0 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4fa8f640 5 monclient: authenticate success, global_id 4420 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4fa8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4fa8f640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4fa8f640 4 set_mon_vals no callback set 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 1 -- 10.20.193.16:0/3021793843 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd440377c0 con 0x7fdd480745e0 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4c802640 10 monclient: dump: 2026-02-17T05:56:46.884 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4fa8f640 10 monclient: shutdown 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4fa8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/3021793843 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd480745e0 msgr2=0x7fdd4815d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4fa8f640 1 --2- 10.20.193.16:0/3021793843 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd480745e0 0x7fdd4815d360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdd4808e730 tx=0x7fdd4402f310 comp rx=0 tx=0).stop 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f961c915640 10 monclient(hunting): handle_auth_done global_id 4435 payload 274 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/3021793843 shutdown_connections 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f961c915640 1 -- 10.20.193.16:0/1562582750 >> v2:10.20.193.16:3300/0 conn(0x7f9618074dc0 msgr2=0x7f96180cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f961c915640 1 --2- 10.20.193.16:0/1562582750 >> v2:10.20.193.16:3300/0 conn(0x7f9618074dc0 0x7f96180cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f961c915640 1 -- 10.20.193.16:0/1562582750 >> v1:10.20.193.128:6789/0 conn(0x7f96181561e0 legacy=0x7f9618074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7f961c915640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7f961c915640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7f961c915640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7f961c915640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7f961c915640 1 -- 10.20.193.16:0/1562582750 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96180c9c20 con 0x7f9618155e00 2026-02-17T05:56:46.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.815+0000 7fdd4fa8f640 1 --2- 10.20.193.16:0/3021793843 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd480745e0 0x7fdd4815d360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7fdd4fa8f640 1 --2- 10.20.193.16:0/3021793843 >> v2:10.20.193.16:3300/0 conn(0x7fdd48156380 0x7fdd480740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/3021793843 >> 10.20.193.16:0/3021793843 conn(0x7fdd480ce6c0 msgr2=0x7fdd480cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/3021793843 shutdown_connections 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/3021793843 wait complete. 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7fdd4fa8f640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7fdd4fa8f640 10 monclient: monmap: 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7fd9f4d2e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fd9f01561e0 0x7fd9f00ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7fdd4fa8f640 1 librados: starting msgr at 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.816+0000 7fdd4fa8f640 1 librados: starting objecter 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 1 Processor -- start 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 1 -- start start 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 1 librados: setting wanted keys 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 1 librados: calling monclient init 2026-02-17T05:56:46.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 10 monclient: init 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 1 -- 10.20.193.16:0/203317304 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93803c070 con 0x7ff944155e00 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 10 monclient: dump: 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fd9f5d30640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fd9f0074dc0 0x7fd9f015f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46460/0 (socket says 10.20.193.16:46460) 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fd9f5d30640 1 -- 10.20.193.16:0/860860856 learned_addr learned my addr 10.20.193.16:0/860860856 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 1 -- 10.20.193.16:0/203317304 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93802fa50 con 0x7ff944155e00 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 1 -- 10.20.193.16:0/203317304 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93802fe20 con 0x7ff944155e00 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 5 monclient: authenticate success, global_id 4429 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 4 set_mon_vals no callback set 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff941ffb640 10 monclient: dump: 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.887 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 1 -- 10.20.193.16:0/492819730 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac02fd00 con 0x7f3bb8073170 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 10 monclient: dump: 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 1 -- 10.20.193.16:0/492819730 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bac0373f0 con 0x7f3bb8073170 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 1 -- 10.20.193.16:0/492819730 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0377c0 con 0x7f3bb8073170 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bb67fc640 10 monclient: dump: 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.888 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 10 monclient: shutdown 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 1 -- 10.20.193.16:0/203317304 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff944155e00 msgr2=0x7ff9440ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 1 --2- 10.20.193.16:0/203317304 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff944155e00 0x7ff9440ca700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff94406ce20 tx=0x7ff93802ede0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 1 -- 10.20.193.16:0/203317304 shutdown_connections 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 1 --2- 10.20.193.16:0/203317304 >> v2:10.20.193.16:3300/0 conn(0x7ff9441561e0 0x7ff9440cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 1 --2- 10.20.193.16:0/203317304 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff944155e00 0x7ff9440ca700 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 1 -- 10.20.193.16:0/203317304 >> 10.20.193.16:0/203317304 conn(0x7ff9440ce6c0 msgr2=0x7ff9440cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 1 -- 10.20.193.16:0/203317304 shutdown_connections 2026-02-17T05:56:46.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7ff9493d3640 1 -- 10.20.193.16:0/203317304 wait complete. 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd480745e0 0x7fdd481f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 10 monclient(hunting): picked mon.noname-a con 0x7fdd480745e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdd48155fa0 0x7fdd480c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 10 monclient(hunting): picked mon.noname-c con 0x7fdd48155fa0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd48156380 0x7fdd480c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bbe10a640 5 monclient: authenticate success, global_id 4380 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bbe10a640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7f3bbe10a640 4 set_mon_vals no callback set 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.817+0000 7fdd4fa8f640 10 monclient(hunting): picked mon.noname-b con 0x7fdd48156380 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 10 monclient: shutdown 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 1 -- 10.20.193.16:0/492819730 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3bb8073170 msgr2=0x7f3bb815bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 1 --2- 10.20.193.16:0/492819730 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3bb8073170 0x7f3bb815bcc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009e50 tx=0x7f3bac02f2a0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fdd4fa8f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fdd4fa8f640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd480cd660 con 0x7fdd480745e0 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fdd4fa8f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fdd4fa8f640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fdd480cd380 con 0x7fdd48156380 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fdd4fa8f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fdd4fa8f640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdd48074d50 con 0x7fdd48155fa0 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fdd4fa8f640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fdd4fa8f640 10 monclient(hunting): authenticate will time out at 1059.387088s 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fd9f4d2e640 10 monclient(hunting): get_auth_request con 0x7fd9f01561e0 auth_method 0 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fd9f4d2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fd9f4d2e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fd9f4d2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 1 -- 10.20.193.16:0/492819730 shutdown_connections 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 1 --2- 10.20.193.16:0/492819730 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3bb8073170 0x7f3bb815bcc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 1 -- 10.20.193.16:0/492819730 >> 10.20.193.16:0/492819730 conn(0x7f3bb80ca5a0 msgr2=0x7f3bb80ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f1f1d99a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f180c5370 0x7f1f1813cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 1 -- 10.20.193.16:0/492819730 shutdown_connections 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 1 -- 10.20.193.16:0/492819730 wait complete. 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7ff9493d3640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7ff9493d3640 10 monclient: monmap: 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7ff9493d3640 1 librados: starting msgr at 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7ff9493d3640 1 librados: starting objecter 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7ff9493d3640 1 Processor -- start 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fd9f4d2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fd9f4d2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7fd9f4d2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f1f1d99a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f180c5370 0x7f1f1813cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50102/0 (socket says 10.20.193.16:50102) 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7ff9493d3640 1 -- start start 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7ff9493d3640 1 librados: setting wanted keys 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7ff9493d3640 1 librados: calling monclient init 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7ff9493d3640 10 monclient: init 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f1f1d99a640 1 -- 10.20.193.16:0/1122171649 learned_addr learned my addr 10.20.193.16:0/1122171649 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f1f1d99a640 10 monclient(hunting): get_auth_request con 0x7f1f180c5370 auth_method 0 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f1f1d99a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f1f1d99a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f1f1d99a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.818+0000 7f3bbe10a640 10 monclient: monmap: 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.892 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f0e0492b640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e000745e0 0x7f0e001f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff944074dc0 0x7ff9441415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 10 monclient(hunting): picked mon.noname-a con 0x7ff944074dc0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fdb377fe640 1 --2- 10.20.193.16:0/3490808065 >> v2:10.20.193.148:3300/0 conn(0x7fdb381561e0 0x7fdb380ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff944155e00 0x7ff9441f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 10 monclient(hunting): picked mon.noname-c con 0x7ff944155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff9441561e0 0x7ff9441fa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 10 monclient(hunting): picked mon.noname-b con 0x7ff9441561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9440c9360 con 0x7ff944074dc0 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7ff9440ca1e0 con 0x7ff9441561e0 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff9440c8c20 con 0x7ff944155e00 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7ff9493d3640 10 monclient(hunting): authenticate will time out at 1059.388075s 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 librados: starting msgr at 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 librados: starting objecter 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 Processor -- start 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f1f1d99a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f1f1d99a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 -- start start 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 librados: setting wanted keys 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 librados: calling monclient init 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient: init 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f1f1d99a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f0e0492b640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e000745e0 0x7f0e001f3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50128/0 (socket says 10.20.193.16:50128) 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f0e0492b640 1 -- 10.20.193.16:0/401859147 learned_addr learned my addr 10.20.193.16:0/401859147 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f0e0492b640 10 monclient(hunting): get_auth_request con 0x7f0e000745e0 auth_method 0 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f0e0492b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fdb377fe640 10 monclient(hunting): get_auth_request con 0x7fdb381561e0 auth_method 0 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fdb377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fdb377fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fdb377fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f0e0492b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f0e0492b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3bb8073170 0x7f3bb81f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient(hunting): picked mon.noname-a con 0x7f3bb8073170 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3bb80cb250 0x7f3bb8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient(hunting): picked mon.noname-b con 0x7f3bb80cb250 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3bb815b8e0 0x7f3bb814c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient(hunting): picked mon.noname-c con 0x7f3bb815b8e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8152380 con 0x7f3bb8073170 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb80c9150 con 0x7f3bb80cb250 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8152500 con 0x7f3bb815b8e0 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f3bbe10a640 10 monclient(hunting): authenticate will time out at 1059.388726s 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7f7ebdbf2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7eb81561e0 0x7f7eb815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fd9f4d2e640 10 monclient(hunting): handle_auth_done global_id 4389 payload 274 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fd9f4d2e640 1 -- 10.20.193.16:0/860860856 >> v1:10.20.193.148:6789/0 conn(0x7fd9f0074dc0 legacy=0x7fd9f015f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fd9f4d2e640 1 -- 10.20.193.16:0/860860856 >> v1:10.20.193.16:6789/0 conn(0x7fd9f0155e00 legacy=0x7fd9f0074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fd9f4d2e640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fd9f4d2e640 1 monclient: found mon.noname-a 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fd9f4d2e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fd9f4d2e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.819+0000 7fd9f4d2e640 1 -- 10.20.193.16:0/860860856 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f00c9c20 con 0x7fd9f01561e0 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f1f1d99a640 10 monclient(hunting): handle_auth_done global_id 4438 payload 824 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f1f1d99a640 1 -- 10.20.193.16:0/1122171649 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f00003dd0 msgr2=0x7f1f180c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f1f1d99a640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f00003dd0 0x7f1f180c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f1f1d99a640 1 -- 10.20.193.16:0/1122171649 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f180c4ef0 msgr2=0x7f1f1813c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f1f1d99a640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f180c4ef0 0x7f1f1813c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f1f1d99a640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f1f1d99a640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f1f1d99a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f1f1d99a640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f1f1d99a640 1 -- 10.20.193.16:0/1122171649 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f180c3b70 con 0x7f1f180c5370 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f344ccfb640 1 --2- 10.20.193.16:0/2008726529 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34481561e0 0x7f34480cc1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34480693f0 tx=0x7f343c02ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b12ee1f63530f60b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7fdb377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7fdb377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7fdb377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f7ebdbf2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7eb81561e0 0x7f7eb815b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50136/0 (socket says 10.20.193.16:50136) 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f7ebdbf2640 1 -- 10.20.193.16:0/2452020534 learned_addr learned my addr 10.20.193.16:0/2452020534 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f7ebdbf2640 10 monclient(hunting): get_auth_request con 0x7f7eb81561e0 auth_method 0 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f7ebdbf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f7ebdbf2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.821+0000 7f7ebdbf2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.822+0000 7fdb36ffd640 1 -- 10.20.193.16:0/3490808065 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117917304 0 0) 0x7fdb2c002ca0 con 0x7fdb38074dc0 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.822+0000 7fdb36ffd640 10 monclient(hunting): _negotiate 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.822+0000 7fdb36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.822+0000 7fdb36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.822+0000 7fdb36ffd640 10 monclient(hunting): my global_id is 4454 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.822+0000 7fdb36ffd640 1 -- 10.20.193.16:0/3490808065 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb1c003620 con 0x7fdb38074dc0 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.823+0000 7f7ebdbf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.823+0000 7f7ebdbf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.823+0000 7f7ebdbf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.823+0000 7f1f1d99a640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f180c5370 0x7f1f1813cde0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f180d0680 tx=0x7f1f1400a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.823+0000 7f1f05ffb640 10 client.?.objecter ms_handle_connect 0x7f1f180c5370 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.823+0000 7f1f05ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.823+0000 7f961c915640 1 --2- 10.20.193.16:0/1562582750 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9618155e00 0x7f96180ca700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f961806ce20 tx=0x7f960802f270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c8b203551b50bac server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 1 -- 10.20.193.16:0/2645955090 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7003c070 con 0x7f6e801561e0 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 10 monclient: dump: 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.897 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 1 -- 10.20.193.16:0/2645955090 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e7002fad0 con 0x7f6e801561e0 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 1 -- 10.20.193.16:0/2645955090 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70037440 con 0x7f6e801561e0 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 5 monclient: authenticate success, global_id 4383 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 4 set_mon_vals no callback set 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e7effd640 10 monclient: dump: 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb36ffd640 1 -- 10.20.193.16:0/3490808065 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3812341100 0 0) 0x7fdb380c96d0 con 0x7fdb38155e00 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb36ffd640 10 monclient(hunting): _negotiate 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb36ffd640 10 monclient(hunting): my global_id is 4392 2026-02-17T05:56:46.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb36ffd640 1 -- 10.20.193.16:0/3490808065 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb2c002ca0 con 0x7fdb38155e00 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 10 monclient: shutdown 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 1 -- 10.20.193.16:0/2645955090 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e801561e0 msgr2=0x7f6e800ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 1 --2- 10.20.193.16:0/2645955090 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e801561e0 0x7f6e800ca700 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e700098a0 tx=0x7f6e7002ee10 comp rx=0 tx=0).stop 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 1 -- 10.20.193.16:0/2645955090 shutdown_connections 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 1 --2- 10.20.193.16:0/2645955090 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e801561e0 0x7f6e800ca700 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 1 -- 10.20.193.16:0/2645955090 >> 10.20.193.16:0/2645955090 conn(0x7f6e800ce6c0 msgr2=0x7f6e800cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 1 -- 10.20.193.16:0/2645955090 shutdown_connections 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb377fe640 10 monclient(hunting): handle_auth_done global_id 4466 payload 274 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb377fe640 1 -- 10.20.193.16:0/3490808065 >> v1:10.20.193.148:6789/0 conn(0x7fdb38074dc0 legacy=0x7fdb3815f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb377fe640 1 -- 10.20.193.16:0/3490808065 >> v1:10.20.193.16:6789/0 conn(0x7fdb38155e00 legacy=0x7fdb38074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb377fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb377fe640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb377fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fdb377fe640 1 -- 10.20.193.16:0/3490808065 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb380c9c20 con 0x7fdb381561e0 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7f6e8695b640 1 -- 10.20.193.16:0/2645955090 wait complete. 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.824+0000 7fd9f4d2e640 1 --2- 10.20.193.16:0/860860856 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd9f01561e0 0x7fd9f00ca700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40098a0 tx=0x7fd9e402ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3135947524757e98 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f6e8695b640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f7ebdbf2640 10 monclient(hunting): handle_auth_done global_id 4441 payload 824 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f7ebdbf2640 1 -- 10.20.193.16:0/2452020534 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7eb8155e00 msgr2=0x7f7eb815b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f7ebdbf2640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7eb8155e00 0x7f7eb815b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f7ebdbf2640 1 -- 10.20.193.16:0/2452020534 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7eb8074dc0 msgr2=0x7f7eb8143be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f7ebdbf2640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7eb8074dc0 0x7f7eb8143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f7ebdbf2640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f7ebdbf2640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f7ebdbf2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f7ebdbf2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f7ebdbf2640 1 -- 10.20.193.16:0/2452020534 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb81513b0 con 0x7f7eb81561e0 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f6e8695b640 10 monclient: monmap: 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.900 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f6e8695b640 1 librados: starting msgr at 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f6e8695b640 1 librados: starting objecter 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f6e8695b640 1 Processor -- start 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f6e8695b640 1 -- start start 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f6e8695b640 1 librados: setting wanted keys 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f6e8695b640 1 librados: calling monclient init 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.825+0000 7f6e8695b640 10 monclient: init 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e80074dc0 0x7f6e801f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 10 monclient(hunting): picked mon.noname-b con 0x7f6e80074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e80155e00 0x7f6e8013ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 10 monclient(hunting): picked mon.noname-a con 0x7f6e80155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e801561e0 0x7f6e8013f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 10 monclient(hunting): picked mon.noname-c con 0x7f6e801561e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e800c9360 con 0x7f6e80155e00 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f6e800ca1e0 con 0x7f6e80074dc0 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8015fa10 con 0x7f6e801561e0 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.826+0000 7f6e8695b640 10 monclient(hunting): authenticate will time out at 1059.395399s 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.827+0000 7f80418a2640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c15d360 0x7f803c1f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.827+0000 7f80418a2640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c15d360 0x7f803c1f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56868/0 (socket says 10.20.193.16:56868) 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.827+0000 7f80418a2640 1 -- 10.20.193.16:0/1238250860 learned_addr learned my addr 10.20.193.16:0/1238250860 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.827+0000 7f80418a2640 10 monclient(hunting): get_auth_request con 0x7f803c15d360 auth_method 0 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.827+0000 7f80418a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.827+0000 7f80418a2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.827+0000 7f80418a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.827+0000 7fdb377fe640 1 --2- 10.20.193.16:0/3490808065 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdb381561e0 0x7fdb380ca700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdb280098a0 tx=0x7fdb2802ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b461ba58a69ded4c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.828+0000 7f80418a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.828+0000 7f80418a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.828+0000 7f80418a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.828+0000 7fdd4e005640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd48156380 0x7fdd480c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.828+0000 7f0e0492b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.828+0000 7f0e0492b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.828+0000 7f0e0492b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7f80408a0640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f803c156380 0x7f803c149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7fdd4e005640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd48156380 0x7fdd480c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50138/0 (socket says 10.20.193.16:50138) 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7fdd4e005640 1 -- 10.20.193.16:0/1659368440 learned_addr learned my addr 10.20.193.16:0/1659368440 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7fdd4e005640 10 monclient(hunting): get_auth_request con 0x7fdd48156380 auth_method 0 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7fdd4e005640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7fdd4e005640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7fdd4e005640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7f80408a0640 10 monclient(hunting): get_auth_request con 0x7f803c156380 auth_method 0 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7f80408a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7f80408a0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.829+0000 7f80408a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80408a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80408a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80408a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 1 -- 10.20.193.16:0/578125960 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098017070 con 0x7f20a40c7d00 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 10 monclient: dump: 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 1 -- 10.20.193.16:0/578125960 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2098005b90 con 0x7f20a40c7d00 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 1 -- 10.20.193.16:0/578125960 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098007070 con 0x7f20a40c7d00 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f208affd640 10 monclient: dump: 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f20a0d72640 4 set_mon_vals no callback set 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80418a2640 10 monclient(hunting): handle_auth_done global_id 4469 payload 824 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80418a2640 1 -- 10.20.193.16:0/1238250860 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c155fa0 msgr2=0x7f803c1f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80418a2640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c155fa0 0x7f803c1f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80418a2640 1 -- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f803c156380 msgr2=0x7f803c149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80418a2640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f803c156380 0x7f803c149230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80418a2640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80418a2640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80418a2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80418a2640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f80418a2640 1 -- 10.20.193.16:0/1238250860 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c1f3ba0 con 0x7f803c15d360 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f20a8d85640 5 monclient: authenticate success, global_id 4377 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f20a8d85640 10 monclient: _renew_subs 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f20a8d85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a4147250 con 0x7f20a40c7d00 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f20a8d85640 10 client.4377.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f20a8d85640 10 monclient: _renew_subs 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f20a8d85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a4147810 con 0x7f20a40c7d00 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f20a8d85640 1 librados: init done 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7fdd4d804640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd480745e0 0x7fdd481f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f0e0492b640 10 monclient(hunting): handle_auth_done global_id 4444 payload 824 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f0e0492b640 1 -- 10.20.193.16:0/401859147 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e00155fa0 msgr2=0x7f0e001f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f0e0492b640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e00155fa0 0x7f0e001f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f0e0492b640 1 -- 10.20.193.16:0/401859147 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e00156380 msgr2=0x7f0e001f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f0e0492b640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e00156380 0x7f0e001f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f0e0492b640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f0e0492b640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f0e0492b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f0e0492b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.830+0000 7f0e0492b640 1 -- 10.20.193.16:0/401859147 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e001f5e80 con 0x7f0e000745e0 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 1 -- 10.20.193.16:0/2413126280 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d403c070 con 0x7fa1e41561e0 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 10 monclient: dump: 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 1 -- 10.20.193.16:0/2413126280 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d402fb40 con 0x7fa1e41561e0 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 1 -- 10.20.193.16:0/2413126280 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4037440 con 0x7fa1e41561e0 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 5 monclient: authenticate success, global_id 4451 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 4 set_mon_vals no callback set 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1e2ffd640 10 monclient: dump: 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.905 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7f20a8d85640 10 client.4377.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7f20a8d85640 15 client.4377.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7f20a8d85640 20 client.4377.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7f20a8d85640 10 client.4377.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7f20a8d85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20a40c9570 con 0x7f20a40c7d00 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7f18f3fff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f18fc1561e0 0x7f18fc0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 10 monclient: shutdown 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/2413126280 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa1e41561e0 msgr2=0x7fa1e40ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 1 --2- 10.20.193.16:0/2413126280 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa1e41561e0 0x7fa1e40ca700 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40693f0 tx=0x7fa1d402eed0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/2413126280 shutdown_connections 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 1 --2- 10.20.193.16:0/2413126280 >> v2:10.20.193.128:3300/0 conn(0x7fa1e4074dc0 0x7fa1e40cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 1 --2- 10.20.193.16:0/2413126280 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa1e41561e0 0x7fa1e40ca700 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/2413126280 >> 10.20.193.16:0/2413126280 conn(0x7fa1e40ce6c0 msgr2=0x7fa1e40cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/2413126280 shutdown_connections 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.831+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/2413126280 wait complete. 2026-02-17T05:56:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fdd4e005640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7f18f3fff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f18fc1561e0 0x7f18fc0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56864/0 (socket says 10.20.193.16:56864) 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7f18f3fff640 1 -- 10.20.193.16:0/2278536074 learned_addr learned my addr 10.20.193.16:0/2278536074 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7f18f3fff640 10 monclient(hunting): get_auth_request con 0x7f18fc1561e0 auth_method 0 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7f18f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7f18f3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7f18f3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fdd4e005640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fdd4e005640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fa1ea8f2640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fa1ea8f2640 10 monclient: monmap: 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fdd4d804640 10 monclient(hunting): get_auth_request con 0x7fdd480745e0 auth_method 0 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fdd4d804640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fdd4d804640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7f43cd85e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f43c815fa10 0x7f43c80ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fdd4d804640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fa1ea8f2640 1 librados: starting msgr at 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fa1ea8f2640 1 librados: starting objecter 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fa1ea8f2640 1 Processor -- start 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7f80418a2640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c15d360 0x7f803c1f6f90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f803800cc10 tx=0x7f803800a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7f8029ffb640 10 client.?.objecter ms_handle_connect 0x7f803c15d360 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7f8029ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fdd4d804640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fdd4d804640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fdd4d804640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fa1ea8f2640 1 -- start start 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fa1ea8f2640 1 librados: setting wanted keys 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fa1ea8f2640 1 librados: calling monclient init 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.832+0000 7fa1ea8f2640 10 monclient: init 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f0e0492b640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e000745e0 0x7f0e001f3d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009a70 tx=0x7f0df00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f0df57fa640 10 client.?.objecter ms_handle_connect 0x7f0e000745e0 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f0df57fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa1e4074dc0 0x7fa1e41f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 10 monclient(hunting): picked mon.noname-c con 0x7fa1e4074dc0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa1e4155e00 0x7fa1e4141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 10 monclient(hunting): picked mon.noname-b con 0x7fa1e4155e00 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa1e41561e0 0x7fa1e41464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 10 monclient(hunting): picked mon.noname-a con 0x7fa1e41561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e40c9360 con 0x7fa1e41561e0 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e40ca1e0 con 0x7fa1e4155e00 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e40c8c20 con 0x7fa1e4074dc0 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7fa1ea8f2640 10 monclient(hunting): authenticate will time out at 1059.402231s 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 1 -- 10.20.193.16:0/1122171649 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1400ad50 con 0x7f1f180c5370 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 10 monclient: dump: 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.909 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 1 -- 10.20.193.16:0/1122171649 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f14014420 con 0x7f1f180c5370 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 1 -- 10.20.193.16:0/1122171649 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14014810 con 0x7f1f180c5370 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f05ffb640 10 monclient: dump: 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f077fe640 4 set_mon_vals no callback set 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 5 monclient: authenticate success, global_id 4438 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 10 monclient: _renew_subs 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f180c3e50 con 0x7f1f180c5370 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 10 client.4438.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 10 monclient: _renew_subs 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f180c4410 con 0x7f1f180c5370 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 1 librados: init done 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7ff9437fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff9441561e0 0x7ff9441fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f18f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f18f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f18f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7ff9437fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff9441561e0 0x7ff9441fa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50142/0 (socket says 10.20.193.16:50142) 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7ff9437fe640 1 -- 10.20.193.16:0/994528860 learned_addr learned my addr 10.20.193.16:0/994528860 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7ff9437fe640 10 monclient(hunting): get_auth_request con 0x7ff9441561e0 auth_method 0 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7ff9437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7ff9437fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7ff9437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 10 client.4438.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 15 client.4438.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 20 client.4438.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 10 client.4438.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.833+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f18153df0 con 0x7f1f180c5370 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7fdd4d804640 10 monclient(hunting): handle_auth_done global_id 4398 payload 824 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7fdd4d804640 1 -- 10.20.193.16:0/1659368440 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdd48155fa0 msgr2=0x7fdd480c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7fdd4d804640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdd48155fa0 0x7fdd480c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7fdd4d804640 1 -- 10.20.193.16:0/1659368440 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd48156380 msgr2=0x7fdd480c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7fdd4d804640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd48156380 0x7fdd480c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7f43cd85e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f43c815fa10 0x7f43c80ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59618/0 (socket says 10.20.193.16:59618) 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7f43cd85e640 1 -- 10.20.193.16:0/2226323223 learned_addr learned my addr 10.20.193.16:0/2226323223 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7f43cd85e640 10 monclient(hunting): get_auth_request con 0x7f43c815fa10 auth_method 0 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7f43cd85e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7f43cd85e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7f43cd85e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7fdd4d804640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7fdd4d804640 1 monclient: found mon.noname-a 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7fdd4d804640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7fdd4d804640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7fdd4d804640 1 -- 10.20.193.16:0/1659368440 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd481f4df0 con 0x7fdd480745e0 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.834+0000 7f18f3fff640 10 monclient(hunting): handle_auth_done global_id 4472 payload 274 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.835+0000 7f18f3fff640 1 -- 10.20.193.16:0/2278536074 >> v2:10.20.193.16:3300/0 conn(0x7f18fc074dc0 msgr2=0x7f18fc0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.835+0000 7f18f3fff640 1 --2- 10.20.193.16:0/2278536074 >> v2:10.20.193.16:3300/0 conn(0x7f18fc074dc0 0x7f18fc0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.835+0000 7f18f3fff640 1 -- 10.20.193.16:0/2278536074 >> v1:10.20.193.148:6789/0 conn(0x7f18fc155e00 legacy=0x7f18fc074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.835+0000 7f18f3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.835+0000 7f18f3fff640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.835+0000 7f18f3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.835+0000 7f18f3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.835+0000 7f18f3fff640 1 -- 10.20.193.16:0/2278536074 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc0c9c20 con 0x7f18fc1561e0 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.836+0000 7f43cd85e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.836+0000 7f43cd85e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.836+0000 7f43cd85e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.836+0000 7ff9437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.836+0000 7ff9437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.836+0000 7ff9437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7fdd4d804640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd480745e0 0x7fdd481f48b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800b550 tx=0x7fdd3800ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7fdd367fc640 10 client.?.objecter ms_handle_connect 0x7fdd480745e0 2026-02-17T05:56:46.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7fdd367fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f7ebdbf2640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7eb81561e0 0x7f7eb815b9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009f50 tx=0x7f7eb4004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f7ea5ffb640 10 client.?.objecter ms_handle_connect 0x7f7eb81561e0 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f7ea5ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f43cd85e640 10 monclient(hunting): handle_auth_done global_id 4401 payload 274 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f43cd85e640 1 -- 10.20.193.16:0/2226323223 >> v1:10.20.193.148:6789/0 conn(0x7f43c81561e0 legacy=0x7f43c815f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f43cd85e640 1 -- 10.20.193.16:0/2226323223 >> v1:10.20.193.16:6789/0 conn(0x7f43c8155e00 legacy=0x7f43c8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 1 -- 10.20.193.16:0/2008726529 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c03c070 con 0x7f34481561e0 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f43cd85e640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f43cd85e640 1 monclient: found mon.noname-a 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f43cd85e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f43cd85e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f43cd85e640 1 -- 10.20.193.16:0/2226323223 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c80c9c20 con 0x7f43c815fa10 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 10 monclient: dump: 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f344f787640 5 monclient: authenticate success, global_id 4457 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 1 -- 10.20.193.16:0/2008726529 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c02faf0 con 0x7f34481561e0 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f344f787640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 1 -- 10.20.193.16:0/2008726529 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c037440 con 0x7f34481561e0 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f18f3fff640 1 --2- 10.20.193.16:0/2278536074 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f18fc1561e0 0x7f18fc0ca700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0693f0 tx=0x7f18e402eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=239c9f83f0e70e56 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.837+0000 7f3437fff640 10 monclient: dump: 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9427fc640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff944155e00 0x7ff9441f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 4 set_mon_vals no callback set 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9427fc640 10 monclient(hunting): get_auth_request con 0x7ff944155e00 auth_method 0 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9427fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 10 monclient: shutdown 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 1 -- 10.20.193.16:0/2008726529 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34481561e0 msgr2=0x7f34480cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 1 --2- 10.20.193.16:0/2008726529 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34481561e0 0x7f34480cc1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34480693f0 tx=0x7f343c02ede0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 1 -- 10.20.193.16:0/2008726529 shutdown_connections 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 1 --2- 10.20.193.16:0/2008726529 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34481561e0 0x7f34480cc1a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 1 --2- 10.20.193.16:0/2008726529 >> v2:10.20.193.16:3300/0 conn(0x7f3448155e00 0x7f34480ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f3bb77fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3bb8073170 0x7f3bb81f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 1 -- 10.20.193.16:0/2008726529 >> 10.20.193.16:0/2008726529 conn(0x7f34480ce6c0 msgr2=0x7f34480cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 1 -- 10.20.193.16:0/2008726529 shutdown_connections 2026-02-17T05:56:46.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7f344f787640 1 -- 10.20.193.16:0/2008726529 wait complete. 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9437fe640 10 monclient(hunting): handle_auth_done global_id 4450 payload 824 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9437fe640 1 -- 10.20.193.16:0/994528860 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff944155e00 msgr2=0x7ff9441f9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9437fe640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff944155e00 0x7ff9441f9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9437fe640 1 -- 10.20.193.16:0/994528860 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff944074dc0 msgr2=0x7ff9441415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9437fe640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff944074dc0 0x7ff9441415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9437fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9437fe640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9437fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.838+0000 7ff9437fe640 1 -- 10.20.193.16:0/994528860 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9441f8270 con 0x7ff9441561e0 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.839+0000 7f344f787640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.839+0000 7f344f787640 10 monclient: monmap: 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.839+0000 7f344f787640 1 librados: starting msgr at 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.839+0000 7f344f787640 1 librados: starting objecter 2026-02-17T05:56:46.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.839+0000 7f43cd85e640 1 --2- 10.20.193.16:0/2226323223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f43c815fa10 0x7f43c80ca700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009e50 tx=0x7f43c402f260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dac529d4217e0b6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.839+0000 7f344f787640 1 Processor -- start 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.839+0000 7f344f787640 1 -- start start 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.839+0000 7f344f787640 1 librados: setting wanted keys 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.839+0000 7f344f787640 1 librados: calling monclient init 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.839+0000 7f344f787640 10 monclient: init 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3448074dc0 0x7f34481f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 10 monclient(hunting): picked mon.noname-b con 0x7f3448074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3448155e00 0x7f344814d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 10 monclient(hunting): picked mon.noname-a con 0x7f3448155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34481561e0 0x7f34481f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 10 monclient(hunting): picked mon.noname-c con 0x7f34481561e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34480c9360 con 0x7f3448155e00 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f34480ca1e0 con 0x7f3448074dc0 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34480c8c20 con 0x7f34481561e0 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.840+0000 7f344f787640 10 monclient(hunting): authenticate will time out at 1059.409260s 2026-02-17T05:56:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f3bb77fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3bb8073170 0x7f3bb81f0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59680/0 (socket says 10.20.193.16:59680) 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f3bb77fe640 1 -- 10.20.193.16:0/2823223881 learned_addr learned my addr 10.20.193.16:0/2823223881 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f3bb77fe640 10 monclient(hunting): get_auth_request con 0x7f3bb8073170 auth_method 0 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f3bb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f3bb77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f3bb77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7ff9437fe640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff9441561e0 0x7ff9441fa490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff93400cb20 tx=0x7ff93400a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7ff9237fe640 10 client.?.objecter ms_handle_connect 0x7ff9441561e0 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7ff9237fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 1 -- 10.20.193.16:0/401859147 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0047070 con 0x7f0e000745e0 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 10 monclient: dump: 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.918 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 1 -- 10.20.193.16:0/401859147 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df0042630 con 0x7f0e000745e0 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 1 -- 10.20.193.16:0/401859147 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df003c040 con 0x7f0e000745e0 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df57fa640 10 monclient: dump: 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0df6ffd640 4 set_mon_vals no callback set 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0e06bb6640 5 monclient: authenticate success, global_id 4444 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0e06bb6640 10 monclient: _renew_subs 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0e06bb6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e001f6160 con 0x7f0e000745e0 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0e06bb6640 10 client.4444.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0e06bb6640 10 monclient: _renew_subs 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0e06bb6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e001f6720 con 0x7f0e000745e0 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.841+0000 7f0e06bb6640 1 librados: init done 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.842+0000 7f0e06bb6640 10 client.4444.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.842+0000 7f0e06bb6640 15 client.4444.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.842+0000 7f0e06bb6640 20 client.4444.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.842+0000 7f3bb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.842+0000 7f3bb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.842+0000 7f3bb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.842+0000 7f0e06bb6640 10 client.4444.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.842+0000 7f0e06bb6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.842+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e001f6960 con 0x7f0e000745e0 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.842+0000 7f3bb7fff640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3bb815b8e0 0x7f3bb814c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f3bb7fff640 10 monclient(hunting): get_auth_request con 0x7f3bb815b8e0 auth_method 0 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f3bb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f3bb7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f3bb7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7fa1e3fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa1e4074dc0 0x7fa1e41f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f6e84ed1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e801561e0 0x7f6e8013f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f3bb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f3bb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f3bb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f6e84ed1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e801561e0 0x7f6e8013f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56914/0 (socket says 10.20.193.16:56914) 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f6e84ed1640 1 -- 10.20.193.16:0/1534513711 learned_addr learned my addr 10.20.193.16:0/1534513711 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f6e84ed1640 10 monclient(hunting): get_auth_request con 0x7f6e801561e0 auth_method 0 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f6e84ed1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f6e84ed1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7f6e84ed1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7fa1e3fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa1e4074dc0 0x7fa1e41f47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56926/0 (socket says 10.20.193.16:56926) 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7fa1e3fff640 1 -- 10.20.193.16:0/4262848294 learned_addr learned my addr 10.20.193.16:0/4262848294 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7fa1e3fff640 10 monclient(hunting): get_auth_request con 0x7fa1e4074dc0 auth_method 0 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7fa1e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7fa1e3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.843+0000 7fa1e3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7f3bb77fe640 10 monclient(hunting): handle_auth_done global_id 4404 payload 824 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7f3bb77fe640 1 -- 10.20.193.16:0/2823223881 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3bb815b8e0 msgr2=0x7f3bb814c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7f3bb77fe640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3bb815b8e0 0x7f3bb814c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7f3bb77fe640 1 -- 10.20.193.16:0/2823223881 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3bb80cb250 msgr2=0x7f3bb8070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7f3bb77fe640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3bb80cb250 0x7f3bb8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7f3bb77fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7f3bb77fe640 1 monclient: found mon.noname-a 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7f3bb77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7f3bb77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7f3bb77fe640 1 -- 10.20.193.16:0/2823223881 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb81f2b40 con 0x7f3bb8073170 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7fd9dffff640 1 -- 10.20.193.16:0/860860856 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e403c070 con 0x7fd9f01561e0 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.844+0000 7fd9dffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 10 monclient: dump: 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 1 -- 10.20.193.16:0/860860856 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e402fad0 con 0x7fd9f01561e0 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 1 -- 10.20.193.16:0/860860856 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4037440 con 0x7fd9f01561e0 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 5 monclient: authenticate success, global_id 4389 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 4 set_mon_vals no callback set 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9dffff640 10 monclient: dump: 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.922 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 10 monclient: shutdown 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 1 -- 10.20.193.16:0/860860856 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd9f01561e0 msgr2=0x7fd9f00ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 1 --2- 10.20.193.16:0/860860856 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd9f01561e0 0x7fd9f00ca700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40098a0 tx=0x7fd9e402ee10 comp rx=0 tx=0).stop 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 1 -- 10.20.193.16:0/860860856 shutdown_connections 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 1 --2- 10.20.193.16:0/860860856 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd9f01561e0 0x7fd9f00ca700 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 1 -- 10.20.193.16:0/860860856 >> 10.20.193.16:0/860860856 conn(0x7fd9f00ce6c0 msgr2=0x7fd9f00cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 1 -- 10.20.193.16:0/860860856 shutdown_connections 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7fd9f77ba640 1 -- 10.20.193.16:0/860860856 wait complete. 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7f6e84ed1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7f6e84ed1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.845+0000 7f6e84ed1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient: monmap: 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.923 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fa1e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fa1e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fa1e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 librados: starting msgr at 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 librados: starting objecter 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 Processor -- start 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 -- start start 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 librados: setting wanted keys 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 librados: calling monclient init 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient: init 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7f3bb77fe640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3bb8073170 0x7f3bb81f0030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b500 tx=0x7f3ba400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7f3b93fff640 10 client.?.objecter ms_handle_connect 0x7f3bb8073170 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7f3b93fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd9f0074dc0 0x7fd9f01f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient(hunting): picked mon.noname-b con 0x7fd9f0074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd9f0155e00 0x7fd9f0153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient(hunting): picked mon.noname-a con 0x7fd9f0155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd9f01561e0 0x7fd9f00c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient(hunting): picked mon.noname-c con 0x7fd9f01561e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f00c9360 con 0x7fd9f0155e00 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f00ca1e0 con 0x7fd9f0074dc0 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f015fa10 con 0x7fd9f01561e0 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.846+0000 7fd9f77ba640 10 monclient(hunting): authenticate will time out at 1059.415953s 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.847+0000 7f6e84ed1640 10 monclient(hunting): handle_auth_done global_id 4481 payload 824 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.847+0000 7f6e84ed1640 1 -- 10.20.193.16:0/1534513711 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e80074dc0 msgr2=0x7f6e801f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.847+0000 7f6e84ed1640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e80074dc0 0x7f6e801f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.848+0000 7f6e84ed1640 1 -- 10.20.193.16:0/1534513711 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e80155e00 msgr2=0x7f6e8013ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.848+0000 7f6e84ed1640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e80155e00 0x7f6e8013ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.848+0000 7f6e84ed1640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.848+0000 7f6e84ed1640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.848+0000 7f6e84ed1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.848+0000 7f6e84ed1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.848+0000 7f6e84ed1640 1 -- 10.20.193.16:0/1534513711 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e801f7420 con 0x7f6e801561e0 2026-02-17T05:56:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.849+0000 7fa1e3fff640 10 monclient(hunting): handle_auth_done global_id 4484 payload 824 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.849+0000 7fa1e3fff640 1 -- 10.20.193.16:0/4262848294 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa1e4155e00 msgr2=0x7fa1e4141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.849+0000 7fa1e3fff640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa1e4155e00 0x7fa1e4141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.849+0000 7fa1e3fff640 1 -- 10.20.193.16:0/4262848294 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa1e41561e0 msgr2=0x7fa1e41464c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.849+0000 7fa1e3fff640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa1e41561e0 0x7fa1e41464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.849+0000 7fa1e3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.849+0000 7fa1e3fff640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.849+0000 7fa1e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.849+0000 7fa1e3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.849+0000 7fa1e3fff640 1 -- 10.20.193.16:0/4262848294 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e4148ef0 con 0x7fa1e4074dc0 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 1 -- 10.20.193.16:0/3490808065 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2803c070 con 0x7fdb381561e0 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 10 monclient: dump: 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 1 -- 10.20.193.16:0/3490808065 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2802fb70 con 0x7fdb381561e0 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 5 monclient: authenticate success, global_id 4466 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 1 -- 10.20.193.16:0/3490808065 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28037440 con 0x7fdb381561e0 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb36ffd640 10 monclient: dump: 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 4 set_mon_vals no callback set 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7f6e84ed1640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e801561e0 0x7f6e8013f2b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400b9c0 tx=0x7f6e7400be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7f6e7cff9640 10 client.?.objecter ms_handle_connect 0x7f6e801561e0 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7f6e7cff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fa1e3fff640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa1e4074dc0 0x7fa1e41f47e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000b570 tx=0x7fa1d000ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fa1e0ff9640 10 client.?.objecter ms_handle_connect 0x7fa1e4074dc0 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fa1e0ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 10 monclient: shutdown 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 1 -- 10.20.193.16:0/3490808065 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdb381561e0 msgr2=0x7fdb380ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 1 --2- 10.20.193.16:0/3490808065 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdb381561e0 0x7fdb380ca700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdb280098a0 tx=0x7fdb2802ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 1 -- 10.20.193.16:0/3490808065 shutdown_connections 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 1 --2- 10.20.193.16:0/3490808065 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdb381561e0 0x7fdb380ca700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 1 -- 10.20.193.16:0/3490808065 >> 10.20.193.16:0/3490808065 conn(0x7fdb380ce6c0 msgr2=0x7fdb380cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7fdb3e61a640 1 -- 10.20.193.16:0/3490808065 shutdown_connections 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7f18f37fe640 1 -- 10.20.193.16:0/2278536074 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e403c070 con 0x7f18fc1561e0 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.851+0000 7f18f37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 10 monclient: dump: 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 1 -- 10.20.193.16:0/2278536074 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18e402fb40 con 0x7f18fc1561e0 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 1 -- 10.20.193.16:0/2278536074 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4037440 con 0x7f18fc1561e0 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f18f37fe640 10 monclient: dump: 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.928 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 5 monclient: authenticate success, global_id 4472 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7fdb3e61a640 1 -- 10.20.193.16:0/3490808065 wait complete. 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 4 set_mon_vals no callback set 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 10 monclient: shutdown 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 1 -- 10.20.193.16:0/2278536074 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f18fc1561e0 msgr2=0x7f18fc0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 1 --2- 10.20.193.16:0/2278536074 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f18fc1561e0 0x7f18fc0ca700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0693f0 tx=0x7f18e402eed0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 1 -- 10.20.193.16:0/2278536074 shutdown_connections 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 1 --2- 10.20.193.16:0/2278536074 >> v2:10.20.193.16:3300/0 conn(0x7f18fc074dc0 0x7f18fc0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 1 --2- 10.20.193.16:0/2278536074 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f18fc1561e0 0x7f18fc0ca700 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 1 -- 10.20.193.16:0/2278536074 >> 10.20.193.16:0/2278536074 conn(0x7f18fc0ce6c0 msgr2=0x7f18fc0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 1 -- 10.20.193.16:0/2278536074 shutdown_connections 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7fdb3e61a640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7fdb3e61a640 10 monclient: monmap: 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.929 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7fdb3e61a640 1 librados: starting msgr at 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7fdb3e61a640 1 librados: starting objecter 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.852+0000 7f1902b75640 1 -- 10.20.193.16:0/2278536074 wait complete. 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 1 Processor -- start 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f1902b75640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f1902b75640 10 monclient: monmap: 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 1 -- start start 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 1 librados: setting wanted keys 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 1 librados: calling monclient init 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient: init 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f1902b75640 1 librados: starting msgr at 2026-02-17T05:56:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f1902b75640 1 librados: starting objecter 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f1902b75640 1 Processor -- start 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f344dcfd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34481561e0 0x7f34481f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f1902b75640 1 -- start start 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f1902b75640 1 librados: setting wanted keys 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f1902b75640 1 librados: calling monclient init 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f1902b75640 10 monclient: init 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdb38074dc0 0x7fdb3814dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient(hunting): picked mon.noname-b con 0x7fdb38074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdb38155e00 0x7fdb381f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient(hunting): picked mon.noname-a con 0x7fdb38155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdb381561e0 0x7fdb380c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient(hunting): picked mon.noname-c con 0x7fdb381561e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb380c9360 con 0x7fdb38155e00 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fdb380ca1e0 con 0x7fdb38074dc0 2026-02-17T05:56:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3815fa10 con 0x7fdb381561e0 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7fdb3e61a640 10 monclient(hunting): authenticate will time out at 1059.422878s 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.853+0000 7f1f05ffb640 1 -- 10.20.193.16:0/1122171649 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1f140133c0 con 0x7f1f180c5370 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1f05ffb640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ee8079190 0x7f1ee807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f18fc074dc0 0x7f18fc0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 10 monclient(hunting): picked mon.noname-b con 0x7f18fc074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f18fc155e00 0x7f18fc14db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 10 monclient(hunting): picked mon.noname-a con 0x7f18fc155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f18fc1561e0 0x7f18fc0c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 10 monclient(hunting): picked mon.noname-c con 0x7f18fc1561e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc0c9360 con 0x7f18fc155e00 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc0ca1e0 con 0x7f18fc074dc0 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc0c8c20 con 0x7f18fc1561e0 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1902b75640 10 monclient(hunting): authenticate will time out at 1059.423082s 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1f1d199640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ee8079190 0x7f1ee807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1f1d199640 10 monclient: get_auth_request con 0x7f1ee8079190 auth_method 0 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 1 -- 10.20.193.16:0/1238250860 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803800ce30 con 0x7f803c15d360 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 10 monclient: dump: 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 1 -- 10.20.193.16:0/1238250860 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8038003a70 con 0x7f803c15d360 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 1 -- 10.20.193.16:0/1238250860 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038003e60 con 0x7f803c15d360 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f804332c640 5 monclient: authenticate success, global_id 4469 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f804332c640 10 monclient: _renew_subs 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f804332c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f802b7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c1f78d0 con 0x7f803c15d360 2026-02-17T05:56:46.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f804332c640 10 client.4469.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f804332c640 10 monclient: _renew_subs 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f804332c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c1f3e40 con 0x7f803c15d360 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f804332c640 1 librados: init done 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f8029ffb640 10 monclient: dump: 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f344dcfd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34481561e0 0x7f34481f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56932/0 (socket says 10.20.193.16:56932) 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f344dcfd640 1 -- 10.20.193.16:0/1220374686 learned_addr learned my addr 10.20.193.16:0/1220374686 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f344dcfd640 10 monclient(hunting): get_auth_request con 0x7f34481561e0 auth_method 0 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f344dcfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f344dcfd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f344dcfd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1f1d199640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ee8079190 0x7f1ee807b650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1f08038000 tx=0x7f1f08033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.854+0000 7f1f05ffb640 10 client.4438.objecter ms_handle_connect 0x7f1ee8079190 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f804332c640 10 client.4469.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f804332c640 15 client.4469.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f804332c640 20 client.4469.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f804332c640 10 client.4469.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f804332c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f803c0c4f80 con 0x7f803c15d360 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 1 -- 10.20.193.16:0/2226323223 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c402fcd0 con 0x7f43c815fa10 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 10 monclient: dump: 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.935 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 1 -- 10.20.193.16:0/2226323223 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c40373f0 con 0x7f43c815fa10 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 1 -- 10.20.193.16:0/2226323223 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c40377c0 con 0x7f43c815fa10 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43b7fff640 10 monclient: dump: 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43cf2e8640 5 monclient: authenticate success, global_id 4401 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43cf2e8640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43cf2e8640 4 set_mon_vals no callback set 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 1 -- 10.20.193.16:0/1122171649 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f1f1405e390 con 0x7f1f180c5370 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 10 client.4438.objecter ms_dispatch2 0x7f1f18000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 3 client.4438.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 3 client.4438.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 10 client.4438.objecter checking command 1 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] conn(0x7f1ee80816d0 0x7f1ee8083b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 20 client.4438.objecter _get_session s=0x7f1ee8081000 osd=5 3 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 20 client.4438.objecter put_session s=0x7f1ee8081000 osd=5 3 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f344dcfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f344dcfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 15 client.4438.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f344dcfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 20 client.4438.objecter _get_session s=0x7f1ee8081000 osd=5 3 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43cf2e8640 10 monclient: shutdown 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43cf2e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43cf2e8640 1 -- 10.20.193.16:0/2226323223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f43c815fa10 msgr2=0x7f43c80ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f43cf2e8640 1 --2- 10.20.193.16:0/2226323223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f43c815fa10 0x7f43c80ca700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009e50 tx=0x7f43c402f260 comp rx=0 tx=0).stop 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 20 client.4438.objecter get_session s=0x7f1ee8081000 osd=5 3 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.855+0000 7f1f05ffb640 15 client.4438.objecter _session_command_op_assign 5 1 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f1f05ffb640 20 client.4438.objecter put_session s=0x7f1ee8081000 osd=5 4 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f1f05ffb640 10 client.4438.objecter _send_command 1 2026-02-17T05:56:46.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f1f05ffb640 1 -- 10.20.193.16:0/1122171649 --> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f14007ae0 con 0x7f1ee80816d0 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f1f05ffb640 20 client.4438.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f43cf2e8640 1 -- 10.20.193.16:0/2226323223 shutdown_connections 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 1 -- 10.20.193.16:0/2823223881 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4004480 con 0x7f3bb8073170 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 10 monclient: dump: 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f344ccfb640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3448155e00 0x7f344814d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 1 -- 10.20.193.16:0/2823223881 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba4014070 con 0x7f3bb8073170 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 5 monclient: authenticate success, global_id 4404 2026-02-17T05:56:46.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 10 monclient: _renew_subs 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f43cf2e8640 1 --2- 10.20.193.16:0/2226323223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f43c815fa10 0x7f43c80ca700 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb81f2de0 con 0x7f3bb8073170 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f1f05ffb640 1 -- 10.20.193.16:0/1122171649 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1f1405e750 con 0x7f1f180c5370 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f43cf2e8640 1 -- 10.20.193.16:0/2226323223 >> 10.20.193.16:0/2226323223 conn(0x7f43c80ce6c0 msgr2=0x7f43c80cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f1f05ffb640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f43cf2e8640 1 -- 10.20.193.16:0/2226323223 shutdown_connections 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 10 client.4404.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 10 monclient: _renew_subs 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb81f33a0 con 0x7f3bb8073170 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 1 librados: init done 2026-02-17T05:56:46.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f43cf2e8640 1 -- 10.20.193.16:0/2226323223 wait complete. 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 1 -- 10.20.193.16:0/2823223881 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4002920 con 0x7f3bb8073170 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bb5ffb640 4 set_mon_vals no callback set 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3b93fff640 10 monclient: dump: 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 1 -- 10.20.193.16:0/1659368440 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38004500 con 0x7fdd480745e0 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 10 monclient: dump: 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 1 -- 10.20.193.16:0/1659368440 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd38015070 con 0x7fdd480745e0 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 1 -- 10.20.193.16:0/1659368440 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd380029c0 con 0x7fdd480745e0 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd367fc640 10 monclient: dump: 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f1f1c998640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] conn(0x7f1ee80816d0 0x7f1ee8083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd37fff640 4 set_mon_vals no callback set 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd4fa8f640 5 monclient: authenticate success, global_id 4398 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd4fa8f640 10 monclient: _renew_subs 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd4fa8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd480c8ca0 con 0x7fdd480745e0 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f43cf2e8640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f43cf2e8640 10 monclient: monmap: 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.941 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd4fa8f640 10 client.4398.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd4fa8f640 10 monclient: _renew_subs 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd4fa8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd480c9240 con 0x7fdd480745e0 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7fdd4fa8f640 1 librados: init done 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 10 client.4404.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 15 client.4404.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 20 client.4404.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 10 client.4404.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3bb815bcc0 con 0x7f3bb8073170 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f344ccfb640 10 monclient(hunting): get_auth_request con 0x7f3448155e00 auth_method 0 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f344ccfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f344ccfb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f344ccfb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f1f1c998640 10 monclient: get_auth_request con 0x7f1ee80816d0 auth_method 0 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f43cf2e8640 1 librados: starting msgr at 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f43cf2e8640 1 librados: starting objecter 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.856+0000 7f43cf2e8640 1 Processor -- start 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 1 -- start start 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 1 librados: setting wanted keys 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 1 librados: calling monclient init 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient: init 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7fdd4fa8f640 10 client.4398.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7fdd4fa8f640 15 client.4398.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7fdd4fa8f640 20 client.4398.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7fdd4fa8f640 10 client.4398.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7fdd4fa8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd48154120 con 0x7fdd480745e0 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f43c8155e00 0x7f43c81f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient(hunting): picked mon.noname-b con 0x7f43c8155e00 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f43c81561e0 0x7f43c81475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient(hunting): picked mon.noname-a con 0x7f43c81561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f43c815fa10 0x7f43c81eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient(hunting): picked mon.noname-c con 0x7f43c815fa10 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80c9360 con 0x7f43c81561e0 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80ca1e0 con 0x7f43c8155e00 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8074dc0 con 0x7f43c815fa10 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f43cf2e8640 10 monclient(hunting): authenticate will time out at 1059.426518s 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f344dcfd640 10 monclient(hunting): handle_auth_done global_id 4487 payload 824 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f344dcfd640 1 -- 10.20.193.16:0/1220374686 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3448074dc0 msgr2=0x7f34481f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f344dcfd640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3448074dc0 0x7f34481f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f344dcfd640 1 -- 10.20.193.16:0/1220374686 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3448155e00 msgr2=0x7f344814d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f344dcfd640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3448155e00 0x7f344814d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f344dcfd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f344dcfd640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f344dcfd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f344dcfd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.857+0000 7f344dcfd640 1 -- 10.20.193.16:0/1220374686 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34481f4a00 con 0x7f34481561e0 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.858+0000 7f1f05ffb640 10 client.4438.objecter ms_handle_connect 0x7f1ee80816d0 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.858+0000 7f1f1c998640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] conn(0x7f1ee80816d0 0x7f1ee8083b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 1 -- 10.20.193.16:0/2452020534 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4046020 con 0x7f7eb81561e0 2026-02-17T05:56:46.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 10 monclient: dump: 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 1 -- 10.20.193.16:0/2452020534 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb4041450 con 0x7f7eb81561e0 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 1 -- 10.20.193.16:0/2452020534 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4037e60 con 0x7f7eb81561e0 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea5ffb640 10 monclient: dump: 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ea77fe640 4 set_mon_vals no callback set 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ebf67c640 5 monclient: authenticate success, global_id 4441 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ebf67c640 10 monclient: _renew_subs 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ebf67c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb8151690 con 0x7f7eb81561e0 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ebf67c640 10 client.4441.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ebf67c640 10 monclient: _renew_subs 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ebf67c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb8151c50 con 0x7f7eb81561e0 2026-02-17T05:56:46.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.862+0000 7f7ebf67c640 1 librados: init done 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f7ebf67c640 10 client.4441.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f7ebf67c640 15 client.4441.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f7ebf67c640 20 client.4441.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f7ebf67c640 10 client.4441.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f7ebf67c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7eb8076e00 con 0x7f7eb81561e0 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 1 -- 10.20.193.16:0/1534513711 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74006e60 con 0x7f6e801561e0 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 10 monclient: dump: 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 1 -- 10.20.193.16:0/1534513711 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e74010070 con 0x7f6e801561e0 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 1 -- 10.20.193.16:0/1534513711 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7400c9d0 con 0x7f6e801561e0 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7cff9640 10 monclient: dump: 2026-02-17T05:56:46.946 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e7e7fc640 4 set_mon_vals no callback set 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e8695b640 5 monclient: authenticate success, global_id 4481 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e8695b640 10 monclient: _renew_subs 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e8695b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e801f55a0 con 0x7f6e801561e0 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e8695b640 10 client.4481.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e8695b640 10 monclient: _renew_subs 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e8695b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e801f5b60 con 0x7f6e801561e0 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f6e8695b640 1 librados: init done 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f344dcfd640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34481561e0 0x7f34481f3b30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f344400cb80 tx=0x7f344400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f3435ffb640 10 client.?.objecter ms_handle_connect 0x7f34481561e0 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.863+0000 7f3435ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.864+0000 7f6e8695b640 10 client.4481.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.864+0000 7f6e8695b640 15 client.4481.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.864+0000 7f6e8695b640 20 client.4481.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.864+0000 7f6e8695b640 10 client.4481.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.864+0000 7f6e8695b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.864+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e800c4ac0 con 0x7f6e801561e0 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.864+0000 7f208affd640 1 -- 10.20.193.16:0/578125960 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f2098007210 con 0x7f20a40c7d00 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.865+0000 7f208affd640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2070079190 0x7f207007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.865+0000 7f20a1d74640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2070079190 0x7f207007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.865+0000 7f20a1d74640 10 monclient: get_auth_request con 0x7f2070079190 auth_method 0 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.865+0000 7f208affd640 10 client.4377.objecter ms_handle_connect 0x7f2070079190 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.865+0000 7f20a1d74640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2070079190 0x7f207007b650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f208c039000 tx=0x7f208c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.866+0000 7f208affd640 1 -- 10.20.193.16:0/578125960 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f209805d930 con 0x7f20a40c7d00 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.866+0000 7f208affd640 10 client.4377.objecter ms_dispatch2 0x7f20a4000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.866+0000 7f208affd640 3 client.4377.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.866+0000 7f208affd640 3 client.4377.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 10 client.4377.objecter checking command 1 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] conn(0x7f2070081730 0x7f2070083b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 20 client.4377.objecter _get_session s=0x7f2070081060 osd=0 3 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 20 client.4377.objecter put_session s=0x7f2070081060 osd=0 3 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 15 client.4377.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 20 client.4377.objecter _get_session s=0x7f2070081060 osd=0 3 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 20 client.4377.objecter get_session s=0x7f2070081060 osd=0 3 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 15 client.4377.objecter _session_command_op_assign 0 1 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 20 client.4377.objecter put_session s=0x7f2070081060 osd=0 4 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 10 client.4377.objecter _send_command 1 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 1 -- 10.20.193.16:0/578125960 --> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2070084240 con 0x7f2070081730 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 20 client.4377.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 1 -- 10.20.193.16:0/578125960 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f209805dcf0 con 0x7f20a40c7d00 2026-02-17T05:56:46.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f208affd640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7fd9f4d2e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd9f0155e00 0x7fd9f0153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f20a2d76640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] conn(0x7f2070081730 0x7f2070083b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.867+0000 7f20a2d76640 10 monclient: get_auth_request con 0x7f2070081730 auth_method 0 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.869+0000 7f0df57fa640 1 -- 10.20.193.16:0/401859147 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f0df003f3c0 con 0x7f0e000745e0 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.870+0000 7f20a2d76640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] conn(0x7f2070081730 0x7f2070083b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.870+0000 7f208affd640 10 client.4377.objecter ms_handle_connect 0x7f2070081730 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.870+0000 7f0df57fa640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0dd0079190 0x7f0dd007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.870+0000 7f0df7fff640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0dd0079190 0x7f0dd007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.870+0000 7f0df7fff640 10 monclient: get_auth_request con 0x7f0dd0079190 auth_method 0 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.870+0000 7f0df7fff640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0dd0079190 0x7f0dd007b650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0de8006fd0 tx=0x7f0de8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.870+0000 7f0df57fa640 10 client.4444.objecter ms_handle_connect 0x7f0dd0079190 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 1 -- 10.20.193.16:0/401859147 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f0df002fac0 con 0x7f0e000745e0 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 10 client.4444.objecter ms_dispatch2 0x7f0e00000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 3 client.4444.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 3 client.4444.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 10 client.4444.objecter checking command 1 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] conn(0x7f0dd00816d0 0x7f0dd0083b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 20 client.4444.objecter _get_session s=0x7f0dd0081000 osd=11 3 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 20 client.4444.objecter put_session s=0x7f0dd0081000 osd=11 3 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 15 client.4444.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 20 client.4444.objecter _get_session s=0x7f0dd0081000 osd=11 3 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 20 client.4444.objecter get_session s=0x7f0dd0081000 osd=11 3 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 15 client.4444.objecter _session_command_op_assign 11 1 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 20 client.4444.objecter put_session s=0x7f0dd0081000 osd=11 4 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 10 client.4444.objecter _send_command 1 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 1 -- 10.20.193.16:0/401859147 --> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0dd00841e0 con 0x7f0dd00816d0 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 20 client.4444.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 1 -- 10.20.193.16:0/401859147 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0df00835f0 con 0x7f0e000745e0 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0df57fa640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7fd9f4d2e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd9f0155e00 0x7fd9f0153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59722/0 (socket says 10.20.193.16:59722) 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7fd9f4d2e640 1 -- 10.20.193.16:0/2689701979 learned_addr learned my addr 10.20.193.16:0/2689701979 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7fd9f4d2e640 10 monclient(hunting): get_auth_request con 0x7fd9f0155e00 auth_method 0 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7fd9f4d2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7fd9f4d2e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7fd9f4d2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.871+0000 7f0e0512c640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] conn(0x7f0dd00816d0 0x7f0dd0083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.872+0000 7f0e0512c640 10 monclient: get_auth_request con 0x7f0dd00816d0 auth_method 0 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.874+0000 7f0df57fa640 10 client.4444.objecter ms_handle_connect 0x7f0dd00816d0 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.874+0000 7f0e0512c640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] conn(0x7f0dd00816d0 0x7f0dd0083b10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.876+0000 7f8029ffb640 1 -- 10.20.193.16:0/1238250860 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f80380050d0 con 0x7f803c15d360 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.876+0000 7fd9f552f640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd9f0074dc0 0x7fd9f01f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.876+0000 7f8029ffb640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 0x7f800c07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f80410a1640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 0x7f800c07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f80410a1640 10 monclient: get_auth_request con 0x7f800c079190 auth_method 0 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f80410a1640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 0x7f800c07b650 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f803c1603b0 tx=0x7f802c031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f8029ffb640 10 client.4469.objecter ms_handle_connect 0x7f800c079190 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7fd9f552f640 10 monclient(hunting): get_auth_request con 0x7fd9f0074dc0 auth_method 0 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7fd9f552f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7fd9f552f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7fd9f552f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 1 -- 10.20.193.16:0/1562582750 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960802fcd0 con 0x7f9618155e00 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 10 monclient: dump: 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.951 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 1 -- 10.20.193.16:0/1562582750 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96080373f0 con 0x7f9618155e00 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f961eba0640 5 monclient: authenticate success, global_id 4435 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f961eba0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 1 -- 10.20.193.16:0/1562582750 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96080377c0 con 0x7f9618155e00 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f960f7fe640 10 monclient: dump: 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f961eba0640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.877+0000 7f961eba0640 4 set_mon_vals no callback set 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 1 -- 10.20.193.16:0/1238250860 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f803805e8a0 con 0x7f803c15d360 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 10 client.4469.objecter ms_dispatch2 0x7f803c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 3 client.4469.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 3 client.4469.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7fd9f552f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7fd9f552f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7fd9f552f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 10 client.4469.objecter checking command 1 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x7f800c0816d0 0x7f800c083b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 20 client.4469.objecter _get_session s=0x7f800c081000 osd=4 3 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 20 client.4469.objecter put_session s=0x7f800c081000 osd=4 3 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 15 client.4469.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 20 client.4469.objecter _get_session s=0x7f800c081000 osd=4 3 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 20 client.4469.objecter get_session s=0x7f800c081000 osd=4 3 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 15 client.4469.objecter _session_command_op_assign 4 1 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 20 client.4469.objecter put_session s=0x7f800c081000 osd=4 4 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 10 client.4469.objecter _send_command 1 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 1 -- 10.20.193.16:0/1238250860 --> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8038004ec0 con 0x7f800c0816d0 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 20 client.4469.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 1 -- 10.20.193.16:0/1238250860 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f803805ec60 con 0x7f803c15d360 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 10 monclient: shutdown 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f19008ea640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f18fc074dc0 0x7f18fc0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 1 -- 10.20.193.16:0/1562582750 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9618155e00 msgr2=0x7f96180ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 1 --2- 10.20.193.16:0/1562582750 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9618155e00 0x7f96180ca700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f961806ce20 tx=0x7f960802f270 comp rx=0 tx=0).stop 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f8029ffb640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 1 -- 10.20.193.16:0/1562582750 shutdown_connections 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 1 --2- 10.20.193.16:0/1562582750 >> v2:10.20.193.16:3300/0 conn(0x7f9618074dc0 0x7f96180cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 1 --2- 10.20.193.16:0/1562582750 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9618155e00 0x7f96180ca700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 1 -- 10.20.193.16:0/1562582750 >> 10.20.193.16:0/1562582750 conn(0x7f96180ce6c0 msgr2=0x7f96180cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 1 -- 10.20.193.16:0/1562582750 shutdown_connections 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 1 -- 10.20.193.16:0/1562582750 wait complete. 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f80408a0640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x7f800c0816d0 0x7f800c083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 10 monclient: build_initial_monmap 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7f961eba0640 10 monclient: monmap: 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 1 -- 10.20.193.16:0/994528860 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93400acb0 con 0x7ff9441561e0 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 10 monclient: dump: 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 1 -- 10.20.193.16:0/994528860 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff934012400 con 0x7ff9441561e0 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 1 -- 10.20.193.16:0/994528860 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9340127f0 con 0x7ff9441561e0 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9417fa640 4 set_mon_vals no callback set 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9493d3640 5 monclient: authenticate success, global_id 4450 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.878+0000 7ff9237fe640 10 monclient: dump: 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 10 monclient: _renew_subs 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff944144740 con 0x7ff9441561e0 2026-02-17T05:56:46.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f19008ea640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f18fc074dc0 0x7f18fc0c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50204/0 (socket says 10.20.193.16:50204) 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f80408a0640 10 monclient: get_auth_request con 0x7f800c0816d0 auth_method 0 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f19008ea640 1 -- 10.20.193.16:0/2230245128 learned_addr learned my addr 10.20.193.16:0/2230245128 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f19008ea640 10 monclient(hunting): get_auth_request con 0x7f18fc074dc0 auth_method 0 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f19008ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f19008ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f19008ea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 librados: starting msgr at 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 librados: starting objecter 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 10 client.4450.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 10 monclient: _renew_subs 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff944144d00 con 0x7ff9441561e0 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 1 librados: init done 2026-02-17T05:56:46.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 Processor -- start 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 -- start start 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 librados: setting wanted keys 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 librados: calling monclient init 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient: init 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9618074dc0 0x7f96181ef620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient(hunting): picked mon.noname-b con 0x7f9618074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9618155e00 0x7f96181434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient(hunting): picked mon.noname-a con 0x7f9618155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f96181561e0 0x7f96180d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient(hunting): picked mon.noname-c con 0x7f96181561e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96180c9360 con 0x7f9618155e00 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f96180ca1e0 con 0x7f9618074dc0 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96180c8c20 con 0x7f96181561e0 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7f961eba0640 10 monclient(hunting): authenticate will time out at 1059.448581s 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 10 client.4450.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 15 client.4450.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 20 client.4450.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 10 client.4450.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.879+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9440c8fc0 con 0x7ff9441561e0 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7f3b93fff640 1 -- 10.20.193.16:0/2823223881 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f3ba4019460 con 0x7f3bb8073170 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7fd9f552f640 10 monclient(hunting): handle_auth_done global_id 4453 payload 824 2026-02-17T05:56:46.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7fd9f552f640 1 -- 10.20.193.16:0/2689701979 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd9f01561e0 msgr2=0x7fd9f00c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7fd9f552f640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd9f01561e0 0x7fd9f00c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7fd9f552f640 1 -- 10.20.193.16:0/2689701979 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd9f0155e00 msgr2=0x7fd9f0153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7fd9f552f640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd9f0155e00 0x7fd9f0153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7fd9f552f640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7fd9f552f640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7fd9f552f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7fd9f552f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7fd9f552f640 1 -- 10.20.193.16:0/2689701979 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f00c7880 con 0x7fd9f0074dc0 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7f80408a0640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x7f800c0816d0 0x7f800c083b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.880+0000 7f8029ffb640 10 client.4469.objecter ms_handle_connect 0x7f800c0816d0 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.881+0000 7f3b93fff640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3b88079190 0x7f3b8807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.881+0000 7f3bb6ffd640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3b88079190 0x7f3b8807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.881+0000 7f3bb6ffd640 10 monclient: get_auth_request con 0x7f3b88079190 auth_method 0 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.881+0000 7f3bb6ffd640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3b88079190 0x7f3b8807b650 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8006fd0 tx=0x7f3ba8006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.881+0000 7f3b93fff640 10 client.4404.objecter ms_handle_connect 0x7f3b88079190 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.881+0000 7f19008ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.881+0000 7f19008ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.881+0000 7f19008ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 1 -- 10.20.193.16:0/2823223881 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f3ba405def0 con 0x7f3bb8073170 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 10 client.4404.objecter ms_dispatch2 0x7f3bb8000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 3 client.4404.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 3 client.4404.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 10 client.4404.objecter checking command 1 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] conn(0x7f3b880816d0 0x7f3b88083b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 20 client.4404.objecter _get_session s=0x7f3b88081000 osd=2 3 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 20 client.4404.objecter put_session s=0x7f3b88081000 osd=2 3 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 15 client.4404.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 20 client.4404.objecter _get_session s=0x7f3b88081000 osd=2 3 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 20 client.4404.objecter get_session s=0x7f3b88081000 osd=2 3 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 15 client.4404.objecter _session_command_op_assign 2 1 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 20 client.4404.objecter put_session s=0x7f3b88081000 osd=2 4 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 10 client.4404.objecter _send_command 1 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 1 -- 10.20.193.16:0/2823223881 --> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b880841e0 con 0x7f3b880816d0 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 20 client.4404.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 1 -- 10.20.193.16:0/2823223881 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3ba405e2b0 con 0x7f3bb8073170 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.882+0000 7f3b93fff640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f1f05ffb640 1 -- 10.20.193.16:0/1122171649 <== osd.5 v2:10.20.193.128:6824/1593443319 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f1f14007ae0 con 0x7f1ee80816d0 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f1f05ffb640 10 client.4438.objecter ms_dispatch2 0x7f1f18000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f1f05ffb640 10 client.4438.objecter _finish_command 1 = system:0 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f1f05ffb640 20 client.4438.objecter put_session s=0x7f1ee8081000 osd=5 4 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f1f05ffb640 15 client.4438.objecter _session_command_op_remove 5 1 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f3bb7fff640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] conn(0x7f3b880816d0 0x7f3b88083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f3bb7fff640 10 monclient: get_auth_request con 0x7f3b880816d0 auth_method 0 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f19008ea640 10 monclient(hunting): handle_auth_done global_id 4456 payload 824 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f19008ea640 1 -- 10.20.193.16:0/2230245128 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f18fc1561e0 msgr2=0x7f18fc0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f19008ea640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f18fc1561e0 0x7f18fc0c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f19008ea640 1 -- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f18fc155e00 msgr2=0x7f18fc14db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f19008ea640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f18fc155e00 0x7f18fc14db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f19008ea640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f19008ea640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f19008ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f19008ea640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.883+0000 7f19008ea640 1 -- 10.20.193.16:0/2230245128 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc1fc500 con 0x7f18fc074dc0 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.884+0000 7fdb37fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdb38074dc0 0x7fdb3814dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.884+0000 7fd9f552f640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd9f0074dc0 0x7fd9f01f46d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000b550 tx=0x7fd9e000ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.884+0000 7fd9ddffb640 10 client.?.objecter ms_handle_connect 0x7fd9f0074dc0 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.884+0000 7fd9ddffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7fdb37fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdb38074dc0 0x7fdb3814dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50202/0 (socket says 10.20.193.16:50202) 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7fdb37fff640 1 -- 10.20.193.16:0/130213896 learned_addr learned my addr 10.20.193.16:0/130213896 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7fdb37fff640 10 monclient(hunting): get_auth_request con 0x7fdb38074dc0 auth_method 0 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7fdb37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7fdb37fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7fdb37fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7f6e7cff9640 1 -- 10.20.193.16:0/1534513711 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f6e7400cb70 con 0x7f6e801561e0 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7f3b93fff640 10 client.4404.objecter ms_handle_connect 0x7f3b880816d0 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7f3bb7fff640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] conn(0x7f3b880816d0 0x7f3b88083b10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7f6e7cff9640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e50079190 0x7f6e5007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7f6e7ffff640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e50079190 0x7f6e5007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7f6e7ffff640 10 monclient: get_auth_request con 0x7f6e50079190 auth_method 0 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7fdb37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7fdb37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.885+0000 7fdb37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 10 client.4481.objecter ms_handle_connect 0x7f6e50079190 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7ffff640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e50079190 0x7f6e5007b650 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c007fd0 tx=0x7f6e6c007b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 1 -- 10.20.193.16:0/1534513711 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f6e74012e00 con 0x7f6e801561e0 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 10 client.4481.objecter ms_dispatch2 0x7f6e80000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 3 client.4481.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 3 client.4481.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 10 client.4481.objecter checking command 1 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] conn(0x7f6e500816d0 0x7f6e50083b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 20 client.4481.objecter _get_session s=0x7f6e50081000 osd=1 3 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 20 client.4481.objecter put_session s=0x7f6e50081000 osd=1 3 2026-02-17T05:56:46.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 15 client.4481.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 20 client.4481.objecter _get_session s=0x7f6e50081000 osd=1 3 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 20 client.4481.objecter get_session s=0x7f6e50081000 osd=1 3 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 15 client.4481.objecter _session_command_op_assign 1 1 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 20 client.4481.objecter put_session s=0x7f6e50081000 osd=1 4 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 10 client.4481.objecter _send_command 1 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 1 -- 10.20.193.16:0/1534513711 --> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e7400ee50 con 0x7f6e500816d0 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 20 client.4481.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f19008ea640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f18fc074dc0 0x7f18fc0c7fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b5d0 tx=0x7f18ec00baa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f18f17fa640 10 client.?.objecter ms_handle_connect 0x7f18fc074dc0 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f18f17fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 1 -- 10.20.193.16:0/1534513711 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6e7405e160 con 0x7f6e801561e0 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.886+0000 7f6e7cff9640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.887+0000 7f6e7f7fe640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] conn(0x7f6e500816d0 0x7f6e50083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.887+0000 7f6e7f7fe640 10 monclient: get_auth_request con 0x7f6e500816d0 auth_method 0 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.887+0000 7fdb3cb90640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdb381561e0 0x7fdb380c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f961d116640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f96181561e0 0x7f96180d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fdb37fff640 10 monclient(hunting): handle_auth_done global_id 4459 payload 824 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 1 -- 10.20.193.16:0/4262848294 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0013020 con 0x7fa1e4074dc0 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fdb37fff640 1 -- 10.20.193.16:0/130213896 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdb381561e0 msgr2=0x7fdb380c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fdb37fff640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdb381561e0 0x7fdb380c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fdb37fff640 1 -- 10.20.193.16:0/130213896 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdb38155e00 msgr2=0x7fdb381f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fdb37fff640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdb38155e00 0x7fdb381f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 10 monclient: dump: 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fdb37fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fdb37fff640 1 monclient: found mon.noname-b 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fdb37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fdb37fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fdb37fff640 1 -- 10.20.193.16:0/130213896 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb380c85f0 con 0x7fdb38074dc0 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 1 -- 10.20.193.16:0/4262848294 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d00044a0 con 0x7fa1e4074dc0 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e27fc640 4 set_mon_vals no callback set 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f961d116640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f96181561e0 0x7f96180d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56958/0 (socket says 10.20.193.16:56958) 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f961d116640 1 -- 10.20.193.16:0/3204312853 learned_addr learned my addr 10.20.193.16:0/3204312853 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f961d116640 10 monclient(hunting): get_auth_request con 0x7f96181561e0 auth_method 0 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f961d116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f961d116640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f961d116640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 1 -- 10.20.193.16:0/4262848294 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d00029c0 con 0x7fa1e4074dc0 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1e0ff9640 10 monclient: dump: 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.964 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1ea8f2640 5 monclient: authenticate success, global_id 4484 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1ea8f2640 10 monclient: _renew_subs 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1ea8f2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e4147020 con 0x7fa1e4074dc0 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1ea8f2640 10 client.4484.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1ea8f2640 10 monclient: _renew_subs 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1ea8f2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e4149190 con 0x7fa1e4074dc0 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7fa1ea8f2640 1 librados: init done 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 1 -- 10.20.193.16:0/1220374686 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444017070 con 0x7f34481561e0 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 10 monclient: dump: 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 1 -- 10.20.193.16:0/1220374686 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344400ad50 con 0x7f34481561e0 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 1 -- 10.20.193.16:0/1220374686 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444012600 con 0x7f34481561e0 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f3435ffb640 10 monclient: dump: 2026-02-17T05:56:46.965 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.888+0000 7f34377fe640 4 set_mon_vals no callback set 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 5 monclient: authenticate success, global_id 4487 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 10 monclient: _renew_subs 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34481f4720 con 0x7f34481561e0 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 10 client.4487.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 10 monclient: _renew_subs 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34480c5b80 con 0x7f34481561e0 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 1 librados: init done 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f43cd85e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f43c815fa10 0x7f43c81eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f961d116640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f961d116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f961d116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7fa1ea8f2640 10 client.4484.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7fa1ea8f2640 15 client.4484.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7fa1ea8f2640 20 client.4484.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7fa1ea8f2640 10 client.4484.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7fa1ea8f2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1e40c8fa0 con 0x7fa1e4074dc0 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f43cd85e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f43c815fa10 0x7f43c81eabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56948/0 (socket says 10.20.193.16:56948) 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f43cd85e640 1 -- 10.20.193.16:0/1961814941 learned_addr learned my addr 10.20.193.16:0/1961814941 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f43cd85e640 10 monclient(hunting): get_auth_request con 0x7f43c815fa10 auth_method 0 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f43cd85e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f43cd85e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f43cd85e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f43cd05d640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f43c8155e00 0x7f43c81f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 10 client.4487.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 15 client.4487.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 20 client.4487.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 10 client.4487.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34480c8fa0 con 0x7f34481561e0 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 10 client.4438.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 15 client.4438.objecter _session_command_op_assign -1 2 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 20 client.4438.objecter _get_session s=0x7f1ee8081000 osd=5 3 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 20 client.4438.objecter put_session s=0x7f1ee8081000 osd=5 3 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 20 client.4438.objecter _get_session s=0x7f1ee8081000 osd=5 3 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 15 client.4438.objecter _session_command_op_remove -1 2 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 20 client.4438.objecter get_session s=0x7f1ee8081000 osd=5 3 2026-02-17T05:56:46.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 15 client.4438.objecter _session_command_op_assign 5 2 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 20 client.4438.objecter put_session s=0x7f1ee8081000 osd=5 4 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 10 client.4438.objecter _send_command 2 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.889+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 --> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f1806bcd0 con 0x7f1ee80816d0 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.890+0000 7f6e7cff9640 10 client.4481.objecter ms_handle_connect 0x7f6e500816d0 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.890+0000 7f6e7f7fe640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] conn(0x7f6e500816d0 0x7f6e50083b10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f961d116640 10 monclient(hunting): handle_auth_done global_id 4490 payload 824 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f961d116640 1 -- 10.20.193.16:0/3204312853 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9618074dc0 msgr2=0x7f96181ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f961d116640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9618074dc0 0x7f96181ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f961d116640 1 -- 10.20.193.16:0/3204312853 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9618155e00 msgr2=0x7f96181434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f961d116640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9618155e00 0x7f96181434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f961d116640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f961d116640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f961d116640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f961d116640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f961d116640 1 -- 10.20.193.16:0/3204312853 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96181f0520 con 0x7f96181561e0 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f43cd85e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f43cd85e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.891+0000 7f43cd85e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:46.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.892+0000 7f208affd640 1 -- 10.20.193.16:0/578125960 <== osd.0 v2:10.20.193.16:6826/821011033 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f2070084240 con 0x7f2070081730 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.892+0000 7f208affd640 10 client.4377.objecter ms_dispatch2 0x7f20a4000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.892+0000 7f208affd640 10 client.4377.objecter _finish_command 1 = system:0 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.892+0000 7f208affd640 20 client.4377.objecter put_session s=0x7f2070081060 osd=0 4 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.892+0000 7f208affd640 15 client.4377.objecter _session_command_op_remove 0 1 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.892+0000 7f43cd85e640 10 monclient(hunting): handle_auth_done global_id 4493 payload 824 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f43cd85e640 1 -- 10.20.193.16:0/1961814941 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f43c8155e00 msgr2=0x7f43c81f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f43cd85e640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f43c8155e00 0x7f43c81f76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f43cd85e640 1 -- 10.20.193.16:0/1961814941 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f43c81561e0 msgr2=0x7f43c81475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f43cd85e640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f43c81561e0 0x7f43c81475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f43cd85e640 10 monclient: _finish_hunting 0 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f43cd85e640 1 monclient: found mon.noname-c 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f43cd85e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f43cd85e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f43cd85e640 1 -- 10.20.193.16:0/1961814941 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c81503e0 con 0x7f43c815fa10 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f05ffb640 1 -- 10.20.193.16:0/1122171649 <== osd.5 v2:10.20.193.128:6824/1593443319 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1f1806bcd0 con 0x7f1ee80816d0 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f05ffb640 10 client.4438.objecter ms_dispatch2 0x7f1f18000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f05ffb640 10 client.4438.objecter _finish_command 2 = system:0 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f05ffb640 20 client.4438.objecter put_session s=0x7f1ee8081000 osd=5 4 2026-02-17T05:56:46.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f05ffb640 15 client.4438.objecter _session_command_op_remove 5 2 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 10 librados: watch_flush enter 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 10 librados: watch_flush exit 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 10 client.4438.objecter close_session for osd.5 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 >> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] conn(0x7f1ee80816d0 msgr2=0x7f1ee8083b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] conn(0x7f1ee80816d0 0x7f1ee8083b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 20 client.4438.objecter put_session s=0x7f1ee8081000 osd=5 1 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 20 client.4438.objecter shutdown clearing up homeless session... 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 10 client.4438.objecter successfully canceled tick 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ee8079190 msgr2=0x7f1ee807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ee8079190 0x7f1ee807b650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1f08038000 tx=0x7f1f08033000 comp rx=0 tx=0).stop 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 10 monclient: shutdown 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f180c5370 msgr2=0x7f1f1813cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 --2- 10.20.193.16:0/1122171649 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f180c5370 0x7f1f1813cde0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f180d0680 tx=0x7f1f1400a430 comp rx=0 tx=0).stop 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1d99a640 1 -- 10.20.193.16:0/1122171649 reap_dead start 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 shutdown_connections 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 >> 10.20.193.16:0/1122171649 conn(0x7f1f180cf580 msgr2=0x7f1f180c7400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 shutdown_connections 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 -- 10.20.193.16:0/1122171649 wait complete. 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f1f1f424640 1 librados: shutdown 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f0df57fa640 1 -- 10.20.193.16:0/401859147 <== osd.11 v2:10.20.193.164:6808/188912883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f0dd00841e0 con 0x7f0dd00816d0 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f0df57fa640 10 client.4444.objecter ms_dispatch2 0x7f0e00000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f0df57fa640 10 client.4444.objecter _finish_command 1 = system:0 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f0df57fa640 20 client.4444.objecter put_session s=0x7f0dd0081000 osd=11 4 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.893+0000 7f0df57fa640 15 client.4444.objecter _session_command_op_remove 11 1 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.894+0000 7f961d116640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f96181561e0 0x7f96180d0b20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f961400b550 tx=0x7f961400ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.894+0000 7f960d7fa640 10 client.?.objecter ms_handle_connect 0x7f96181561e0 2026-02-17T05:56:46.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.894+0000 7f960d7fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.895+0000 7fdd367fc640 1 -- 10.20.193.16:0/1659368440 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fdd38010480 con 0x7fdd480745e0 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.896+0000 7fdd367fc640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd1c079140 0x7fdd1c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.896+0000 7fdd4d003640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd1c079140 0x7fdd1c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.896+0000 7fdd4d003640 10 monclient: get_auth_request con 0x7fdd1c079140 auth_method 0 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.896+0000 7fdd4d003640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd1c079140 0x7fdd1c07b600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdd480cb290 tx=0x7fdd3c006cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.896+0000 7fdd367fc640 10 client.4398.objecter ms_handle_connect 0x7fdd1c079140 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7f7ea5ffb640 1 -- 10.20.193.16:0/2452020534 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f7eb4041820 con 0x7f7eb81561e0 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7f7ea5ffb640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7e88079190 0x7f7e8807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7f7ebd3f1640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7e88079190 0x7f7e8807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7f7ebd3f1640 10 monclient: get_auth_request con 0x7f7e88079190 auth_method 0 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 1 -- 10.20.193.16:0/1659368440 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fdd38097ee0 con 0x7fdd480745e0 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 10 client.4398.objecter ms_dispatch2 0x7fdd48000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 3 client.4398.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 3 client.4398.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 10 client.4398.objecter checking command 1 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] conn(0x7fdd1c081680 0x7fdd1c083ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 20 client.4398.objecter _get_session s=0x7fdd1c080fb0 osd=8 3 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 20 client.4398.objecter put_session s=0x7fdd1c080fb0 osd=8 3 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 15 client.4398.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 20 client.4398.objecter _get_session s=0x7fdd1c080fb0 osd=8 3 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 20 client.4398.objecter get_session s=0x7fdd1c080fb0 osd=8 3 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 15 client.4398.objecter _session_command_op_assign 8 1 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 20 client.4398.objecter put_session s=0x7fdd1c080fb0 osd=8 4 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 10 client.4398.objecter _send_command 1 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 1 -- 10.20.193.16:0/1659368440 --> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd3801be50 con 0x7fdd1c081680 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 20 client.4398.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 1 -- 10.20.193.16:0/1659368440 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdd3805e7e0 con 0x7fdd480745e0 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7f7ea5ffb640 10 client.4441.objecter ms_handle_connect 0x7f7e88079190 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7f7ebd3f1640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7e88079190 0x7f7e8807b650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80098a0 tx=0x7f7ea8006d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.897+0000 7fdd367fc640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.898+0000 7fdd4e005640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] conn(0x7fdd1c081680 0x7fdd1c083ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.898+0000 7fdd4e005640 10 monclient: get_auth_request con 0x7fdd1c081680 auth_method 0 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f43cd85e640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f43c815fa10 0x7f43c81eabc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009e50 tx=0x7f43c4004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f43b5ffb640 10 client.?.objecter ms_handle_connect 0x7f43c815fa10 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f43b5ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 1 -- 10.20.193.16:0/2452020534 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f7eb4083e20 con 0x7f7eb81561e0 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 10 client.4441.objecter ms_dispatch2 0x7f7eb8000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 3 client.4441.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 3 client.4441.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 10 client.4441.objecter checking command 1 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] conn(0x7f7e880816d0 0x7f7e88083b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 20 client.4441.objecter _get_session s=0x7f7e88081000 osd=10 3 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 20 client.4441.objecter put_session s=0x7f7e88081000 osd=10 3 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 15 client.4441.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 20 client.4441.objecter _get_session s=0x7f7e88081000 osd=10 3 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 20 client.4441.objecter get_session s=0x7f7e88081000 osd=10 3 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 15 client.4441.objecter _session_command_op_assign 10 1 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 20 client.4441.objecter put_session s=0x7f7e88081000 osd=10 4 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 10 client.4441.objecter _send_command 1 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 1 -- 10.20.193.16:0/2452020534 --> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e880841e0 con 0x7f7e880816d0 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 20 client.4441.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 1 -- 10.20.193.16:0/2452020534 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7eb40841e0 con 0x7f7eb81561e0 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7f7ea5ffb640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7fdb37fff640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdb38074dc0 0x7fdb3814dec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400c700 tx=0x7fdb2400cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7fdb34ff9640 10 client.?.objecter ms_handle_connect 0x7fdb38074dc0 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.899+0000 7fdb34ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 10 client.4377.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 15 client.4377.objecter _session_command_op_assign -1 2 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 20 client.4377.objecter _get_session s=0x7f2070081060 osd=0 3 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 20 client.4377.objecter put_session s=0x7f2070081060 osd=0 3 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 20 client.4377.objecter _get_session s=0x7f2070081060 osd=0 3 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 15 client.4377.objecter _session_command_op_remove -1 2 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 20 client.4377.objecter get_session s=0x7f2070081060 osd=0 3 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 15 client.4377.objecter _session_command_op_assign 0 2 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 20 client.4377.objecter put_session s=0x7f2070081060 osd=0 4 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 10 client.4377.objecter _send_command 2 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 --> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20a40cd0a0 con 0x7f2070081730 2026-02-17T05:56:46.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f7ebcbf0640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] conn(0x7f7e880816d0 0x7f7e88083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f7ebcbf0640 10 monclient: get_auth_request con 0x7f7e880816d0 auth_method 0 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7fdd4e005640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] conn(0x7fdd1c081680 0x7fdd1c083ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7fdd367fc640 10 client.4398.objecter ms_handle_connect 0x7fdd1c081680 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f18f17fa640 1 -- 10.20.193.16:0/2230245128 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec012070 con 0x7f18fc074dc0 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f18f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f18f17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f18f17fa640 10 monclient: dump: 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f18f17fa640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f18f17fa640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f18f17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f18f17fa640 1 -- 10.20.193.16:0/2230245128 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ec004480 con 0x7f18fc074dc0 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f18f17fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f18f2ffd640 4 set_mon_vals no callback set 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f960d7fa640 1 -- 10.20.193.16:0/3204312853 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614013020 con 0x7f96181561e0 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.900+0000 7f960d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 10 monclient: dump: 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.974 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f18f17fa640 1 -- 10.20.193.16:0/2230245128 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec0029c0 con 0x7f18fc074dc0 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f18f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f18f17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 1 -- 10.20.193.16:0/3204312853 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96140044a0 con 0x7f96181561e0 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 5 monclient: authenticate success, global_id 4490 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 10 monclient: _renew_subs 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96180d0360 con 0x7f96181561e0 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 1 -- 10.20.193.16:0/3204312853 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614002920 con 0x7f96181561e0 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960effd640 4 set_mon_vals no callback set 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f18f17fa640 10 monclient: dump: 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 5 monclient: authenticate success, global_id 4456 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f960d7fa640 10 monclient: dump: 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 10 client.4490.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 10 monclient: _renew_subs 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96181eff60 con 0x7f96181561e0 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 10 monclient: _renew_subs 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18fc1f8ce0 con 0x7f18fc074dc0 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 10 client.4456.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 10 monclient: _renew_subs 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 1 librados: init done 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18fc1fba20 con 0x7f18fc074dc0 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 1 librados: init done 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 10 client.4456.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 15 client.4456.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 20 client.4456.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 10 client.4456.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18fc1f8fc0 con 0x7f18fc074dc0 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 10 client.4490.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 15 client.4490.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 20 client.4490.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 10 client.4490.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96180c8f90 con 0x7f96181561e0 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stdout:115964117001 2026-02-17T05:56:46.978 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 10 client.4444.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 15 client.4444.objecter _session_command_op_assign -1 2 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 20 client.4444.objecter _get_session s=0x7f0dd0081000 osd=11 3 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 20 client.4444.objecter put_session s=0x7f0dd0081000 osd=11 3 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 20 client.4444.objecter _get_session s=0x7f0dd0081000 osd=11 3 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 15 client.4444.objecter _session_command_op_remove -1 2 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 20 client.4444.objecter get_session s=0x7f0dd0081000 osd=11 3 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 15 client.4444.objecter _session_command_op_assign 11 2 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 20 client.4444.objecter put_session s=0x7f0dd0081000 osd=11 4 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 10 client.4444.objecter _send_command 2 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.901+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 --> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0e000cc1a0 con 0x7f0dd00816d0 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7f8029ffb640 1 -- 10.20.193.16:0/1238250860 <== osd.4 v2:10.20.193.16:6818/1157611076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f8038004ec0 con 0x7f800c0816d0 2026-02-17T05:56:46.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7f8029ffb640 10 client.4469.objecter ms_dispatch2 0x7f803c000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7f8029ffb640 10 client.4469.objecter _finish_command 1 = system:0 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7f8029ffb640 20 client.4469.objecter put_session s=0x7f800c081000 osd=4 4 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7f8029ffb640 15 client.4469.objecter _session_command_op_remove 4 1 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7f7ea5ffb640 10 client.4441.objecter ms_handle_connect 0x7f7e880816d0 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7f7ebcbf0640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] conn(0x7f7e880816d0 0x7f7e88083b10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 1 -- 10.20.193.16:0/2689701979 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0004500 con 0x7fd9f0074dc0 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 10 monclient: dump: 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 1 -- 10.20.193.16:0/2689701979 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e0015070 con 0x7fd9f0074dc0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 1 -- 10.20.193.16:0/2689701979 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e00029c0 con 0x7fd9f0074dc0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9ddffb640 10 monclient: dump: 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 5 monclient: authenticate success, global_id 4453 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 10 monclient: _renew_subs 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.902+0000 7fd9df7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f00c7b60 con 0x7fd9f0074dc0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 10 client.4453.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 10 monclient: _renew_subs 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f00c8120 con 0x7fd9f0074dc0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 1 librados: init done 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 10 client.4453.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 15 client.4453.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 20 client.4453.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 10 client.4453.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.903+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd9f00cbb60 con 0x7fd9f0074dc0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.904+0000 7f0df57fa640 1 -- 10.20.193.16:0/401859147 <== osd.11 v2:10.20.193.164:6808/188912883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0e000cc1a0 con 0x7f0dd00816d0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.904+0000 7f0df57fa640 10 client.4444.objecter ms_dispatch2 0x7f0e00000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.904+0000 7f0df57fa640 10 client.4444.objecter _finish_command 2 = system:0 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.904+0000 7f0df57fa640 20 client.4444.objecter put_session s=0x7f0dd0081000 osd=11 4 2026-02-17T05:56:46.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.904+0000 7f0df57fa640 15 client.4444.objecter _session_command_op_remove 11 2 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 10 librados: watch_flush enter 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 10 librados: watch_flush exit 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 10 client.4444.objecter close_session for osd.11 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 >> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] conn(0x7f0dd00816d0 msgr2=0x7f0dd0083b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] conn(0x7f0dd00816d0 0x7f0dd0083b10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 20 client.4444.objecter put_session s=0x7f0dd0081000 osd=11 1 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 20 client.4444.objecter shutdown clearing up homeless session... 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 10 client.4444.objecter successfully canceled tick 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0dd0079190 msgr2=0x7f0dd007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0dd0079190 0x7f0dd007b650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0de8006fd0 tx=0x7f0de8008040 comp rx=0 tx=0).stop 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 10 monclient: shutdown 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e000745e0 msgr2=0x7f0e001f3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 --2- 10.20.193.16:0/401859147 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e000745e0 0x7f0e001f3d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009a70 tx=0x7f0df00047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e0512c640 1 -- 10.20.193.16:0/401859147 reap_dead start 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 shutdown_connections 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 >> 10.20.193.16:0/401859147 conn(0x7f0e000ce6c0 msgr2=0x7f0e0006d470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 shutdown_connections 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 -- 10.20.193.16:0/401859147 wait complete. 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f0e06bb6640 1 librados: shutdown 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f208affd640 1 -- 10.20.193.16:0/578125960 <== osd.0 v2:10.20.193.16:6826/821011033 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20a40cd0a0 con 0x7f2070081730 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f208affd640 10 client.4377.objecter ms_dispatch2 0x7f20a4000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f208affd640 10 client.4377.objecter _finish_command 2 = system:0 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f208affd640 20 client.4377.objecter put_session s=0x7f2070081060 osd=0 4 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f208affd640 15 client.4377.objecter _session_command_op_remove 0 2 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 10 librados: watch_flush enter 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 10 librados: watch_flush exit 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 10 client.4377.objecter close_session for osd.0 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] conn(0x7f2070081730 msgr2=0x7f2070083b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] conn(0x7f2070081730 0x7f2070083b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 20 client.4377.objecter put_session s=0x7f2070081060 osd=0 1 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 20 client.4377.objecter shutdown clearing up homeless session... 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 10 client.4377.objecter successfully canceled tick 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2070079190 msgr2=0x7f207007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2070079190 0x7f207007b650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f208c039000 tx=0x7f208c033000 comp rx=0 tx=0).stop 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 10 monclient: shutdown 2026-02-17T05:56:46.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20a40c7d00 msgr2=0x7f20a4143010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 1 --2- 10.20.193.16:0/578125960 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f20a40c7d00 0x7f20a4143010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f209800c760 tx=0x7f209800cc30 comp rx=0 tx=0).stop 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a2d76640 1 -- 10.20.193.16:0/578125960 reap_dead start 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 shutdown_connections 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 >> 10.20.193.16:0/578125960 conn(0x7f20a40cd770 msgr2=0x7f20a4159420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.905+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 shutdown_connections 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.906+0000 7f20a8d85640 1 -- 10.20.193.16:0/578125960 wait complete. 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.906+0000 7f20a8d85640 1 librados: shutdown 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 10 client.4469.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 15 client.4469.objecter _session_command_op_assign -1 2 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 20 client.4469.objecter _get_session s=0x7f800c081000 osd=4 3 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 20 client.4469.objecter put_session s=0x7f800c081000 osd=4 3 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 20 client.4469.objecter _get_session s=0x7f800c081000 osd=4 3 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 15 client.4469.objecter _session_command_op_remove -1 2 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 20 client.4469.objecter get_session s=0x7f800c081000 osd=4 3 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 15 client.4469.objecter _session_command_op_assign 4 2 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 20 client.4469.objecter put_session s=0x7f800c081000 osd=4 4 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 10 client.4469.objecter _send_command 2 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.909+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 --> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f803c14d050 con 0x7f800c0816d0 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.910+0000 7f3b93fff640 1 -- 10.20.193.16:0/2823223881 <== osd.2 v2:10.20.193.148:6816/1588035471 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f3b880841e0 con 0x7f3b880816d0 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.910+0000 7f3b93fff640 10 client.4404.objecter ms_dispatch2 0x7f3bb8000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.910+0000 7f3b93fff640 10 client.4404.objecter _finish_command 1 = system:0 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.910+0000 7f3b93fff640 20 client.4404.objecter put_session s=0x7f3b88081000 osd=2 4 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.910+0000 7f3b93fff640 15 client.4404.objecter _session_command_op_remove 2 1 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.911+0000 7f6e7cff9640 1 -- 10.20.193.16:0/1534513711 <== osd.1 v2:10.20.193.128:6807/1618049789 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f6e7400ee50 con 0x7f6e500816d0 2026-02-17T05:56:46.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.911+0000 7f6e7cff9640 10 client.4481.objecter ms_dispatch2 0x7f6e80000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.911+0000 7f6e7cff9640 10 client.4481.objecter _finish_command 1 = system:0 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.911+0000 7f6e7cff9640 20 client.4481.objecter put_session s=0x7f6e50081000 osd=1 4 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.911+0000 7f6e7cff9640 15 client.4481.objecter _session_command_op_remove 1 1 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.912+0000 7fa1e0ff9640 1 -- 10.20.193.16:0/4262848294 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fa1d0002b60 con 0x7fa1e4074dc0 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.912+0000 7fa1e0ff9640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa1b4079140 0x7fa1b407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.912+0000 7fa1e37fe640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa1b4079140 0x7fa1b407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.912+0000 7fa1e37fe640 10 monclient: get_auth_request con 0x7fa1b4079140 auth_method 0 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7fa1e0ff9640 10 client.4484.objecter ms_handle_connect 0x7fa1b4079140 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7fa1e37fe640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa1b4079140 0x7fa1b407b600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa1e406ce20 tx=0x7fa1d4031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f8029ffb640 1 -- 10.20.193.16:0/1238250860 <== osd.4 v2:10.20.193.16:6818/1157611076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f803c14d050 con 0x7f800c0816d0 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f8029ffb640 10 client.4469.objecter ms_dispatch2 0x7f803c000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f8029ffb640 10 client.4469.objecter _finish_command 2 = system:0 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f8029ffb640 20 client.4469.objecter put_session s=0x7f800c081000 osd=4 4 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f8029ffb640 15 client.4469.objecter _session_command_op_remove 4 2 2026-02-17T05:56:46.984 INFO:teuthology.orchestra.run.trial016.stdout:98784247813 2026-02-17T05:56:46.985 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-17T05:56:46.985 INFO:teuthology.orchestra.run.trial016.stdout:120259084295 2026-02-17T05:56:46.985 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 10 librados: watch_flush enter 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 10 librados: watch_flush exit 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 10 client.4469.objecter close_session for osd.4 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x7f800c0816d0 msgr2=0x7f800c083b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x7f800c0816d0 0x7f800c083b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 20 client.4469.objecter put_session s=0x7f800c081000 osd=4 1 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 20 client.4469.objecter shutdown clearing up homeless session... 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 10 client.4469.objecter successfully canceled tick 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 msgr2=0x7f800c07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 0x7f800c07b650 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f803c1603b0 tx=0x7f802c031000 comp rx=0 tx=0).stop 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 10 monclient: shutdown 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c15d360 msgr2=0x7f803c1f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 1 --2- 10.20.193.16:0/1238250860 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c15d360 0x7f803c1f6f90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f803800cc10 tx=0x7f803800a430 comp rx=0 tx=0).stop 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f80418a2640 1 -- 10.20.193.16:0/1238250860 reap_dead start 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 shutdown_connections 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 >> 10.20.193.16:0/1238250860 conn(0x7f803c0ce6c0 msgr2=0x7f803c15d760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.913+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 shutdown_connections 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.914+0000 7f804332c640 1 -- 10.20.193.16:0/1238250860 wait complete. 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.914+0000 7f804332c640 1 librados: shutdown 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.916+0000 7f18f17fa640 1 -- 10.20.193.16:0/2230245128 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f18ec00d460 con 0x7f18fc074dc0 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.916+0000 7f18f17fa640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f18cc079190 0x7f18cc07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.916+0000 7f18f3fff640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f18cc079190 0x7f18cc07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.916+0000 7f18f3fff640 10 monclient: get_auth_request con 0x7f18cc079190 auth_method 0 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 10 client.4456.objecter ms_handle_connect 0x7f18cc079190 2026-02-17T05:56:46.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f3fff640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f18cc079190 0x7f18cc07b650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f18fc06ce20 tx=0x7f18e4031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 10 client.4404.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 15 client.4404.objecter _session_command_op_assign -1 2 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 20 client.4404.objecter _get_session s=0x7f3b88081000 osd=2 3 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 20 client.4404.objecter put_session s=0x7f3b88081000 osd=2 3 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 20 client.4404.objecter _get_session s=0x7f3b88081000 osd=2 3 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 15 client.4404.objecter _session_command_op_remove -1 2 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 20 client.4404.objecter get_session s=0x7f3b88081000 osd=2 3 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 15 client.4404.objecter _session_command_op_assign 2 2 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 20 client.4404.objecter put_session s=0x7f3b88081000 osd=2 4 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 10 client.4404.objecter _send_command 2 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 --> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3bb8152160 con 0x7f3b880816d0 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 1 -- 10.20.193.16:0/2230245128 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f18ec05dcf0 con 0x7f18fc074dc0 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 10 client.4456.objecter ms_dispatch2 0x7f18fc000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 3 client.4456.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 3 client.4456.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 10 client.4456.objecter checking command 1 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] conn(0x7f18cc081730 0x7f18cc083b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 20 client.4456.objecter _get_session s=0x7f18cc081060 osd=12 3 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 20 client.4456.objecter put_session s=0x7f18cc081060 osd=12 3 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 15 client.4456.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 20 client.4456.objecter _get_session s=0x7f18cc081060 osd=12 3 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 20 client.4456.objecter get_session s=0x7f18cc081060 osd=12 3 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 15 client.4456.objecter _session_command_op_assign 12 1 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 20 client.4456.objecter put_session s=0x7f18cc081060 osd=12 4 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 10 client.4456.objecter _send_command 1 2026-02-17T05:56:46.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 1 -- 10.20.193.16:0/2230245128 --> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18cc084240 con 0x7f18cc081730 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.917+0000 7f18f17fa640 20 client.4456.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f18f17fa640 1 -- 10.20.193.16:0/2230245128 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f18ec05e0b0 con 0x7f18fc074dc0 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f18f17fa640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 1 -- 10.20.193.16:0/4262848294 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fa1d0097ce0 con 0x7fa1e4074dc0 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 10 client.4484.objecter ms_dispatch2 0x7fa1e4000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 3 client.4484.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 3 client.4484.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 10 client.4484.objecter checking command 1 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] conn(0x7fa1b40816e0 0x7fa1b4083b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 20 client.4484.objecter _get_session s=0x7fa1b4081010 osd=14 3 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 20 client.4484.objecter put_session s=0x7fa1b4081010 osd=14 3 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 15 client.4484.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 20 client.4484.objecter _get_session s=0x7fa1b4081010 osd=14 3 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f19010eb640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] conn(0x7f18cc081730 0x7f18cc083b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 20 client.4484.objecter get_session s=0x7fa1b4081010 osd=14 3 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 15 client.4484.objecter _session_command_op_assign 14 1 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 20 client.4484.objecter put_session s=0x7fa1b4081010 osd=14 4 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 10 client.4484.objecter _send_command 1 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 1 -- 10.20.193.16:0/4262848294 --> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1d001eec0 con 0x7fa1b40816e0 2026-02-17T05:56:46.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7fa1e0ff9640 20 client.4484.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 10 client.4481.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 15 client.4481.objecter _session_command_op_assign -1 2 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 20 client.4481.objecter _get_session s=0x7f6e50081000 osd=1 3 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 20 client.4481.objecter put_session s=0x7f6e50081000 osd=1 3 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 20 client.4481.objecter _get_session s=0x7f6e50081000 osd=1 3 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 15 client.4481.objecter _session_command_op_remove -1 2 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 20 client.4481.objecter get_session s=0x7f6e50081000 osd=1 3 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 15 client.4481.objecter _session_command_op_assign 1 2 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 20 client.4481.objecter put_session s=0x7f6e50081000 osd=1 4 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 10 client.4481.objecter _send_command 2 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 --> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6e800ca700 con 0x7f6e500816d0 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.918+0000 7f19010eb640 10 monclient: get_auth_request con 0x7f18cc081730 auth_method 0 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.919+0000 7fa1e8e68640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] conn(0x7fa1b40816e0 0x7fa1b4083b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.919+0000 7fa1e8e68640 10 monclient: get_auth_request con 0x7fa1b40816e0 auth_method 0 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.920+0000 7f19010eb640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] conn(0x7f18cc081730 0x7f18cc083b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.12 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.920+0000 7f18f17fa640 10 client.4456.objecter ms_handle_connect 0x7f18cc081730 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3b93fff640 1 -- 10.20.193.16:0/2823223881 <== osd.2 v2:10.20.193.148:6816/1588035471 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3bb8152160 con 0x7f3b880816d0 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3b93fff640 10 client.4404.objecter ms_dispatch2 0x7f3bb8000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3b93fff640 10 client.4404.objecter _finish_command 2 = system:0 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3b93fff640 20 client.4404.objecter put_session s=0x7f3b88081000 osd=2 4 2026-02-17T05:56:46.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3b93fff640 15 client.4404.objecter _session_command_op_remove 2 2 2026-02-17T05:56:46.990 INFO:teuthology.orchestra.run.trial016.stdout:107374182405 2026-02-17T05:56:46.990 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-17T05:56:46.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 10 librados: watch_flush enter 2026-02-17T05:56:46.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 10 librados: watch_flush exit 2026-02-17T05:56:46.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 10 client.4404.objecter close_session for osd.2 2026-02-17T05:56:46.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 >> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] conn(0x7f3b880816d0 msgr2=0x7f3b88083b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] conn(0x7f3b880816d0 0x7f3b88083b10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 20 client.4404.objecter put_session s=0x7f3b88081000 osd=2 1 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 20 client.4404.objecter shutdown clearing up homeless session... 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 10 client.4404.objecter successfully canceled tick 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3b88079190 msgr2=0x7f3b8807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3b88079190 0x7f3b8807b650 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8006fd0 tx=0x7f3ba8006d40 comp rx=0 tx=0).stop 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 10 monclient: shutdown 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3bb8073170 msgr2=0x7f3bb81f0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 --2- 10.20.193.16:0/2823223881 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3bb8073170 0x7f3bb81f0030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b500 tx=0x7f3ba400b9d0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bb7fff640 1 -- 10.20.193.16:0/2823223881 reap_dead start 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 shutdown_connections 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 >> 10.20.193.16:0/2823223881 conn(0x7f3bb80ca5a0 msgr2=0x7f3bb8151a40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 shutdown_connections 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 -- 10.20.193.16:0/2823223881 wait complete. 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7f3bbe10a640 1 librados: shutdown 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7fa1e8e68640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] conn(0x7fa1b40816e0 0x7fa1b4083b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.14 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.921+0000 7fa1e0ff9640 10 client.4484.objecter ms_handle_connect 0x7fa1b40816e0 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e7cff9640 1 -- 10.20.193.16:0/1534513711 <== osd.1 v2:10.20.193.128:6807/1618049789 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6e800ca700 con 0x7f6e500816d0 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e7cff9640 10 client.4481.objecter ms_dispatch2 0x7f6e80000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e7cff9640 10 client.4481.objecter _finish_command 2 = system:0 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e7cff9640 20 client.4481.objecter put_session s=0x7f6e50081000 osd=1 4 2026-02-17T05:56:46.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e7cff9640 15 client.4481.objecter _session_command_op_remove 1 2 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 10 librados: watch_flush enter 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 10 librados: watch_flush exit 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 10 client.4481.objecter close_session for osd.1 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 >> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] conn(0x7f6e500816d0 msgr2=0x7f6e50083b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] conn(0x7f6e500816d0 0x7f6e50083b10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 20 client.4481.objecter put_session s=0x7f6e50081000 osd=1 1 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 20 client.4481.objecter shutdown clearing up homeless session... 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 10 client.4481.objecter successfully canceled tick 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e50079190 msgr2=0x7f6e5007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e50079190 0x7f6e5007b650 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c007fd0 tx=0x7f6e6c007b20 comp rx=0 tx=0).stop 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 10 monclient: shutdown 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e801561e0 msgr2=0x7f6e8013f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 1 --2- 10.20.193.16:0/1534513711 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e801561e0 0x7f6e8013f2b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400b9c0 tx=0x7f6e7400be90 comp rx=0 tx=0).stop 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e84ed1640 1 -- 10.20.193.16:0/1534513711 reap_dead start 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 shutdown_connections 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 >> 10.20.193.16:0/1534513711 conn(0x7f6e800ce6c0 msgr2=0x7f6e80153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.923+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 shutdown_connections 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.924+0000 7f6e8695b640 1 -- 10.20.193.16:0/1534513711 wait complete. 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.924+0000 7f6e8695b640 1 librados: shutdown 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.924+0000 7fdd367fc640 1 -- 10.20.193.16:0/1659368440 <== osd.8 v2:10.20.193.16:6802/15364298 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fdd3801be50 con 0x7fdd1c081680 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.924+0000 7fdd367fc640 10 client.4398.objecter ms_dispatch2 0x7fdd48000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.924+0000 7fdd367fc640 10 client.4398.objecter _finish_command 1 = system:0 2026-02-17T05:56:46.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.924+0000 7fdd367fc640 20 client.4398.objecter put_session s=0x7fdd1c080fb0 osd=8 4 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.924+0000 7fdd367fc640 15 client.4398.objecter _session_command_op_remove 8 1 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.926+0000 7f7ea5ffb640 1 -- 10.20.193.16:0/2452020534 <== osd.10 v2:10.20.193.148:6808/2531105113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f7e880841e0 con 0x7f7e880816d0 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.926+0000 7f7ea5ffb640 10 client.4441.objecter ms_dispatch2 0x7f7eb8000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.926+0000 7f7ea5ffb640 10 client.4441.objecter _finish_command 1 = system:0 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.926+0000 7f7ea5ffb640 20 client.4441.objecter put_session s=0x7f7e88081000 osd=10 4 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.926+0000 7f7ea5ffb640 15 client.4441.objecter _session_command_op_remove 10 1 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.927+0000 7fa1e0ff9640 1 -- 10.20.193.16:0/4262848294 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa1d005e660 con 0x7fa1e4074dc0 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.927+0000 7fa1e0ff9640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.928+0000 7f960d7fa640 1 -- 10.20.193.16:0/3204312853 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f9614002ac0 con 0x7f96181561e0 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.928+0000 7f960d7fa640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f95e8079140 0x7f95e807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.928+0000 7f961c915640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f95e8079140 0x7f95e807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.928+0000 7f961c915640 10 monclient: get_auth_request con 0x7f95e8079140 auth_method 0 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.929+0000 7f961c915640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f95e8079140 0x7f95e807b600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9608038000 tx=0x7f9608031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.929+0000 7f960d7fa640 10 client.4490.objecter ms_handle_connect 0x7f95e8079140 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.929+0000 7f960d7fa640 1 -- 10.20.193.16:0/3204312853 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f9614097e20 con 0x7f96181561e0 2026-02-17T05:56:46.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.929+0000 7f960d7fa640 10 client.4490.objecter ms_dispatch2 0x7f9618000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.929+0000 7f960d7fa640 3 client.4490.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.929+0000 7f960d7fa640 3 client.4490.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.929+0000 7f960d7fa640 10 client.4490.objecter checking command 1 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] conn(0x7f95e8081680 0x7f95e8083ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 20 client.4490.objecter _get_session s=0x7f95e8080fb0 osd=7 3 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 20 client.4490.objecter put_session s=0x7f95e8080fb0 osd=7 3 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 15 client.4490.objecter _session_command_op_remove -1 1 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 20 client.4490.objecter _get_session s=0x7f95e8080fb0 osd=7 3 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 20 client.4490.objecter get_session s=0x7f95e8080fb0 osd=7 3 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 15 client.4490.objecter _session_command_op_assign 7 1 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 20 client.4490.objecter put_session s=0x7f95e8080fb0 osd=7 4 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 10 client.4490.objecter _send_command 1 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 1 -- 10.20.193.16:0/3204312853 --> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f961401ce70 con 0x7f95e8081680 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 20 client.4490.objecter dump_active .. 0 homeless 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stdout:90194313223 2026-02-17T05:56:46.994 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 1 -- 10.20.193.16:0/3204312853 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f961405e690 con 0x7f96181561e0 2026-02-17T05:56:46.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960d7fa640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 1 -- 10.20.193.16:0/1961814941 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4046020 con 0x7f43c815fa10 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 10 monclient: dump: 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 1 -- 10.20.193.16:0/1961814941 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c4041450 con 0x7f43c815fa10 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43cf2e8640 5 monclient: authenticate success, global_id 4493 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43cf2e8640 10 monclient: _renew_subs 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43cf2e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c81506c0 con 0x7f43c815fa10 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 1 -- 10.20.193.16:0/1961814941 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4037e60 con 0x7f43c815fa10 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b5ffb640 10 monclient: dump: 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.995 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43b77fe640 4 set_mon_vals no callback set 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960ffff640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] conn(0x7f95e8081680 0x7f95e8083ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43cf2e8640 10 client.4493.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43cf2e8640 10 monclient: _renew_subs 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43cf2e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c8150c80 con 0x7f43c815fa10 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f43cf2e8640 1 librados: init done 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.930+0000 7f960ffff640 10 monclient: get_auth_request con 0x7f95e8081680 auth_method 0 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.931+0000 7f43cf2e8640 10 client.4493.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.931+0000 7f43cf2e8640 15 client.4493.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.931+0000 7f43cf2e8640 20 client.4493.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.931+0000 7f43cf2e8640 10 client.4493.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.931+0000 7f43cf2e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.931+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43c8147af0 con 0x7f43c815fa10 2026-02-17T05:56:46.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 10 client.4398.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 15 client.4398.objecter _session_command_op_assign -1 2 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 20 client.4398.objecter _get_session s=0x7fdd1c080fb0 osd=8 3 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 20 client.4398.objecter put_session s=0x7fdd1c080fb0 osd=8 3 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 20 client.4398.objecter _get_session s=0x7fdd1c080fb0 osd=8 3 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 15 client.4398.objecter _session_command_op_remove -1 2 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 20 client.4398.objecter get_session s=0x7fdd1c080fb0 osd=8 3 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 15 client.4398.objecter _session_command_op_assign 8 2 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 20 client.4398.objecter put_session s=0x7fdd1c080fb0 osd=8 4 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 10 client.4398.objecter _send_command 2 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.932+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 --> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdd48153e40 con 0x7fdd1c081680 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stdout:98784247813 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7f960ffff640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] conn(0x7f95e8081680 0x7f95e8083ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:46.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7f960d7fa640 10 client.4490.objecter ms_handle_connect 0x7f95e8081680 2026-02-17T05:56:46.997 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 1 -- 10.20.193.16:0/130213896 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb240052d0 con 0x7fdb38074dc0 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 10 monclient: dump: 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 1 -- 10.20.193.16:0/130213896 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb24005490 con 0x7fdb38074dc0 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 1 -- 10.20.193.16:0/130213896 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb240038d0 con 0x7fdb38074dc0 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 5 monclient: authenticate success, global_id 4459 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 10 monclient: _renew_subs 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb381f39a0 con 0x7fdb38074dc0 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb367fc640 4 set_mon_vals no callback set 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb34ff9640 10 monclient: dump: 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:46.998 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 10 client.4459.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 10 monclient: _renew_subs 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb381f3f60 con 0x7fdb38074dc0 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 1 librados: init done 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 10 client.4459.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 15 client.4459.objecter _session_command_op_assign -1 1 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 20 client.4459.objecter _get_session osd=-1 returning homeless 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 10 client.4459.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb380ca700 con 0x7fdb38074dc0 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.933+0000 7fd9ddffb640 1 -- 10.20.193.16:0/2689701979 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fd9e0010480 con 0x7fd9f0074dc0 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 10 client.4441.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 15 client.4441.objecter _session_command_op_assign -1 2 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 20 client.4441.objecter _get_session s=0x7f7e88081000 osd=10 3 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 20 client.4441.objecter put_session s=0x7f7e88081000 osd=10 3 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 20 client.4441.objecter _get_session s=0x7f7e88081000 osd=10 3 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 15 client.4441.objecter _session_command_op_remove -1 2 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 20 client.4441.objecter get_session s=0x7f7e88081000 osd=10 3 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 15 client.4441.objecter _session_command_op_assign 10 2 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 20 client.4441.objecter put_session s=0x7f7e88081000 osd=10 4 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 10 client.4441.objecter _send_command 2 2026-02-17T05:56:46.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 --> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7eb80c8f90 con 0x7f7e880816d0 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7fd9ddffb640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd9c00791c0 0x7fd9c007b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7fd9f4d2e640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd9c00791c0 0x7fd9c007b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7fd9f4d2e640 10 monclient: get_auth_request con 0x7fd9c00791c0 auth_method 0 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7fd9f4d2e640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd9c00791c0 0x7fd9c007b680 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd9e402ee50 tx=0x7fd9e4033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.934+0000 7fd9ddffb640 10 client.4453.objecter ms_handle_connect 0x7fd9c00791c0 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 1 -- 10.20.193.16:0/2689701979 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fd9e0097e90 con 0x7fd9f0074dc0 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 10 client.4453.objecter ms_dispatch2 0x7fd9f0000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 3 client.4453.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 3 client.4453.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 10 client.4453.objecter checking command 1 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] conn(0x7fd9c0081760 0x7fd9c0083ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 20 client.4453.objecter _get_session s=0x7fd9c0081090 osd=9 3 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 20 client.4453.objecter put_session s=0x7fd9c0081090 osd=9 3 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 15 client.4453.objecter _session_command_op_remove -1 1 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 20 client.4453.objecter _get_session s=0x7fd9c0081090 osd=9 3 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 20 client.4453.objecter get_session s=0x7fd9c0081090 osd=9 3 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 15 client.4453.objecter _session_command_op_assign 9 1 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 20 client.4453.objecter put_session s=0x7fd9c0081090 osd=9 4 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 10 client.4453.objecter _send_command 1 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 1 -- 10.20.193.16:0/2689701979 --> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9e0016ec0 con 0x7fd9c0081760 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9ddffb640 20 client.4453.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.935+0000 7fd9f5d30640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] conn(0x7fd9c0081760 0x7fd9c0083ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fd9f5d30640 10 monclient: get_auth_request con 0x7fd9c0081760 auth_method 0 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fd9ddffb640 1 -- 10.20.193.16:0/2689701979 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd9e005e810 con 0x7fd9f0074dc0 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fd9ddffb640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd367fc640 1 -- 10.20.193.16:0/1659368440 <== osd.8 v2:10.20.193.16:6802/15364298 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdd48153e40 con 0x7fdd1c081680 2026-02-17T05:56:47.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd367fc640 10 client.4398.objecter ms_dispatch2 0x7fdd48000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd367fc640 10 client.4398.objecter _finish_command 2 = system:0 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd367fc640 20 client.4398.objecter put_session s=0x7fdd1c080fb0 osd=8 4 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd367fc640 15 client.4398.objecter _session_command_op_remove 8 2 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 10 librados: watch_flush enter 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 10 librados: watch_flush exit 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 10 client.4398.objecter close_session for osd.8 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] conn(0x7fdd1c081680 msgr2=0x7fdd1c083ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] conn(0x7fdd1c081680 0x7fdd1c083ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 20 client.4398.objecter put_session s=0x7fdd1c080fb0 osd=8 1 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 20 client.4398.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 10 client.4398.objecter successfully canceled tick 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd1c079140 msgr2=0x7fdd1c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd1c079140 0x7fdd1c07b600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdd480cb290 tx=0x7fdd3c006cf0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 10 monclient: shutdown 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd480745e0 msgr2=0x7fdd481f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 --2- 10.20.193.16:0/1659368440 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd480745e0 0x7fdd481f48b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800b550 tx=0x7fdd3800ba20 comp rx=0 tx=0).stop 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4e005640 1 -- 10.20.193.16:0/1659368440 reap_dead start 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 shutdown_connections 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 >> 10.20.193.16:0/1659368440 conn(0x7fdd480ce6c0 msgr2=0x7fdd48154920 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 shutdown_connections 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 -- 10.20.193.16:0/1659368440 wait complete. 2026-02-17T05:56:47.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.936+0000 7fdd4fa8f640 1 librados: shutdown 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.937+0000 7f3435ffb640 1 -- 10.20.193.16:0/1220374686 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f344401b3b0 con 0x7f34481561e0 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.937+0000 7f3435ffb640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3418079140 0x7f341807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.937+0000 7f7ea5ffb640 1 -- 10.20.193.16:0/2452020534 <== osd.10 v2:10.20.193.148:6808/2531105113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7eb80c8f90 con 0x7f7e880816d0 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.937+0000 7f7ea5ffb640 10 client.4441.objecter ms_dispatch2 0x7f7eb8000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.937+0000 7f7ea5ffb640 10 client.4441.objecter _finish_command 2 = system:0 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.937+0000 7f7ea5ffb640 20 client.4441.objecter put_session s=0x7f7e88081000 osd=10 4 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.937+0000 7f7ea5ffb640 15 client.4441.objecter _session_command_op_remove 10 2 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.937+0000 7f344d4fc640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3418079140 0x7f341807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.937+0000 7f344d4fc640 10 monclient: get_auth_request con 0x7f3418079140 auth_method 0 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 10 librados: watch_flush enter 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 10 librados: watch_flush exit 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 10 client.4441.objecter close_session for osd.10 2026-02-17T05:56:47.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 >> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] conn(0x7f7e880816d0 msgr2=0x7f7e88083b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] conn(0x7f7e880816d0 0x7f7e88083b10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 20 client.4441.objecter put_session s=0x7f7e88081000 osd=10 1 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 20 client.4441.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 10 client.4441.objecter successfully canceled tick 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7e88079190 msgr2=0x7f7e8807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7e88079190 0x7f7e8807b650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80098a0 tx=0x7f7ea8006d90 comp rx=0 tx=0).stop 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 10 monclient: shutdown 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7eb81561e0 msgr2=0x7f7eb815b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 --2- 10.20.193.16:0/2452020534 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7eb81561e0 0x7f7eb815b9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009f50 tx=0x7f7eb4004b40 comp rx=0 tx=0).stop 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebdbf2640 1 -- 10.20.193.16:0/2452020534 reap_dead start 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f3435ffb640 10 client.4487.objecter ms_handle_connect 0x7f3418079140 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f344d4fc640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3418079140 0x7f341807b600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3438006b90 tx=0x7f3438006b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 shutdown_connections 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 >> 10.20.193.16:0/2452020534 conn(0x7f7eb80ce6c0 msgr2=0x7f7eb80ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 shutdown_connections 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 -- 10.20.193.16:0/2452020534 wait complete. 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7f7ebf67c640 1 librados: shutdown 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7fd9ddffb640 10 client.4453.objecter ms_handle_connect 0x7fd9c0081760 2026-02-17T05:56:47.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.938+0000 7fd9f5d30640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] conn(0x7fd9c0081760 0x7fd9c0083ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 1 -- 10.20.193.16:0/1220374686 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f3444097fc0 con 0x7f34481561e0 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 10 client.4487.objecter ms_dispatch2 0x7f3448000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 3 client.4487.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 3 client.4487.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 10 client.4487.objecter checking command 1 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] conn(0x7f3418081680 0x7f3418083ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 20 client.4487.objecter _get_session s=0x7f3418080fb0 osd=6 3 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 20 client.4487.objecter put_session s=0x7f3418080fb0 osd=6 3 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 15 client.4487.objecter _session_command_op_remove -1 1 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 20 client.4487.objecter _get_session s=0x7f3418080fb0 osd=6 3 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 20 client.4487.objecter get_session s=0x7f3418080fb0 osd=6 3 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 15 client.4487.objecter _session_command_op_assign 6 1 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 20 client.4487.objecter put_session s=0x7f3418080fb0 osd=6 4 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 10 client.4487.objecter _send_command 1 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 1 -- 10.20.193.16:0/1220374686 --> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3418084190 con 0x7f3418081680 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 20 client.4487.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 1 -- 10.20.193.16:0/1220374686 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f344405e9f0 con 0x7f34481561e0 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f3435ffb640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.939+0000 7f344ccfb640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] conn(0x7f3418081680 0x7f3418083ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.940+0000 7f344ccfb640 10 monclient: get_auth_request con 0x7f3418081680 auth_method 0 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.942+0000 7f3435ffb640 10 client.4487.objecter ms_handle_connect 0x7f3418081680 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.942+0000 7f344ccfb640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] conn(0x7f3418081680 0x7f3418083ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.942+0000 7fa1e0ff9640 1 -- 10.20.193.16:0/4262848294 <== osd.14 v2:10.20.193.148:6800/1855767579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fa1d001eec0 con 0x7fa1b40816e0 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.942+0000 7fa1e0ff9640 10 client.4484.objecter ms_dispatch2 0x7fa1e4000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.943+0000 7fa1e0ff9640 10 client.4484.objecter _finish_command 1 = system:0 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.943+0000 7fa1e0ff9640 20 client.4484.objecter put_session s=0x7fa1b4081010 osd=14 4 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.943+0000 7fa1e0ff9640 15 client.4484.objecter _session_command_op_remove 14 1 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.943+0000 7f18f17fa640 1 -- 10.20.193.16:0/2230245128 <== osd.12 v2:10.20.193.16:6810/3960967243 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f18cc084240 con 0x7f18cc081730 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.943+0000 7f18f17fa640 10 client.4456.objecter ms_dispatch2 0x7f18fc000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:47.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.943+0000 7f18f17fa640 10 client.4456.objecter _finish_command 1 = system:0 2026-02-17T05:56:47.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.943+0000 7f18f17fa640 20 client.4456.objecter put_session s=0x7f18cc081060 osd=12 4 2026-02-17T05:56:47.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.943+0000 7f18f17fa640 15 client.4456.objecter _session_command_op_remove 12 1 2026-02-17T05:56:47.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.943+0000 7ff9237fe640 1 -- 10.20.193.16:0/994528860 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7ff934012990 con 0x7ff9441561e0 2026-02-17T05:56:47.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.944+0000 7ff9237fe640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff910079140 0x7ff91007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.944+0000 7ff942ffd640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff910079140 0x7ff91007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.944+0000 7ff942ffd640 10 monclient: get_auth_request con 0x7ff910079140 auth_method 0 2026-02-17T05:56:47.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.944+0000 7ff9237fe640 10 client.4450.objecter ms_handle_connect 0x7ff910079140 2026-02-17T05:56:47.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.944+0000 7ff942ffd640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff910079140 0x7ff91007b600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff938039000 tx=0x7ff938031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.005 INFO:teuthology.orchestra.run.trial016.stdout:107374182405 2026-02-17T05:56:47.005 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 1 -- 10.20.193.16:0/994528860 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7ff9340985c0 con 0x7ff9441561e0 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 10 client.4450.objecter ms_dispatch2 0x7ff944000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 3 client.4450.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 3 client.4450.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 10 client.4450.objecter checking command 1 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] conn(0x7ff910081680 0x7ff910083ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 20 client.4450.objecter _get_session s=0x7ff910080fb0 osd=15 3 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 20 client.4450.objecter put_session s=0x7ff910080fb0 osd=15 3 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 15 client.4450.objecter _session_command_op_remove -1 1 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 20 client.4450.objecter _get_session s=0x7ff910080fb0 osd=15 3 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 20 client.4450.objecter get_session s=0x7ff910080fb0 osd=15 3 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 15 client.4450.objecter _session_command_op_assign 15 1 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 20 client.4450.objecter put_session s=0x7ff910080fb0 osd=15 4 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 10 client.4450.objecter _send_command 1 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 1 -- 10.20.193.16:0/994528860 --> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff934013e70 con 0x7ff910081680 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.945+0000 7ff9237fe640 20 client.4450.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.006 INFO:teuthology.orchestra.run.trial016.stdout:90194313223 2026-02-17T05:56:47.006 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.948+0000 7ff9237fe640 1 -- 10.20.193.16:0/994528860 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff93405ef40 con 0x7ff9441561e0 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.948+0000 7ff9237fe640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 10 client.4456.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 15 client.4456.objecter _session_command_op_assign -1 2 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 20 client.4456.objecter _get_session s=0x7f18cc081060 osd=12 3 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 20 client.4456.objecter put_session s=0x7f18cc081060 osd=12 3 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 20 client.4456.objecter _get_session s=0x7f18cc081060 osd=12 3 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 15 client.4456.objecter _session_command_op_remove -1 2 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 20 client.4456.objecter get_session s=0x7f18cc081060 osd=12 3 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 15 client.4456.objecter _session_command_op_assign 12 2 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 20 client.4456.objecter put_session s=0x7f18cc081060 osd=12 4 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 10 client.4456.objecter _send_command 2 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.949+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 --> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18fc0c8fe0 con 0x7f18cc081730 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 10 client.4484.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 15 client.4484.objecter _session_command_op_assign -1 2 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 20 client.4484.objecter _get_session s=0x7fa1b4081010 osd=14 3 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 20 client.4484.objecter put_session s=0x7fa1b4081010 osd=14 3 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 20 client.4484.objecter _get_session s=0x7fa1b4081010 osd=14 3 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 15 client.4484.objecter _session_command_op_remove -1 2 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 20 client.4484.objecter get_session s=0x7fa1b4081010 osd=14 3 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 15 client.4484.objecter _session_command_op_assign 14 2 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 20 client.4484.objecter put_session s=0x7fa1b4081010 osd=14 4 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 10 client.4484.objecter _send_command 2 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.951+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 --> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1e414bf20 con 0x7fa1b40816e0 2026-02-17T05:56:47.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f43b5ffb640 1 -- 10.20.193.16:0/1961814941 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f43c4041840 con 0x7f43c815fa10 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f18f17fa640 1 -- 10.20.193.16:0/2230245128 <== osd.12 v2:10.20.193.16:6810/3960967243 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f18fc0c8fe0 con 0x7f18cc081730 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f18f17fa640 10 client.4456.objecter ms_dispatch2 0x7f18fc000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f18f17fa640 10 client.4456.objecter _finish_command 2 = system:0 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f18f17fa640 20 client.4456.objecter put_session s=0x7f18cc081060 osd=12 4 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f18f17fa640 15 client.4456.objecter _session_command_op_remove 12 2 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f43b5ffb640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4398079190 0x7f439807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f43cd05d640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4398079190 0x7f439807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f43cd05d640 10 monclient: get_auth_request con 0x7f4398079190 auth_method 0 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 10 librados: watch_flush enter 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 10 librados: watch_flush exit 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 10 client.4456.objecter close_session for osd.12 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] conn(0x7f18cc081730 msgr2=0x7f18cc083b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] conn(0x7f18cc081730 0x7f18cc083b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 20 client.4456.objecter put_session s=0x7f18cc081060 osd=12 1 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 20 client.4456.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 10 client.4456.objecter successfully canceled tick 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f18cc079190 msgr2=0x7f18cc07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f18cc079190 0x7f18cc07b650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f18fc06ce20 tx=0x7f18e4031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 10 monclient: shutdown 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f18fc074dc0 msgr2=0x7f18fc0c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 --2- 10.20.193.16:0/2230245128 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f18fc074dc0 0x7f18fc0c7fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b5d0 tx=0x7f18ec00baa0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f19010eb640 1 -- 10.20.193.16:0/2230245128 reap_dead start 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 shutdown_connections 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 >> 10.20.193.16:0/2230245128 conn(0x7f18fc0ce6c0 msgr2=0x7f18fc0ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 shutdown_connections 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 -- 10.20.193.16:0/2230245128 wait complete. 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f43b5ffb640 10 client.4493.objecter ms_handle_connect 0x7f4398079190 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f43cd05d640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4398079190 0x7f439807b650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f43b800a8d0 tx=0x7f43b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.952+0000 7f1902b75640 1 librados: shutdown 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.953+0000 7f43b5ffb640 1 -- 10.20.193.16:0/1961814941 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f43c4084120 con 0x7f43c815fa10 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.953+0000 7f43b5ffb640 10 client.4493.objecter ms_dispatch2 0x7f43c8000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.953+0000 7f43b5ffb640 3 client.4493.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.953+0000 7f43b5ffb640 3 client.4493.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 10 client.4493.objecter checking command 1 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] conn(0x7f43980816d0 0x7f4398083b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 20 client.4493.objecter _get_session s=0x7f4398081000 osd=13 3 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 20 client.4493.objecter put_session s=0x7f4398081000 osd=13 3 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 15 client.4493.objecter _session_command_op_remove -1 1 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 20 client.4493.objecter _get_session s=0x7f4398081000 osd=13 3 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 20 client.4493.objecter get_session s=0x7f4398081000 osd=13 3 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 15 client.4493.objecter _session_command_op_assign 13 1 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 20 client.4493.objecter put_session s=0x7f4398081000 osd=13 4 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 10 client.4493.objecter _send_command 1 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 1 -- 10.20.193.16:0/1961814941 --> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f43980841e0 con 0x7f43980816d0 2026-02-17T05:56:47.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 20 client.4493.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 1 -- 10.20.193.16:0/1961814941 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f43c40844e0 con 0x7f43c815fa10 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43b5ffb640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f960d7fa640 1 -- 10.20.193.16:0/3204312853 <== osd.7 v2:10.20.193.164:6816/3942637745 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f961401ce70 con 0x7f95e8081680 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f960d7fa640 10 client.4490.objecter ms_dispatch2 0x7f9618000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f960d7fa640 10 client.4490.objecter _finish_command 1 = system:0 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f960d7fa640 20 client.4490.objecter put_session s=0x7f95e8080fb0 osd=7 4 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f960d7fa640 15 client.4490.objecter _session_command_op_remove 7 1 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43cc85c640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] conn(0x7f43980816d0 0x7f4398083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.954+0000 7f43cc85c640 10 monclient: get_auth_request con 0x7f43980816d0 auth_method 0 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1e0ff9640 1 -- 10.20.193.16:0/4262848294 <== osd.14 v2:10.20.193.148:6800/1855767579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa1e414bf20 con 0x7fa1b40816e0 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1e0ff9640 10 client.4484.objecter ms_dispatch2 0x7fa1e4000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1e0ff9640 10 client.4484.objecter _finish_command 2 = system:0 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1e0ff9640 20 client.4484.objecter put_session s=0x7fa1b4081010 osd=14 4 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1e0ff9640 15 client.4484.objecter _session_command_op_remove 14 2 2026-02-17T05:56:47.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7ff9427fc640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] conn(0x7ff910081680 0x7ff910083ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 10 librados: watch_flush enter 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 10 librados: watch_flush exit 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 10 client.4484.objecter close_session for osd.14 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 >> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] conn(0x7fa1b40816e0 msgr2=0x7fa1b4083b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] conn(0x7fa1b40816e0 0x7fa1b4083b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 20 client.4484.objecter put_session s=0x7fa1b4081010 osd=14 1 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 20 client.4484.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 10 client.4484.objecter successfully canceled tick 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa1b4079140 msgr2=0x7fa1b407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa1b4079140 0x7fa1b407b600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa1e406ce20 tx=0x7fa1d4031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 10 monclient: shutdown 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa1e4074dc0 msgr2=0x7fa1e41f47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 --2- 10.20.193.16:0/4262848294 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa1e4074dc0 0x7fa1e41f47e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000b570 tx=0x7fa1d000ba40 comp rx=0 tx=0).stop 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7ff9427fc640 10 monclient: get_auth_request con 0x7ff910081680 auth_method 0 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1e8e68640 1 -- 10.20.193.16:0/4262848294 reap_dead start 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 shutdown_connections 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 >> 10.20.193.16:0/4262848294 conn(0x7fa1e40ce6c0 msgr2=0x7fa1e40cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 shutdown_connections 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 -- 10.20.193.16:0/4262848294 wait complete. 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.955+0000 7fa1ea8f2640 1 librados: shutdown 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.956+0000 7fdb34ff9640 1 -- 10.20.193.16:0/130213896 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fdb24003a70 con 0x7fdb38074dc0 2026-02-17T05:56:47.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.957+0000 7f43b5ffb640 10 client.4493.objecter ms_handle_connect 0x7f43980816d0 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.957+0000 7f43cc85c640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] conn(0x7f43980816d0 0x7f4398083b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.13 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.957+0000 7fdb34ff9640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdb080791c0 0x7fdb0807b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.957+0000 7fdb377fe640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdb080791c0 0x7fdb0807b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.957+0000 7fdb377fe640 10 monclient: get_auth_request con 0x7fdb080791c0 auth_method 0 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.957+0000 7fdb377fe640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdb080791c0 0x7fdb0807b680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb28039000 tx=0x7fdb28031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.957+0000 7fdb34ff9640 10 client.4459.objecter ms_handle_connect 0x7fdb080791c0 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.957+0000 7ff9237fe640 10 client.4450.objecter ms_handle_connect 0x7ff910081680 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.957+0000 7ff9427fc640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] conn(0x7ff910081680 0x7ff910083ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.15 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 1 -- 10.20.193.16:0/130213896 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fdb2405e260 con 0x7fdb38074dc0 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 10 client.4459.objecter ms_dispatch2 0x7fdb38000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 3 client.4459.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 3 client.4459.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 10 client.4459.objecter checking command 1 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x7fdb08081700 0x7fdb08083b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 20 client.4459.objecter _get_session s=0x7fdb08081030 osd=3 3 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 20 client.4459.objecter put_session s=0x7fdb08081030 osd=3 3 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 15 client.4459.objecter _session_command_op_remove -1 1 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 20 client.4459.objecter _get_session s=0x7fdb08081030 osd=3 3 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 20 client.4459.objecter get_session s=0x7fdb08081030 osd=3 3 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 15 client.4459.objecter _session_command_op_assign 3 1 2026-02-17T05:56:47.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 20 client.4459.objecter put_session s=0x7fdb08081030 osd=3 4 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 10 client.4459.objecter _send_command 1 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 1 -- 10.20.193.16:0/130213896 --> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb24013e70 con 0x7fdb08081700 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 20 client.4459.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 1 -- 10.20.193.16:0/130213896 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdb2405e620 con 0x7fdb38074dc0 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb34ff9640 10 monclient: handle_get_version_reply finishing 1 version 33 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fd9ddffb640 1 -- 10.20.193.16:0/2689701979 <== osd.9 v2:10.20.193.128:6800/1813385411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fd9e0016ec0 con 0x7fd9c0081760 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fd9ddffb640 10 client.4453.objecter ms_dispatch2 0x7fd9f0000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fd9ddffb640 10 client.4453.objecter _finish_command 1 = system:0 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fd9ddffb640 20 client.4453.objecter put_session s=0x7fd9c0081090 osd=9 4 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fd9ddffb640 15 client.4453.objecter _session_command_op_remove 9 1 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb3cb90640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x7fdb08081700 0x7fdb08083b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.959+0000 7fdb3cb90640 10 monclient: get_auth_request con 0x7fdb08081700 auth_method 0 2026-02-17T05:56:47.013 INFO:teuthology.orchestra.run.trial016.stdout:120259084296 2026-02-17T05:56:47.013 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 10 client.4490.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 15 client.4490.objecter _session_command_op_assign -1 2 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 20 client.4490.objecter _get_session s=0x7f95e8080fb0 osd=7 3 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 20 client.4490.objecter put_session s=0x7f95e8080fb0 osd=7 3 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 20 client.4490.objecter _get_session s=0x7f95e8080fb0 osd=7 3 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 15 client.4490.objecter _session_command_op_remove -1 2 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 20 client.4490.objecter get_session s=0x7f95e8080fb0 osd=7 3 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 15 client.4490.objecter _session_command_op_assign 7 2 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 20 client.4490.objecter put_session s=0x7f95e8080fb0 osd=7 4 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 10 client.4490.objecter _send_command 2 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.960+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 --> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9618151600 con 0x7f95e8081680 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.961+0000 7fdb3cb90640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x7fdb08081700 0x7fdb08083b40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.961+0000 7fdb34ff9640 10 client.4459.objecter ms_handle_connect 0x7fdb08081700 2026-02-17T05:56:47.014 INFO:teuthology.orchestra.run.trial016.stdout:107374182409 2026-02-17T05:56:47.014 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f3435ffb640 1 -- 10.20.193.16:0/1220374686 <== osd.6 v2:10.20.193.148:6824/2491860138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f3418084190 con 0x7f3418081680 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f3435ffb640 10 client.4487.objecter ms_dispatch2 0x7f3448000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f3435ffb640 10 client.4487.objecter _finish_command 1 = system:0 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f3435ffb640 20 client.4487.objecter put_session s=0x7f3418080fb0 osd=6 4 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f3435ffb640 15 client.4487.objecter _session_command_op_remove 6 1 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f960d7fa640 1 -- 10.20.193.16:0/3204312853 <== osd.7 v2:10.20.193.164:6816/3942637745 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9618151600 con 0x7f95e8081680 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f960d7fa640 10 client.4490.objecter ms_dispatch2 0x7f9618000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f960d7fa640 10 client.4490.objecter _finish_command 2 = system:0 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f960d7fa640 20 client.4490.objecter put_session s=0x7f95e8080fb0 osd=7 4 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f960d7fa640 15 client.4490.objecter _session_command_op_remove 7 2 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 10 librados: watch_flush enter 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 10 librados: watch_flush exit 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 10 client.4490.objecter close_session for osd.7 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 >> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] conn(0x7f95e8081680 msgr2=0x7f95e8083ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] conn(0x7f95e8081680 0x7f95e8083ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 20 client.4490.objecter put_session s=0x7f95e8080fb0 osd=7 1 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 20 client.4490.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 10 client.4490.objecter successfully canceled tick 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f95e8079140 msgr2=0x7f95e807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f95e8079140 0x7f95e807b600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9608038000 tx=0x7f9608031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 10 monclient: shutdown 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f96181561e0 msgr2=0x7f96180d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 --2- 10.20.193.16:0/3204312853 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f96181561e0 0x7f96180d0b20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f961400b550 tx=0x7f961400ba20 comp rx=0 tx=0).stop 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961d116640 1 -- 10.20.193.16:0/3204312853 reap_dead start 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 shutdown_connections 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 >> 10.20.193.16:0/3204312853 conn(0x7f96180ce6c0 msgr2=0x7f961806d470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 shutdown_connections 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 -- 10.20.193.16:0/3204312853 wait complete. 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.964+0000 7f961eba0640 1 librados: shutdown 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 10 client.4453.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 15 client.4453.objecter _session_command_op_assign -1 2 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 20 client.4453.objecter _get_session s=0x7fd9c0081090 osd=9 3 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 20 client.4453.objecter put_session s=0x7fd9c0081090 osd=9 3 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 20 client.4453.objecter _get_session s=0x7fd9c0081090 osd=9 3 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 15 client.4453.objecter _session_command_op_remove -1 2 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 20 client.4453.objecter get_session s=0x7fd9c0081090 osd=9 3 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 15 client.4453.objecter _session_command_op_assign 9 2 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 20 client.4453.objecter put_session s=0x7fd9c0081090 osd=9 4 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 10 client.4453.objecter _send_command 2 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.966+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 --> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd9f00ca700 con 0x7fd9c0081760 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9ddffb640 1 -- 10.20.193.16:0/2689701979 <== osd.9 v2:10.20.193.128:6800/1813385411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd9f00ca700 con 0x7fd9c0081760 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9ddffb640 10 client.4453.objecter ms_dispatch2 0x7fd9f0000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9ddffb640 10 client.4453.objecter _finish_command 2 = system:0 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9ddffb640 20 client.4453.objecter put_session s=0x7fd9c0081090 osd=9 4 2026-02-17T05:56:47.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9ddffb640 15 client.4453.objecter _session_command_op_remove 9 2 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 10 librados: watch_flush enter 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 10 librados: watch_flush exit 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 10 client.4453.objecter close_session for osd.9 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 >> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] conn(0x7fd9c0081760 msgr2=0x7fd9c0083ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] conn(0x7fd9c0081760 0x7fd9c0083ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 20 client.4453.objecter put_session s=0x7fd9c0081090 osd=9 1 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 20 client.4453.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 10 client.4453.objecter successfully canceled tick 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd9c00791c0 msgr2=0x7fd9c007b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd9c00791c0 0x7fd9c007b680 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd9e402ee50 tx=0x7fd9e4033000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 10 monclient: shutdown 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd9f0074dc0 msgr2=0x7fd9f01f46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 --2- 10.20.193.16:0/2689701979 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd9f0074dc0 0x7fd9f01f46d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000b550 tx=0x7fd9e000ba20 comp rx=0 tx=0).stop 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f5d30640 1 -- 10.20.193.16:0/2689701979 reap_dead start 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 shutdown_connections 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 >> 10.20.193.16:0/2689701979 conn(0x7fd9f00ce6c0 msgr2=0x7fd9f00ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 shutdown_connections 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 -- 10.20.193.16:0/2689701979 wait complete. 2026-02-17T05:56:47.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.970+0000 7fd9f77ba640 1 librados: shutdown 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 10 client.4487.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 15 client.4487.objecter _session_command_op_assign -1 2 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 20 client.4487.objecter _get_session s=0x7f3418080fb0 osd=6 3 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 20 client.4487.objecter put_session s=0x7f3418080fb0 osd=6 3 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 20 client.4487.objecter _get_session s=0x7f3418080fb0 osd=6 3 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 15 client.4487.objecter _session_command_op_remove -1 2 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 20 client.4487.objecter get_session s=0x7f3418080fb0 osd=6 3 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 15 client.4487.objecter _session_command_op_assign 6 2 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 20 client.4487.objecter put_session s=0x7f3418080fb0 osd=6 4 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 10 client.4487.objecter _send_command 2 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.971+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 --> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f34480c8fa0 con 0x7f3418081680 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stdout:115964117001 2026-02-17T05:56:47.018 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.974+0000 7f3435ffb640 1 -- 10.20.193.16:0/1220374686 <== osd.6 v2:10.20.193.148:6824/2491860138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f34480c8fa0 con 0x7f3418081680 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.974+0000 7f3435ffb640 10 client.4487.objecter ms_dispatch2 0x7f3448000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.974+0000 7f3435ffb640 10 client.4487.objecter _finish_command 2 = system:0 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.974+0000 7f3435ffb640 20 client.4487.objecter put_session s=0x7f3418080fb0 osd=6 4 2026-02-17T05:56:47.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.974+0000 7f3435ffb640 15 client.4487.objecter _session_command_op_remove 6 2 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.974+0000 7f344f787640 10 librados: watch_flush enter 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 10 librados: watch_flush exit 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 10 client.4487.objecter close_session for osd.6 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 >> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] conn(0x7f3418081680 msgr2=0x7f3418083ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] conn(0x7f3418081680 0x7f3418083ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 20 client.4487.objecter put_session s=0x7f3418080fb0 osd=6 1 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 20 client.4487.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 10 client.4487.objecter successfully canceled tick 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3418079140 msgr2=0x7f341807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3418079140 0x7f341807b600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3438006b90 tx=0x7f3438006b20 comp rx=0 tx=0).stop 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 10 monclient: shutdown 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34481561e0 msgr2=0x7f34481f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 --2- 10.20.193.16:0/1220374686 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34481561e0 0x7f34481f3b30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f344400cb80 tx=0x7f344400a430 comp rx=0 tx=0).stop 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344dcfd640 1 -- 10.20.193.16:0/1220374686 reap_dead start 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 shutdown_connections 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 >> 10.20.193.16:0/1220374686 conn(0x7f34480ce6c0 msgr2=0x7f34480ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 shutdown_connections 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 -- 10.20.193.16:0/1220374686 wait complete. 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.975+0000 7f344f787640 1 librados: shutdown 2026-02-17T05:56:47.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.977+0000 7f43b5ffb640 1 -- 10.20.193.16:0/1961814941 <== osd.13 v2:10.20.193.128:6816/1434515412 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f43980841e0 con 0x7f43980816d0 2026-02-17T05:56:47.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.977+0000 7f43b5ffb640 10 client.4493.objecter ms_dispatch2 0x7f43c8000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:47.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.977+0000 7f43b5ffb640 10 client.4493.objecter _finish_command 1 = system:0 2026-02-17T05:56:47.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.977+0000 7f43b5ffb640 20 client.4493.objecter put_session s=0x7f4398081000 osd=13 4 2026-02-17T05:56:47.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.977+0000 7f43b5ffb640 15 client.4493.objecter _session_command_op_remove 13 1 2026-02-17T05:56:47.020 INFO:teuthology.orchestra.run.trial016.stdout:98784247812 2026-02-17T05:56:47.020 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-17T05:56:47.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.980+0000 7ff9237fe640 1 -- 10.20.193.16:0/994528860 <== osd.15 v2:10.20.193.164:6800/1881720827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7ff934013e70 con 0x7ff910081680 2026-02-17T05:56:47.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.980+0000 7ff9237fe640 10 client.4450.objecter ms_dispatch2 0x7ff944000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:47.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.980+0000 7ff9237fe640 10 client.4450.objecter _finish_command 1 = system:0 2026-02-17T05:56:47.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.980+0000 7ff9237fe640 20 client.4450.objecter put_session s=0x7ff910080fb0 osd=15 4 2026-02-17T05:56:47.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.980+0000 7ff9237fe640 15 client.4450.objecter _session_command_op_remove 15 1 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stdout:90194313221 2026-02-17T05:56:47.021 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.982+0000 7fdb34ff9640 1 -- 10.20.193.16:0/130213896 <== osd.3 v2:10.20.193.164:6824/3772404125 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fdb24013e70 con 0x7fdb08081700 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.982+0000 7fdb34ff9640 10 client.4459.objecter ms_dispatch2 0x7fdb38000b90 command_reply(tid 1: 0 ) 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.982+0000 7fdb34ff9640 10 client.4459.objecter _finish_command 1 = system:0 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.982+0000 7fdb34ff9640 20 client.4459.objecter put_session s=0x7fdb08081030 osd=3 4 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.982+0000 7fdb34ff9640 15 client.4459.objecter _session_command_op_remove 3 1 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 10 client.4493.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 15 client.4493.objecter _session_command_op_assign -1 2 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 20 client.4493.objecter _get_session s=0x7f4398081000 osd=13 3 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 20 client.4493.objecter put_session s=0x7f4398081000 osd=13 3 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 20 client.4493.objecter _get_session s=0x7f4398081000 osd=13 3 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 15 client.4493.objecter _session_command_op_remove -1 2 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 20 client.4493.objecter get_session s=0x7f4398081000 osd=13 3 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 15 client.4493.objecter _session_command_op_assign 13 2 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 20 client.4493.objecter put_session s=0x7f4398081000 osd=13 4 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 10 client.4493.objecter _send_command 2 2026-02-17T05:56:47.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.985+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 --> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f43c80ca700 con 0x7f43980816d0 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 10 client.4450.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 15 client.4450.objecter _session_command_op_assign -1 2 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 20 client.4450.objecter _get_session s=0x7ff910080fb0 osd=15 3 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 20 client.4450.objecter put_session s=0x7ff910080fb0 osd=15 3 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 20 client.4450.objecter _get_session s=0x7ff910080fb0 osd=15 3 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 15 client.4450.objecter _session_command_op_remove -1 2 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 20 client.4450.objecter get_session s=0x7ff910080fb0 osd=15 3 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 15 client.4450.objecter _session_command_op_assign 15 2 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 20 client.4450.objecter put_session s=0x7ff910080fb0 osd=15 4 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 10 client.4450.objecter _send_command 2 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 --> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff944141d30 con 0x7ff910081680 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43b5ffb640 1 -- 10.20.193.16:0/1961814941 <== osd.13 v2:10.20.193.128:6816/1434515412 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f43c80ca700 con 0x7f43980816d0 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43b5ffb640 10 client.4493.objecter ms_dispatch2 0x7f43c8000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43b5ffb640 10 client.4493.objecter _finish_command 2 = system:0 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43b5ffb640 20 client.4493.objecter put_session s=0x7f4398081000 osd=13 4 2026-02-17T05:56:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43b5ffb640 15 client.4493.objecter _session_command_op_remove 13 2 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 10 librados: watch_flush enter 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 10 librados: watch_flush exit 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 10 client.4493.objecter close_session for osd.13 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 >> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] conn(0x7f43980816d0 msgr2=0x7f4398083b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] conn(0x7f43980816d0 0x7f4398083b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 20 client.4493.objecter put_session s=0x7f4398081000 osd=13 1 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 20 client.4493.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 10 client.4493.objecter successfully canceled tick 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4398079190 msgr2=0x7f439807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4398079190 0x7f439807b650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f43b800a8d0 tx=0x7f43b8008040 comp rx=0 tx=0).stop 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 10 monclient: shutdown 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f43c815fa10 msgr2=0x7f43c81eabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 --2- 10.20.193.16:0/1961814941 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f43c815fa10 0x7f43c81eabc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009e50 tx=0x7f43c4004b40 comp rx=0 tx=0).stop 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cd85e640 1 -- 10.20.193.16:0/1961814941 reap_dead start 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 shutdown_connections 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 >> 10.20.193.16:0/1961814941 conn(0x7f43c80ce6c0 msgr2=0x7f43c806d470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 shutdown_connections 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 -- 10.20.193.16:0/1961814941 wait complete. 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.988+0000 7f43cf2e8640 1 librados: shutdown 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 10 client.4459.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 15 client.4459.objecter _session_command_op_assign -1 2 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 20 client.4459.objecter _get_session s=0x7fdb08081030 osd=3 3 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 20 client.4459.objecter put_session s=0x7fdb08081030 osd=3 3 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 20 client.4459.objecter _get_session s=0x7fdb08081030 osd=3 3 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 15 client.4459.objecter _session_command_op_remove -1 2 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 20 client.4459.objecter get_session s=0x7fdb08081030 osd=3 3 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 15 client.4459.objecter _session_command_op_assign 3 2 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 20 client.4459.objecter put_session s=0x7fdb08081030 osd=3 4 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 10 client.4459.objecter _send_command 2 2026-02-17T05:56:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.989+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 --> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb3806e090 con 0x7fdb08081700 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.991+0000 7ff9237fe640 1 -- 10.20.193.16:0/994528860 <== osd.15 v2:10.20.193.164:6800/1881720827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff944141d30 con 0x7ff910081680 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.991+0000 7ff9237fe640 10 client.4450.objecter ms_dispatch2 0x7ff944000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.991+0000 7ff9237fe640 10 client.4450.objecter _finish_command 2 = system:0 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.991+0000 7ff9237fe640 20 client.4450.objecter put_session s=0x7ff910080fb0 osd=15 4 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.991+0000 7ff9237fe640 15 client.4450.objecter _session_command_op_remove 15 2 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.991+0000 7ff9493d3640 10 librados: watch_flush enter 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.991+0000 7ff9493d3640 10 librados: watch_flush exit 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.991+0000 7ff9493d3640 10 client.4450.objecter close_session for osd.15 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.991+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 >> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] conn(0x7ff910081680 msgr2=0x7ff910083ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] conn(0x7ff910081680 0x7ff910083ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 20 client.4450.objecter put_session s=0x7ff910080fb0 osd=15 1 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 20 client.4450.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 10 client.4450.objecter successfully canceled tick 2026-02-17T05:56:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff910079140 msgr2=0x7ff91007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff910079140 0x7ff91007b600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff938039000 tx=0x7ff938031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 10 monclient: shutdown 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff9441561e0 msgr2=0x7ff9441fa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 1 --2- 10.20.193.16:0/994528860 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff9441561e0 0x7ff9441fa490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff93400cb20 tx=0x7ff93400a060 comp rx=0 tx=0).stop 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9437fe640 1 -- 10.20.193.16:0/994528860 reap_dead start 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 shutdown_connections 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 >> 10.20.193.16:0/994528860 conn(0x7ff9440ce6c0 msgr2=0x7ff94406d470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 shutdown_connections 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 1 -- 10.20.193.16:0/994528860 wait complete. 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.992+0000 7ff9493d3640 1 librados: shutdown 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb34ff9640 1 -- 10.20.193.16:0/130213896 <== osd.3 v2:10.20.193.164:6824/3772404125 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdb3806e090 con 0x7fdb08081700 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb34ff9640 10 client.4459.objecter ms_dispatch2 0x7fdb38000b90 command_reply(tid 2: 0 ) 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb34ff9640 10 client.4459.objecter _finish_command 2 = system:0 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb34ff9640 20 client.4459.objecter put_session s=0x7fdb08081030 osd=3 4 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb34ff9640 15 client.4459.objecter _session_command_op_remove 3 2 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 10 librados: watch_flush enter 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 10 librados: watch_flush exit 2026-02-17T05:56:47.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 10 client.4459.objecter close_session for osd.3 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x7fdb08081700 msgr2=0x7fdb08083b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x7fdb08081700 0x7fdb08083b40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 20 client.4459.objecter put_session s=0x7fdb08081030 osd=3 1 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 20 client.4459.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 10 client.4459.objecter successfully canceled tick 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdb080791c0 msgr2=0x7fdb0807b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdb080791c0 0x7fdb0807b680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb28039000 tx=0x7fdb28031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 10 monclient: shutdown 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdb38074dc0 msgr2=0x7fdb3814dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 --2- 10.20.193.16:0/130213896 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdb38074dc0 0x7fdb3814dec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400c700 tx=0x7fdb2400cbd0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3cb90640 1 -- 10.20.193.16:0/130213896 reap_dead start 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 shutdown_connections 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 >> 10.20.193.16:0/130213896 conn(0x7fdb380ce6c0 msgr2=0x7fdb380c8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 shutdown_connections 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 -- 10.20.193.16:0/130213896 wait complete. 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:46.993+0000 7fdb3e61a640 1 librados: shutdown 2026-02-17T05:56:47.026 INFO:teuthology.orchestra.run.trial016.stdout:115964116998 2026-02-17T05:56:47.026 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2026-02-17T05:56:47.027 INFO:teuthology.orchestra.run.trial016.stdout:94489280519 2026-02-17T05:56:47.027 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-17T05:56:47.027 INFO:teuthology.orchestra.run.trial016.stdout:98784247812 2026-02-17T05:56:47.028 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-17T05:56:47.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.103+0000 7fe913314640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.103+0000 7fe913314640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.103+0000 7fe913314640 10 monclient: monmap: 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.104567+0000 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.104567+0000 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.104+0000 7faa6844c640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.104+0000 7faa6844c640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.104+0000 7faa6844c640 10 monclient: monmap: 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.105293+0000 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.105293+0000 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.104 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.104+0000 7fe913314640 1 Processor -- start 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.104+0000 7fe913314640 1 -- start start 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.104+0000 7fe913314640 10 monclient: init 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.104+0000 7fd0236fa640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.104+0000 7fd0236fa640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.104+0000 7fd0236fa640 10 monclient: monmap: 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.105792+0000 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.105792+0000 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.104+0000 7faa6844c640 1 Processor -- start 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fd0236fa640 1 Processor -- start 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fd0236fa640 1 -- start start 2026-02-17T05:56:47.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fd0236fa640 10 monclient: init 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 10 monclient: _add_conns ranks=[5,2,1,0,3,4] 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fe90c155fa0 0x7fe90c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 10 monclient(hunting): picked mon.noname-c con 0x7fe90c155fa0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe90c156380 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fe90c15d360 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe90c0ccae0 con 0x7fe90c15d360 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe90c0ccdc0 con 0x7fe90c156380 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7faa6844c640 1 -- start start 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7faa6844c640 10 monclient: init 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe90c0cd9d0 con 0x7fe90c155fa0 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7fe913314640 10 monclient(hunting): authenticate will time out at 1059.674839s 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7faa6844c640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7faa6844c640 10 monclient: _add_conns ranks=[4,2,0,5,1,3] 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7faa6844c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7faa60155fa0 0x7faa600cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7f1fb8e05640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7f1fb8e05640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7f1fb8e05640 10 monclient: monmap: 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.106979+0000 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.106979+0000 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7faa6844c640 10 monclient(hunting): picked mon.noname-b con 0x7faa60155fa0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.105+0000 7faa6844c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faa60156380 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7faa6844c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faa6015d360 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7faa6844c640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa600ccae0 con 0x7faa6015d360 2026-02-17T05:56:47.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fe911089640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fe90c155fa0 0x7fe90c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7faa6844c640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa600ccdc0 con 0x7faa60156380 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7faa6844c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7faa6844c640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7faa600cd9d0 con 0x7faa60155fa0 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7faa6844c640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7faa6844c640 10 monclient(hunting): authenticate will time out at 1059.675179s 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fe91188a640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fe90c15d360 0x7fe90c15d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:53754/0 (socket says 10.20.193.16:53754) 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fe91188a640 1 -- 10.20.193.16:0/1084230737 learned_addr learned my addr 10.20.193.16:0/1084230737 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fe911089640 10 monclient(hunting): get_auth_request con 0x7fe90c155fa0 auth_method 0 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 10 monclient: _add_conns ranks=[2,1,4,5,0,3] 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd01c14e480 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fe911089640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fe911089640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fd01c0c8050 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:47.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd01c14e860 0x7fd01c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fe911089640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 10 monclient(hunting): picked mon.noname-b con 0x7fd01c14e860 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7f1fb8e05640 1 Processor -- start 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7faa669c2640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7faa6015d360 0x7faa6015d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:56104/0 (socket says 10.20.193.16:56104) 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7faa669c2640 1 -- 10.20.193.16:0/2712949632 learned_addr learned my addr 10.20.193.16:0/2712949632 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd01c0c84d0 con 0x7fd01c0c8050 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd01c0c87b0 con 0x7fd01c14e480 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fd01c0c8a90 con 0x7fd01c14e860 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7fd0236fa640 10 monclient(hunting): authenticate will time out at 1059.675775s 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7faa661c1640 1 --2- 10.20.193.16:0/2712949632 >> v2:10.20.193.128:3300/0 conn(0x7faa60155fa0 0x7faa600cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7fd021c70640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd01c14e860 0x7fd01c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7fe911089640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7fe911089640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7fe911089640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7faa661c1640 10 monclient(hunting): get_auth_request con 0x7faa60155fa0 auth_method 0 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7faa661c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7faa661c1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7faa661c1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7fd020c6e640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fd01c0c8050 0x7fd01c15b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:53766/0 (socket says 10.20.193.16:53766) 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7fd020c6e640 1 -- 10.20.193.16:0/3452114239 learned_addr learned my addr 10.20.193.16:0/3452114239 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7f1fb8e05640 1 -- start start 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.106+0000 7f1fb8e05640 10 monclient: init 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 10 monclient: _add_conns ranks=[4,2,1,0,3,5] 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1fb414e870 0x7f1fb40c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 10 monclient(hunting): picked mon.noname-b con 0x7f1fb414e870 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7fd021c70640 10 monclient(hunting): get_auth_request con 0x7fd01c14e860 auth_method 0 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1fb414ec50 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f1fb40c84b0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:47.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb40c8930 con 0x7f1fb40c84b0 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7faa661c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7faa661c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7faa661c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb40c8c10 con 0x7f1fb414ec50 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7fd021c70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7fd021c70640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7fd021c70640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb40c9820 con 0x7f1fb414e870 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.107+0000 7f1fb8e05640 10 monclient(hunting): authenticate will time out at 1059.676819s 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fd021c70640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fd021c70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fd021c70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fe911089640 10 monclient(hunting): handle_auth_done global_id 4496 payload 274 2026-02-17T05:56:47.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fe911089640 1 -- 10.20.193.16:0/1084230737 >> v1:10.20.193.148:6789/0 conn(0x7fe90c156380 legacy=0x7fe90c0740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fe911089640 1 -- 10.20.193.16:0/1084230737 >> v1:10.20.193.128:6789/0 conn(0x7fe90c15d360 legacy=0x7fe90c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fe911089640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fe911089640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fe911089640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fe911089640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7fe911089640 1 -- 10.20.193.16:0/1084230737 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c0cd0a0 con 0x7fe90c155fa0 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7faa651bf640 1 -- 10.20.193.16:0/2712949632 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 208350509 0 0) 0x7faa600ccae0 con 0x7faa6015d360 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7faa651bf640 10 monclient(hunting): _negotiate 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7faa651bf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7faa651bf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7faa651bf640 10 monclient(hunting): my global_id is 4413 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.108+0000 7faa651bf640 1 -- 10.20.193.16:0/2712949632 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa48003620 con 0x7faa6015d360 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7faa661c1640 10 monclient(hunting): handle_auth_done global_id 4462 payload 274 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7faa661c1640 1 -- 10.20.193.16:0/2712949632 >> v1:10.20.193.148:6789/0 conn(0x7faa60156380 legacy=0x7faa600740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7faa661c1640 1 -- 10.20.193.16:0/2712949632 >> v1:10.20.193.16:6789/0 conn(0x7faa6015d360 legacy=0x7faa6015d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7faa661c1640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7faa661c1640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7faa661c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7faa661c1640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7faa661c1640 1 -- 10.20.193.16:0/2712949632 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa600cd0a0 con 0x7faa60155fa0 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7fd021c70640 10 monclient(hunting): handle_auth_done global_id 4465 payload 274 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7fd021c70640 1 -- 10.20.193.16:0/3452114239 >> v1:10.20.193.148:6789/0 conn(0x7fd01c14e480 legacy=0x7fd01c1590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7fd021c70640 1 -- 10.20.193.16:0/3452114239 >> v1:10.20.193.128:6789/0 conn(0x7fd01c0c8050 legacy=0x7fd01c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7fd021c70640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7fd021c70640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7fd021c70640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7fd021c70640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.110+0000 7fd021c70640 1 -- 10.20.193.16:0/3452114239 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c0c8d70 con 0x7fd01c14e860 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.111+0000 7f1956b79640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.111+0000 7f1956b79640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.111+0000 7f1956b79640 10 monclient: monmap: 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.112590+0000 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.112590+0000 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.111+0000 7fe911089640 1 --2- 10.20.193.16:0/1084230737 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe90c155fa0 0x7fe90c0cc1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0cdb50 tx=0x7fe8fc02ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4126ef3fc1db46e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.111+0000 7efc948b0640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.111+0000 7efc948b0640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.111+0000 7efc948b0640 10 monclient: monmap: 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.112938+0000 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.112938+0000 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.113 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7f1956b79640 1 Processor -- start 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7f1956b79640 1 -- start start 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7f1956b79640 10 monclient: init 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7efc948b0640 1 Processor -- start 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7efc948b0640 1 -- start start 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7efc948b0640 10 monclient: init 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7f1956b79640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7f1956b79640 10 monclient: _add_conns ranks=[4,0,5,2,3,1] 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7f1fb1d74640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f1fb414ec50 0x7f1fb415b590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46520/0 (socket says 10.20.193.16:46520) 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7f1fb1d74640 1 -- 10.20.193.16:0/3654238760 learned_addr learned my addr 10.20.193.16:0/3654238760 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7f1956b79640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1950155e00 0x7f19500ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7f1956b79640 10 monclient(hunting): picked mon.noname-b con 0x7f1950155e00 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.112+0000 7f1956b79640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f19501561e0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7fd021c70640 1 --2- 10.20.193.16:0/3452114239 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd01c14e860 0x7fd01c06e0b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd018009e10 tx=0x7fd018030500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d44e112a515d200c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1956b79640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f1950074dc0 0x7f19500cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1956b79640 10 monclient(hunting): picked mon.noname-c con 0x7f1950074dc0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1956b79640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19500c96d0 con 0x7f19501561e0 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1956b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1956b79640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f1950074860 con 0x7f1950155e00 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1956b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1956b79640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f19500c9940 con 0x7f1950074dc0 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1956b79640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1956b79640 10 monclient(hunting): authenticate will time out at 1059.682178s 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 10 monclient: _add_conns ranks=[3,1,4,5,2,0] 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7efc8c155fa0 0x7efc8c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 10 monclient(hunting): picked mon.noname-a con 0x7efc8c155fa0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7faa661c1640 1 --2- 10.20.193.16:0/2712949632 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa60155fa0 0x7faa600cc1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faa600cdb50 tx=0x7faa5002ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9060504c04f0799b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7efc8c156380 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7efc8c15d360 0x7efc8c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 10 monclient(hunting): picked mon.noname-b con 0x7efc8c15d360 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc8c0ccae0 con 0x7efc8c156380 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7efc8c0cd9d0 con 0x7efc8c155fa0 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7efc8c0ccdc0 con 0x7efc8c15d360 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7efc948b0640 10 monclient(hunting): authenticate will time out at 1059.682400s 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1947fff640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f19501561e0 0x7f1950074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:56120/0 (socket says 10.20.193.16:56120) 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.113+0000 7f1947fff640 1 -- 10.20.193.16:0/4037480458 learned_addr learned my addr 10.20.193.16:0/4037480458 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7efc92625640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7efc8c155fa0 0x7efc8c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f7406d49640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f7406d49640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f7406d49640 10 monclient: monmap: 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.115066+0000 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.115066+0000 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.116 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7efc92625640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7efc8c155fa0 0x7efc8c0cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59794/0 (socket says 10.20.193.16:59794) 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7efc92625640 1 -- 10.20.193.16:0/2973515293 learned_addr learned my addr 10.20.193.16:0/2973515293 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7efc92625640 10 monclient(hunting): get_auth_request con 0x7efc8c155fa0 auth_method 0 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7efc92625640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7efc92625640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7efc92625640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f7406d49640 1 Processor -- start 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f7406d49640 1 -- start start 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f7406d49640 10 monclient: init 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 499375503 0 0) 0x7f19500c96d0 con 0x7f19501561e0 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f19477fe640 10 monclient(hunting): _negotiate 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f19477fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f19477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f19477fe640 10 monclient(hunting): my global_id is 4416 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.114+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f193c003620 con 0x7f19501561e0 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7efc92625640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7efc92625640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7efc92625640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 10 monclient: _add_conns ranks=[4,5,0,3,1,2] 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f7400155e00 0x7f74000ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 10 monclient(hunting): picked mon.noname-b con 0x7f7400155e00 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f74001561e0 0x7f74000cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 10 monclient(hunting): picked mon.noname-c con 0x7f74001561e0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7400074dc0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74000c96d0 con 0x7f7400074dc0 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f7400074860 con 0x7f7400155e00 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f74000c9940 con 0x7f74001561e0 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f7406d49640 10 monclient(hunting): authenticate will time out at 1059.684618s 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f97f9379640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f97f9379640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7f97f9379640 10 monclient: monmap: 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.116869+0000 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.116869+0000 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7fe8fbfff640 1 -- 10.20.193.16:0/1084230737 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc03c070 con 0x7fe90c155fa0 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.115+0000 7fe8fbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 10 monclient: dump: 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 1 -- 10.20.193.16:0/1084230737 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8fc02fb90 con 0x7fe90c155fa0 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 1 -- 10.20.193.16:0/1084230737 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc037440 con 0x7fe90c155fa0 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe8fbfff640 10 monclient: dump: 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe913314640 5 monclient: authenticate success, global_id 4496 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe913314640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe913314640 4 set_mon_vals no callback set 2026-02-17T05:56:47.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f19550ef640 1 --2- 10.20.193.16:0/4037480458 >> v2:10.20.193.148:3300/0 conn(0x7f1950074dc0 0x7f19500cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7efc92625640 10 monclient(hunting): handle_auth_done global_id 4419 payload 274 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7efc92625640 1 -- 10.20.193.16:0/2973515293 >> v2:10.20.193.128:3300/0 conn(0x7efc8c15d360 msgr2=0x7efc8c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7efc92625640 1 --2- 10.20.193.16:0/2973515293 >> v2:10.20.193.128:3300/0 conn(0x7efc8c15d360 0x7efc8c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7efc92625640 1 -- 10.20.193.16:0/2973515293 >> v1:10.20.193.128:6789/0 conn(0x7efc8c156380 legacy=0x7efc8c0740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7efc92625640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7efc92625640 1 monclient: found mon.noname-a 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7efc92625640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7efc92625640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7efc92625640 1 -- 10.20.193.16:0/2973515293 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8c0cd0a0 con 0x7efc8c155fa0 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f8363961640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f8363961640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f8363961640 10 monclient: monmap: 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.117270+0000 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.117270+0000 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f97f9379640 1 Processor -- start 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f73f7fff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f74001561e0 0x7f74000cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f97f9379640 1 -- start start 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f97f9379640 10 monclient: init 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430290317 0 0) 0x7f1fb40c8c10 con 0x7f1fb414ec50 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f1fb1573640 10 monclient(hunting): _negotiate 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f1fb1573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f1fb1573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f1fb1573640 10 monclient(hunting): my global_id is 4505 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa0003620 con 0x7f1fb414ec50 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe913314640 10 monclient: shutdown 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe913314640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe913314640 1 -- 10.20.193.16:0/1084230737 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe90c155fa0 msgr2=0x7fe90c0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe913314640 1 --2- 10.20.193.16:0/1084230737 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe90c155fa0 0x7fe90c0cc1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0cdb50 tx=0x7fe8fc02ee80 comp rx=0 tx=0).stop 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe913314640 1 -- 10.20.193.16:0/1084230737 shutdown_connections 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe913314640 1 --2- 10.20.193.16:0/1084230737 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe90c155fa0 0x7fe90c0cc1a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f8363961640 1 Processor -- start 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7fe913314640 1 -- 10.20.193.16:0/1084230737 >> 10.20.193.16:0/1084230737 conn(0x7fe90c0ce6c0 msgr2=0x7fe90c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f8363961640 1 -- start start 2026-02-17T05:56:47.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f8363961640 10 monclient: init 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f19550ef640 10 monclient(hunting): get_auth_request con 0x7f1950074dc0 auth_method 0 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f19550ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f19550ef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f19550ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f19550ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f19550ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.116+0000 7f19550ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7fe913314640 1 -- 10.20.193.16:0/1084230737 shutdown_connections 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7fe913314640 1 -- 10.20.193.16:0/1084230737 wait complete. 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f73f7fff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f74001561e0 0x7f74000cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56976/0 (socket says 10.20.193.16:56976) 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f73f7fff640 1 -- 10.20.193.16:0/3590427467 learned_addr learned my addr 10.20.193.16:0/3590427467 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f73f7fff640 10 monclient(hunting): get_auth_request con 0x7f74001561e0 auth_method 0 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f73f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f73f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f73f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 10 monclient: _add_conns ranks=[2,5,0,3,4,1] 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f97f4155e00 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f97f41561e0 0x7f97f40ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 10 monclient(hunting): picked mon.noname-c con 0x7f97f41561e0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f97f4074dc0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f40c96d0 con 0x7f97f4074dc0 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f4074860 con 0x7f97f4155e00 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f97f40c9940 con 0x7f97f41561e0 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f97f9379640 10 monclient(hunting): authenticate will time out at 1059.686235s 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f19548ee640 1 --2- 10.20.193.16:0/4037480458 >> v2:10.20.193.128:3300/0 conn(0x7f1950155e00 0x7f19500ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 10 monclient: _add_conns ranks=[4,0,5,1,3,2] 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f835c155e00 0x7f835c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 10 monclient(hunting): picked mon.noname-b con 0x7f835c155e00 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f835c1561e0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f835c074dc0 0x7f835c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 10 monclient(hunting): picked mon.noname-c con 0x7f835c074dc0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835c0c96d0 con 0x7f835c1561e0 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f835c074860 con 0x7f835c155e00 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f835c0c9940 con 0x7f835c074dc0 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f8363961640 10 monclient(hunting): authenticate will time out at 1059.686494s 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f19548ee640 10 monclient(hunting): get_auth_request con 0x7f1950155e00 auth_method 0 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f19548ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f19548ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7f19548ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7fe913314640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7fe913314640 10 monclient: monmap: 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7fe913314640 1 librados: starting msgr at 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.117+0000 7fe913314640 1 librados: starting objecter 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 1 Processor -- start 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 1 -- start start 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 1 librados: setting wanted keys 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 1 librados: calling monclient init 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient: init 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19548ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19548ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19548ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe90c155fa0 0x7fe90c148430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient(hunting): picked mon.noname-b con 0x7fe90c155fa0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe90c156380 0x7fe90c1495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient(hunting): picked mon.noname-a con 0x7fe90c156380 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe90c15d360 0x7fe90c149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient(hunting): picked mon.noname-c con 0x7fe90c15d360 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c0747b0 con 0x7fe90c156380 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c0cd660 con 0x7fe90c155fa0 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 <== mon.0 v1:10.20.193.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 41194207 0 0) 0x7f193c003620 con 0x7f19501561e0 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 >> v2:10.20.193.148:3300/0 conn(0x7f1950074dc0 msgr2=0x7f19500cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 --2- 10.20.193.16:0/4037480458 >> v2:10.20.193.148:3300/0 conn(0x7f1950074dc0 0x7f19500cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c074d50 con 0x7fe90c15d360 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fe913314640 10 monclient(hunting): authenticate will time out at 1059.687626s 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 >> v2:10.20.193.128:3300/0 conn(0x7f1950155e00 msgr2=0x7f19500ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 --2- 10.20.193.16:0/4037480458 >> v2:10.20.193.128:3300/0 conn(0x7f1950155e00 0x7f19500ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.16:6789/0 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 --> v1:10.20.193.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19500c9c20 con 0x7f19501561e0 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 <== mon.0 v1:10.20.193.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f1938002f00 con 0x7f19501561e0 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:56:47.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient: dump: 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] but i'm connected to v1:10.20.193.16:6789/0, reconnecting 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 >> v1:10.20.193.16:6789/0 conn(0x7f19501561e0 legacy=0x7f1950074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f193c003ba0 0x7f193c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient(hunting): picked mon.b con 0x7f193c003ba0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c004900 0x7f193c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient(hunting): picked mon.c con 0x7f193c004900 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f193c0052c0 0x7f193c00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient(hunting): picked mon.a con 0x7f193c0052c0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1950074860 con 0x7f193c0052c0 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 1 -- 10.20.193.16:0/2712949632 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5003c070 con 0x7faa60155fa0 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 10 monclient: dump: 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f19500c9940 con 0x7f193c003ba0 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f193c00c190 con 0x7f193c004900 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7f19477fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 1 -- 10.20.193.16:0/2712949632 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa5002fb70 con 0x7faa60155fa0 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 1 -- 10.20.193.16:0/2712949632 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50037440 con 0x7faa60155fa0 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa651bf640 10 monclient: dump: 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa6844c640 5 monclient: authenticate success, global_id 4462 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa6844c640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7faa6844c640 4 set_mon_vals no callback set 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7faa6844c640 10 monclient: shutdown 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7faa6844c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7faa6844c640 1 -- 10.20.193.16:0/2712949632 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa60155fa0 msgr2=0x7faa600cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7faa6844c640 1 --2- 10.20.193.16:0/2712949632 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa60155fa0 0x7faa600cc1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faa600cdb50 tx=0x7faa5002ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:47.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd00bfff640 1 -- 10.20.193.16:0/3452114239 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018007740 con 0x7fd01c14e860 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd00bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd00bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd00bfff640 10 monclient: dump: 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd00bfff640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd00bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd00bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd00bfff640 1 -- 10.20.193.16:0/3452114239 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0180078e0 con 0x7fd01c14e860 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd0236fa640 5 monclient: authenticate success, global_id 4465 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd0236fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.118+0000 7fd00bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd00bfff640 1 -- 10.20.193.16:0/3452114239 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018007cb0 con 0x7fd01c14e860 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 4 set_mon_vals no callback set 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd00bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd00bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd00bfff640 10 monclient: dump: 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7efc92625640 1 --2- 10.20.193.16:0/2973515293 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc8c155fa0 0x7efc8c0cc1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efc8c08e730 tx=0x7efc8402ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce45d04359fa3df server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7faa6844c640 1 -- 10.20.193.16:0/2712949632 shutdown_connections 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7faa6844c640 1 --2- 10.20.193.16:0/2712949632 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa60155fa0 0x7faa600cc1a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7faa6844c640 1 -- 10.20.193.16:0/2712949632 >> 10.20.193.16:0/2712949632 conn(0x7faa600ce6c0 msgr2=0x7faa600cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7faa6844c640 1 -- 10.20.193.16:0/2712949632 shutdown_connections 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7faa6844c640 1 -- 10.20.193.16:0/2712949632 wait complete. 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 10 monclient: shutdown 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 1 -- 10.20.193.16:0/3452114239 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd01c14e860 msgr2=0x7fd01c06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 1 --2- 10.20.193.16:0/3452114239 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd01c14e860 0x7fd01c06e0b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd018009e10 tx=0x7fd018030500 comp rx=0 tx=0).stop 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 1 -- 10.20.193.16:0/3452114239 shutdown_connections 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 1 --2- 10.20.193.16:0/3452114239 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd01c14e860 0x7fd01c06e0b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 1 -- 10.20.193.16:0/3452114239 >> 10.20.193.16:0/3452114239 conn(0x7fd01c0ca150 msgr2=0x7fd01c0ca560 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 1 -- 10.20.193.16:0/3452114239 shutdown_connections 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fbfc4d49640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fbfc4d49640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fbfc4d49640 10 monclient: monmap: 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.120651+0000 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.120651+0000 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.129 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7fd0236fa640 1 -- 10.20.193.16:0/3452114239 wait complete. 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.119+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 10 monclient: monmap: 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fe910888640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe90c156380 0x7fe90c1495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient: monmap: 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.130 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 1 librados: starting msgr at 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 1 librados: starting objecter 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 1 Processor -- start 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fbfc4d49640 1 Processor -- start 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 1 -- start start 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 1 librados: setting wanted keys 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 1 librados: calling monclient init 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 10 monclient: init 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fbfc4d49640 1 -- start start 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fbfc4d49640 10 monclient: init 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 librados: starting msgr at 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 librados: starting objecter 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 Processor -- start 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 -- start start 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 librados: setting wanted keys 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 librados: calling monclient init 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient: init 2026-02-17T05:56:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7f1fb2575640 1 --2- 10.20.193.16:0/3654238760 >> v2:10.20.193.128:3300/0 conn(0x7f1fb414e870 0x7f1fb40c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fbfc4d49640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fbfc4d49640 10 monclient: _add_conns ranks=[1,4,0,2,5,3] 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd01c0c8050 0x7fd01c1f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fbfc4d49640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fbfc0155fa0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient(hunting): picked mon.noname-a con 0x7fd01c0c8050 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd01c14e480 0x7fd01c153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fbfc4d49640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fbfc0156380 0x7fbfc00740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient(hunting): picked mon.noname-b con 0x7fd01c14e480 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd01c14e860 0x7fd01c0701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient(hunting): picked mon.noname-c con 0x7fd01c14e860 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fbfc4d49640 10 monclient(hunting): picked mon.noname-b con 0x7fbfc0156380 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fbfc4d49640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbfc00745e0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c070f50 con 0x7fd01c0c8050 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c0c9330 con 0x7fd01c14e480 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c0710d0 con 0x7fd01c14e860 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fbfc4d49640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc00ccae0 con 0x7fbfc00745e0 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7faa6844c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faa60155fa0 0x7faa600ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.120+0000 7fd0236fa640 10 monclient(hunting): authenticate will time out at 1059.689981s 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fbfc4d49640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc00ccdc0 con 0x7fbfc0155fa0 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 10 monclient(hunting): picked mon.noname-c con 0x7faa60155fa0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7faa60156380 0x7faa6014bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fbfc4d49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fbfc4d49640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc00cc8b0 con 0x7fbfc0156380 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fbfc4d49640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fbfc4d49640 10 monclient(hunting): authenticate will time out at 1059.690095s 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 10 monclient(hunting): picked mon.noname-a con 0x7faa60156380 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa6015d360 0x7faa601f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 10 monclient(hunting): picked mon.noname-b con 0x7faa6015d360 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa600747b0 con 0x7faa60156380 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7faa600cd660 con 0x7faa6015d360 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7faa60074d50 con 0x7faa60155fa0 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7faa6844c640 10 monclient(hunting): authenticate will time out at 1059.690207s 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fe910888640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe90c156380 0x7fe90c1495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59798/0 (socket says 10.20.193.16:59798) 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fe910888640 1 -- 10.20.193.16:0/161933326 learned_addr learned my addr 10.20.193.16:0/161933326 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fe910888640 10 monclient(hunting): get_auth_request con 0x7fe90c156380 auth_method 0 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fe910888640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fe910888640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fe910888640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f30ef9b4640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f30ef9b4640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f30ef9b4640 10 monclient: monmap: 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.134 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.122396+0000 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.122396+0000 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f1fb2575640 10 monclient(hunting): get_auth_request con 0x7f1fb414e870 auth_method 0 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f1fb2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f1fb2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f1fb2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fe910888640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fe910888640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f021586f640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f021586f640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f021586f640 10 monclient: monmap: 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.122689+0000 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.122689+0000 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.135 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7fe910888640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f30ef9b4640 1 Processor -- start 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f30ef9b4640 1 -- start start 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f30ef9b4640 10 monclient: init 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f73f7fff640 10 monclient(hunting): handle_auth_done global_id 4511 payload 274 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f73f7fff640 1 -- 10.20.193.16:0/3590427467 >> v2:10.20.193.128:3300/0 conn(0x7f7400155e00 msgr2=0x7f74000ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f73f7fff640 1 --2- 10.20.193.16:0/3590427467 >> v2:10.20.193.128:3300/0 conn(0x7f7400155e00 0x7f74000ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f73f7fff640 1 -- 10.20.193.16:0/3590427467 >> v1:10.20.193.16:6789/0 conn(0x7f7400074dc0 legacy=0x7f7400074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f73f7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f73f7fff640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f73f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f73f7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.121+0000 7f73f7fff640 1 -- 10.20.193.16:0/3590427467 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74000c9c20 con 0x7f74001561e0 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 1 Processor -- start 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 1 -- start start 2026-02-17T05:56:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 10 monclient: init 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f97f37fe640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f97f4074dc0 0x7f97f415f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:56132/0 (socket says 10.20.193.16:56132) 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f97f37fe640 1 -- 10.20.193.16:0/2026972037 learned_addr learned my addr 10.20.193.16:0/2026972037 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f1fb2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f1fb2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f1fb2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 10 monclient: _add_conns ranks=[2,1,4,5,0,3] 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f30e8155e00 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f30e81561e0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f30e815fa10 0x7f30e80ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 10 monclient(hunting): picked mon.noname-b con 0x7f30e815fa10 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30e80c96d0 con 0x7f30e81561e0 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30e8074860 con 0x7f30e8155e00 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f30e80c9940 con 0x7f30e815fa10 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f30ef9b4640 10 monclient(hunting): authenticate will time out at 1059.691630s 2026-02-17T05:56:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 10 monclient: _add_conns ranks=[0,5,1,4,2,3] 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0210155e00 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f8361ed7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f835c074dc0 0x7f835c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f02101561e0 0x7f02100ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 10 monclient(hunting): picked mon.noname-c con 0x7f02101561e0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0210074dc0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02100c96d0 con 0x7f0210155e00 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0210074860 con 0x7f0210074dc0 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f02100c9940 con 0x7f02101561e0 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.122+0000 7f021586f640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f021586f640 10 monclient(hunting): authenticate will time out at 1059.692014s 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f73f7fff640 1 --2- 10.20.193.16:0/3590427467 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f74001561e0 0x7f74000cc1a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f74000693f0 tx=0x7f73e802eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5463365329a40ea server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7ffba3096640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7ffba3096640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7ffba3096640 10 monclient: monmap: 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.124072+0000 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.124072+0000 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 <== mon.2 v1:10.20.193.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2296298325 0 0) 0x7f1fa0003620 con 0x7f1fb414ec50 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 >> v2:10.20.193.128:3300/0 conn(0x7f1fb414e870 msgr2=0x7f1fb40c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 --2- 10.20.193.16:0/3654238760 >> v2:10.20.193.128:3300/0 conn(0x7f1fb414e870 0x7f1fb40c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 >> v1:10.20.193.128:6789/0 conn(0x7f1fb40c84b0 legacy=0x7f1fb4158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f8361ed7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f835c074dc0 0x7f835c0cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57000/0 (socket says 10.20.193.16:57000) 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f8361ed7640 1 -- 10.20.193.16:0/2935238851 learned_addr learned my addr 10.20.193.16:0/2935238851 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f8361ed7640 10 monclient(hunting): get_auth_request con 0x7f835c074dc0 auth_method 0 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f8361ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f8361ed7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f8361ed7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.148:6789/0 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 --> v1:10.20.193.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb40c8ef0 con 0x7f1fb414ec50 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 <== mon.2 v1:10.20.193.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f1f9c002f00 con 0x7f1fb414ec50 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient: dump: 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] but i'm connected to v1:10.20.193.148:6789/0, reconnecting 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 >> v1:10.20.193.148:6789/0 conn(0x7f1fb414ec50 legacy=0x7f1fb415b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 --2- 10.20.193.16:0/3654238760 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1fa0003ba0 0x7f1fa0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient(hunting): picked mon.b con 0x7f1fa0003ba0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 --2- 10.20.193.16:0/3654238760 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1fa0004900 0x7f1fa0004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient(hunting): picked mon.c con 0x7f1fa0004900 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 --2- 10.20.193.16:0/3654238760 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fa0005310 0x7f1fa000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient(hunting): picked mon.a con 0x7f1fa0005310 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb40c9820 con 0x7f1fa0005310 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa0010750 con 0x7f1fa0003ba0 2026-02-17T05:56:47.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa0010950 con 0x7f1fa0004900 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f1fb1573640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7ffba3096640 1 Processor -- start 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f97f27fc640 1 --2- 10.20.193.16:0/2026972037 >> v2:10.20.193.148:3300/0 conn(0x7f97f41561e0 0x7f97f40ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7ffba3096640 1 -- start start 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7f19550ef640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f193c0052c0 0x7f193c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.123+0000 7ffba3096640 10 monclient: init 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 10 monclient: _add_conns ranks=[1,2,4,0,5,3] 2026-02-17T05:56:47.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ffb9c0c6fc0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ffb9c0c73a0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7ffb9c14fec0 0x7ffb9c150310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 10 monclient(hunting): picked mon.noname-b con 0x7ffb9c14fec0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9c0c5820 con 0x7ffb9c0c6fc0 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9c15a810 con 0x7ffb9c0c73a0 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7ffb9c0c4780 con 0x7ffb9c14fec0 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7ffba3096640 10 monclient(hunting): authenticate will time out at 1059.693355s 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7f8361ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7f8361ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.124+0000 7f8361ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7f97f27fc640 10 monclient(hunting): get_auth_request con 0x7f97f41561e0 auth_method 0 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7f97f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7f97f27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7f97f27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7fe910888640 10 monclient(hunting): handle_auth_done global_id 4422 payload 824 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7fe910888640 1 -- 10.20.193.16:0/161933326 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe90c15d360 msgr2=0x7fe90c149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7fe910888640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe90c15d360 0x7fe90c149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7fe910888640 1 -- 10.20.193.16:0/161933326 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe90c155fa0 msgr2=0x7fe90c148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7fe910888640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe90c155fa0 0x7fe90c148430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7fe910888640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7fe910888640 1 monclient: found mon.noname-a 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7fe910888640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7fe910888640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7fe910888640 1 -- 10.20.193.16:0/161933326 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c1600c0 con 0x7fe90c156380 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.125+0000 7f83616d6640 1 --2- 10.20.193.16:0/2935238851 >> v2:10.20.193.128:3300/0 conn(0x7f835c155e00 0x7f835c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f83616d6640 10 monclient(hunting): get_auth_request con 0x7f835c155e00 auth_method 0 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f83616d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f83616d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f83616d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f8361ed7640 10 monclient(hunting): handle_auth_done global_id 4514 payload 274 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19550ef640 10 monclient(hunting): get_auth_request con 0x7f193c0052c0 auth_method 0 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19550ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19550ef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19550ef640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f8361ed7640 1 -- 10.20.193.16:0/2935238851 >> v2:10.20.193.128:3300/0 conn(0x7f835c155e00 msgr2=0x7f835c0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f8361ed7640 1 --2- 10.20.193.16:0/2935238851 >> v2:10.20.193.128:3300/0 conn(0x7f835c155e00 0x7f835c0ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f8361ed7640 1 -- 10.20.193.16:0/2935238851 >> v1:10.20.193.16:6789/0 conn(0x7f835c1561e0 legacy=0x7f835c074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f8361ed7640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f8361ed7640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f8361ed7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f8361ed7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f8361ed7640 1 -- 10.20.193.16:0/2935238851 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c0c9c20 con 0x7f835c074dc0 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19548ee640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f193c003ba0 0x7f193c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19550ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19550ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19550ef640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19548ee640 10 monclient(hunting): get_auth_request con 0x7f193c003ba0 auth_method 0 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19548ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19548ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f19548ee640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f97f27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f97f27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.126+0000 7f97f27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f1947fff640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c004900 0x7f193c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19548ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19548ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19548ee640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7fed7bcb7640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7fed7bcb7640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7fed7bcb7640 10 monclient: monmap: 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.128331+0000 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.128331+0000 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.144 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f479167e640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f479167e640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f479167e640 10 monclient: monmap: 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.128250+0000 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.128250+0000 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 10 monclient(hunting): handle_auth_done global_id 4416 payload 293 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 1 -- 10.20.193.16:0/4037480458 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c004900 msgr2=0x7f193c004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c004900 0x7f193c004d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 1 -- 10.20.193.16:0/4037480458 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f193c003ba0 msgr2=0x7f193c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f193c003ba0 0x7f193c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 1 monclient: found mon.a 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 1 -- 10.20.193.16:0/4037480458 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19500c9c20 con 0x7f193c0052c0 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f1956b79640 5 monclient: authenticate success, global_id 4416 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f1956b79640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7fed7bcb7640 1 Processor -- start 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f479167e640 1 Processor -- start 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7fed7bcb7640 1 -- start start 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7fed7bcb7640 10 monclient: init 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f479167e640 1 -- start start 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f479167e640 10 monclient: init 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f19550ef640 1 -- 10.20.193.16:0/4037480458 reap_dead start 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.127+0000 7f8361ed7640 1 --2- 10.20.193.16:0/2935238851 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c074dc0 0x7f835c0cc1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f835c06cbb0 tx=0x7f835802f270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec01fc4f3ced564 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fd02146f640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd01c0c8050 0x7fd01c1f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fe910888640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe90c156380 0x7fe90c1495a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe90000c730 tx=0x7fe90000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fe8f9ffb640 10 client.?.objecter ms_handle_connect 0x7fe90c156380 2026-02-17T05:56:47.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fe8f9ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 10 monclient: _add_conns ranks=[4,3,1,5,0,2] 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f478c155e00 0x7f478c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 10 monclient(hunting): picked mon.noname-b con 0x7f478c155e00 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f478c1561e0 0x7f478c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 10 monclient(hunting): picked mon.noname-a con 0x7f478c1561e0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f478c074dc0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f478c0c96d0 con 0x7f478c074dc0 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f478c074860 con 0x7f478c1561e0 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f478c0c9940 con 0x7f478c155e00 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f479167e640 10 monclient(hunting): authenticate will time out at 1059.697478s 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f46fe751640 10 monclient: get_monmap_and_config 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f46fe751640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f46fe751640 10 monclient: monmap: 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:47.129294+0000 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:47.129294+0000 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:47.147 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 10 monclient: _add_conns ranks=[3,2,0,5,4,1] 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fed74155e00 0x7fed740ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 10 monclient(hunting): picked mon.noname-a con 0x7fed74155e00 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fed741561e0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fed74074dc0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed740c96d0 con 0x7fed74074dc0 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed74074860 con 0x7fed741561e0 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fed740c9940 con 0x7fed74155e00 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7fed7bcb7640 10 monclient(hunting): authenticate will time out at 1059.697619s 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f46fe751640 1 Processor -- start 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f46fe751640 1 -- start start 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f46fe751640 10 monclient: init 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f30ed729640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f30e8155e00 0x7f30e8074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46532/0 (socket says 10.20.193.16:46532) 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.128+0000 7f30ed729640 1 -- 10.20.193.16:0/1979355478 learned_addr learned my addr 10.20.193.16:0/1979355478 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7faa669c2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa6015d360 0x7faa601f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7faa669c2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa6015d360 0x7faa601f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50332/0 (socket says 10.20.193.16:50332) 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7faa669c2640 1 -- 10.20.193.16:0/1476379032 learned_addr learned my addr 10.20.193.16:0/1476379032 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7faa669c2640 10 monclient(hunting): get_auth_request con 0x7faa6015d360 auth_method 0 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7faa669c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7faa669c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7faa669c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 10 monclient: _add_conns ranks=[5,3,1,2,4,0] 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f46f8155e00 0x7f46f80ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 10 monclient(hunting): picked mon.noname-c con 0x7f46f8155e00 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f46f81561e0 0x7f46f80cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7fd02146f640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd01c0c8050 0x7fd01c1f4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59810/0 (socket says 10.20.193.16:59810) 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 10 monclient(hunting): picked mon.noname-a con 0x7f46f81561e0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f46f8074dc0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f80c96d0 con 0x7f46f8074dc0 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f46f8074860 con 0x7f46f81561e0 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7fd02146f640 1 -- 10.20.193.16:0/3225517002 learned_addr learned my addr 10.20.193.16:0/3225517002 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7fd02146f640 10 monclient(hunting): get_auth_request con 0x7fd01c0c8050 auth_method 0 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7fd02146f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7fd02146f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7fd02146f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f46f80c9940 con 0x7f46f8155e00 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f46fe751640 10 monclient(hunting): authenticate will time out at 1059.698495s 2026-02-17T05:56:47.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7f19550ef640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f193c0052c0 0x7f193c00bac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f194c009fc0 tx=0x7f194c02f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b7add09d0b4a30 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.129+0000 7fd021c70640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd01c14e860 0x7fd01c0701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.130+0000 7faa669c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.130+0000 7faa669c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.130+0000 7faa669c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.130+0000 7fbfbdd74640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fbfc0156380 0x7fbfc00740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.130+0000 7fd021c70640 10 monclient(hunting): get_auth_request con 0x7fd01c14e860 auth_method 0 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.130+0000 7fd021c70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.130+0000 7fd021c70640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.130+0000 7fd021c70640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 1 -- 10.20.193.16:0/3590427467 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e803c070 con 0x7f74001561e0 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 10 monclient: dump: 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.150 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 1 -- 10.20.193.16:0/3590427467 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e802fb60 con 0x7f74001561e0 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 1 -- 10.20.193.16:0/3590427467 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8037440 con 0x7f74001561e0 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f73f77fe640 10 monclient: dump: 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 5 monclient: authenticate success, global_id 4511 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 4 set_mon_vals no callback set 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fd021c70640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fd021c70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7faa669c2640 10 monclient(hunting): handle_auth_done global_id 4477 payload 824 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7faa669c2640 1 -- 10.20.193.16:0/1476379032 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faa60155fa0 msgr2=0x7faa600ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7faa669c2640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faa60155fa0 0x7faa600ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fd021c70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7faa669c2640 1 -- 10.20.193.16:0/1476379032 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7faa60156380 msgr2=0x7faa6014bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7faa669c2640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7faa60156380 0x7faa6014bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7faa669c2640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7faa669c2640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7faa669c2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7faa669c2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7faa669c2640 1 -- 10.20.193.16:0/1476379032 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa600c8b20 con 0x7faa6015d360 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fd02146f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fd02146f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fd02146f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 10 monclient: shutdown 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 1 -- 10.20.193.16:0/3590427467 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f74001561e0 msgr2=0x7f74000cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 1 --2- 10.20.193.16:0/3590427467 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f74001561e0 0x7f74000cc1a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f74000693f0 tx=0x7f73e802eef0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 1 -- 10.20.193.16:0/3590427467 shutdown_connections 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 1 --2- 10.20.193.16:0/3590427467 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f74001561e0 0x7f74000cc1a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 1 --2- 10.20.193.16:0/3590427467 >> v2:10.20.193.128:3300/0 conn(0x7f7400155e00 0x7f74000ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fbfbdd74640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fbfc0156380 0x7fbfc00740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50316/0 (socket says 10.20.193.16:50316) 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fbfbdd74640 1 -- 10.20.193.16:0/2833506156 learned_addr learned my addr 10.20.193.16:0/2833506156 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fbfbdd74640 10 monclient(hunting): get_auth_request con 0x7fbfc0156380 auth_method 0 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 1 -- 10.20.193.16:0/3590427467 >> 10.20.193.16:0/3590427467 conn(0x7f74000ce6c0 msgr2=0x7f74000cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fbfbdd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fbfbdd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7fbfbdd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 1 -- 10.20.193.16:0/3590427467 shutdown_connections 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.131+0000 7f7406d49640 1 -- 10.20.193.16:0/3590427467 wait complete. 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7f7406d49640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7f7406d49640 10 monclient: monmap: 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7f7406d49640 1 librados: starting msgr at 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7f7406d49640 1 librados: starting objecter 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7f7406d49640 1 Processor -- start 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7f7406d49640 1 -- start start 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7f7406d49640 1 librados: setting wanted keys 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7f7406d49640 1 librados: calling monclient init 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7f7406d49640 10 monclient: init 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7fbfbdd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7fbfbdd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.132+0000 7fbfbdd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7400074dc0 0x7f74001500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 10 monclient(hunting): picked mon.noname-a con 0x7f7400074dc0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7400155e00 0x7f74001e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 10 monclient(hunting): picked mon.noname-c con 0x7f7400155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f74001561e0 0x7f740013eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 10 monclient(hunting): picked mon.noname-b con 0x7f74001561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74000c9360 con 0x7f7400074dc0 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f74000ca1e0 con 0x7f74001561e0 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f74000c8c20 con 0x7f7400155e00 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7faa669c2640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa6015d360 0x7faa601f4c90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00cbb0 tx=0x7faa5c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7f7406d49640 10 monclient(hunting): authenticate will time out at 1059.702225s 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7faa46ffd640 10 client.?.objecter ms_handle_connect 0x7faa6015d360 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7faa46ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7fd02146f640 10 monclient(hunting): handle_auth_done global_id 4428 payload 824 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7fd02146f640 1 -- 10.20.193.16:0/3225517002 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd01c14e860 msgr2=0x7fd01c0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7fd02146f640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd01c14e860 0x7fd01c0701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7fd02146f640 1 -- 10.20.193.16:0/3225517002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd01c14e480 msgr2=0x7fd01c153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7fd02146f640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd01c14e480 0x7fd01c153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7fd02146f640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7fd02146f640 1 monclient: found mon.noname-a 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7fd02146f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7fd02146f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.133+0000 7fd02146f640 1 -- 10.20.193.16:0/3225517002 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c152840 con 0x7fd01c0c8050 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7fbfbdd74640 10 monclient(hunting): handle_auth_done global_id 4480 payload 274 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7fbfbdd74640 1 -- 10.20.193.16:0/2833506156 >> v1:10.20.193.128:6789/0 conn(0x7fbfc0155fa0 legacy=0x7fbfc00cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7fbfbdd74640 1 -- 10.20.193.16:0/2833506156 >> v1:10.20.193.16:6789/0 conn(0x7fbfc00745e0 legacy=0x7fbfc015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7fbfbdd74640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7fbfbdd74640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7fbfbdd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7fbfbdd74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7fbfbdd74640 1 -- 10.20.193.16:0/2833506156 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc00cd0a0 con 0x7fbfc0156380 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7f97f1ffb640 1 -- 10.20.193.16:0/2026972037 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398039643 0 0) 0x7f97e8002920 con 0x7f97f4155e00 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7f97f1ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7f97f1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7f97f1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7f97f1ffb640 10 monclient(hunting): my global_id is 4523 2026-02-17T05:56:47.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7f97f1ffb640 1 -- 10.20.193.16:0/2026972037 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97e0003620 con 0x7f97f4155e00 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.134+0000 7ffba160c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7ffb9c14fec0 0x7ffb9c150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.135+0000 7ffba160c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7ffb9c14fec0 0x7ffb9c150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50364/0 (socket says 10.20.193.16:50364) 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.135+0000 7ffba160c640 1 -- 10.20.193.16:0/771121767 learned_addr learned my addr 10.20.193.16:0/771121767 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.135+0000 7ffba160c640 10 monclient(hunting): get_auth_request con 0x7ffb9c14fec0 auth_method 0 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.135+0000 7ffba160c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.135+0000 7ffba160c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.135+0000 7ffba160c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.135+0000 7f30edf2a640 1 --2- 10.20.193.16:0/1979355478 >> v2:10.20.193.128:3300/0 conn(0x7f30e815fa10 0x7f30e80ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.135+0000 7fd02146f640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd01c0c8050 0x7fd01c1f4cd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00b5a0 tx=0x7fd00c00ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.135+0000 7fd009ffb640 10 client.?.objecter ms_handle_connect 0x7fd01c0c8050 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.135+0000 7fd009ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.136+0000 7ffba160c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.136+0000 7ffba160c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.136+0000 7ffba160c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.136+0000 7f30edf2a640 10 monclient(hunting): get_auth_request con 0x7f30e815fa10 auth_method 0 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.136+0000 7f30edf2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.136+0000 7f30edf2a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.136+0000 7f30edf2a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c03a020 con 0x7f193c0052c0 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f19477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f19477fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f19477fe640 10 monclient: dump: 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194c003cd0 con 0x7f193c0052c0 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f19477fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 4 set_mon_vals no callback set 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f19477fe640 1 -- 10.20.193.16:0/4037480458 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c009c10 con 0x7f193c0052c0 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f19477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f19477fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f19477fe640 10 monclient: dump: 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.157 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 10 monclient: shutdown 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 1 -- 10.20.193.16:0/4037480458 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f193c0052c0 msgr2=0x7f193c00bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f193c0052c0 0x7f193c00bac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f194c009fc0 tx=0x7f194c02f5e0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 1 -- 10.20.193.16:0/4037480458 shutdown_connections 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 1 --2- 10.20.193.16:0/4037480458 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f193c0052c0 0x7f193c00bac0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 1 -- 10.20.193.16:0/4037480458 >> 10.20.193.16:0/4037480458 conn(0x7f19500ce6c0 msgr2=0x7f19500cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 1 -- 10.20.193.16:0/4037480458 shutdown_connections 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.137+0000 7f1956b79640 1 -- 10.20.193.16:0/4037480458 wait complete. 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 10 monclient: monmap: 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 1 librados: starting msgr at 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 1 librados: starting objecter 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 1 Processor -- start 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 1 -- start start 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 1 librados: setting wanted keys 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 1 librados: calling monclient init 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 10 monclient: init 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f020e7fc640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f02101561e0 0x7f02100ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7ffba160c640 10 monclient(hunting): handle_auth_done global_id 4486 payload 274 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7ffba160c640 1 -- 10.20.193.16:0/771121767 >> v1:10.20.193.148:6789/0 conn(0x7ffb9c0c73a0 legacy=0x7ffb9c0c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7ffba160c640 1 -- 10.20.193.16:0/771121767 >> v1:10.20.193.128:6789/0 conn(0x7ffb9c0c6fc0 legacy=0x7ffb9c0747c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7ffba160c640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7ffba160c640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7ffba160c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7ffba160c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7ffba160c640 1 -- 10.20.193.16:0/771121767 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c0c4a60 con 0x7ffb9c14fec0 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f30edf2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f30edf2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f30edf2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c0052c0 0x7f19501f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 10 monclient(hunting): picked mon.noname-c con 0x7f193c0052c0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f834bfff640 1 -- 10.20.193.16:0/2935238851 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835802fcd0 con 0x7f835c074dc0 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f834bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f1956b79640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1950142130 0x7f19501f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f834bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.138+0000 7f834bfff640 10 monclient: dump: 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.159 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f834bfff640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f834bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f834bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f834bfff640 1 -- 10.20.193.16:0/2935238851 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83580373f0 con 0x7f835c074dc0 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 5 monclient: authenticate success, global_id 4514 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f834bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f834bfff640 1 -- 10.20.193.16:0/2935238851 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580377c0 con 0x7f835c074dc0 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 4 set_mon_vals no callback set 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f834bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f834bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f834bfff640 10 monclient: dump: 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1fb2d76640 1 --2- 10.20.193.16:0/3654238760 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fa0005310 0x7f1fa000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 10 monclient(hunting): picked mon.noname-b con 0x7f1950142130 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f195013e2f0 0x7f19500c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 10 monclient(hunting): picked mon.noname-a con 0x7f195013e2f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19500c9360 con 0x7f195013e2f0 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f19500ca1e0 con 0x7f1950142130 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f19500ca700 con 0x7f193c0052c0 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1956b79640 10 monclient(hunting): authenticate will time out at 1059.708089s 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f020e7fc640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f02101561e0 0x7f02100ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57036/0 (socket says 10.20.193.16:57036) 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f020e7fc640 1 -- 10.20.193.16:0/170626629 learned_addr learned my addr 10.20.193.16:0/170626629 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f020e7fc640 10 monclient(hunting): get_auth_request con 0x7f02101561e0 auth_method 0 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f020e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f020e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f020e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1fb2d76640 10 monclient(hunting): get_auth_request con 0x7f1fa0005310 auth_method 0 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1fb2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1fb2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1fb2d76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1fb2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1fb2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f1fb2d76640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 10 monclient: shutdown 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 1 -- 10.20.193.16:0/2935238851 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c074dc0 msgr2=0x7f835c0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 1 --2- 10.20.193.16:0/2935238851 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c074dc0 0x7f835c0cc1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f835c06cbb0 tx=0x7f835802f270 comp rx=0 tx=0).stop 2026-02-17T05:56:47.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 1 -- 10.20.193.16:0/2935238851 shutdown_connections 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 1 --2- 10.20.193.16:0/2935238851 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c074dc0 0x7f835c0cc1a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 1 --2- 10.20.193.16:0/2935238851 >> v2:10.20.193.128:3300/0 conn(0x7f835c155e00 0x7f835c0ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 1 -- 10.20.193.16:0/2935238851 >> 10.20.193.16:0/2935238851 conn(0x7f835c0ce6c0 msgr2=0x7f835c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 1 -- 10.20.193.16:0/2935238851 shutdown_connections 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f8363961640 1 -- 10.20.193.16:0/2935238851 wait complete. 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f97f27fc640 10 monclient(hunting): handle_auth_done global_id 4517 payload 274 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f97f27fc640 1 -- 10.20.193.16:0/2026972037 >> v1:10.20.193.148:6789/0 conn(0x7f97f4155e00 legacy=0x7f97f4074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f97f27fc640 1 -- 10.20.193.16:0/2026972037 >> v1:10.20.193.16:6789/0 conn(0x7f97f4074dc0 legacy=0x7f97f415f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f97f27fc640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f97f27fc640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f97f27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f97f27fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f97f27fc640 1 -- 10.20.193.16:0/2026972037 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f40c9c20 con 0x7f97f41561e0 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.139+0000 7f46f7fff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f46f8155e00 0x7f46f80ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f8363961640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f8363961640 10 monclient: monmap: 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.162 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 10 monclient(hunting): handle_auth_done global_id 4505 payload 293 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f30edf2a640 10 monclient(hunting): handle_auth_done global_id 4489 payload 274 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f30edf2a640 1 -- 10.20.193.16:0/1979355478 >> v1:10.20.193.148:6789/0 conn(0x7f30e8155e00 legacy=0x7f30e8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f30edf2a640 1 -- 10.20.193.16:0/1979355478 >> v1:10.20.193.128:6789/0 conn(0x7f30e81561e0 legacy=0x7f30e815f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f30edf2a640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f30edf2a640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f30edf2a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f30edf2a640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f30edf2a640 1 -- 10.20.193.16:0/1979355478 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e80c9c20 con 0x7f30e815fa10 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 1 -- 10.20.193.16:0/3654238760 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1fa0004900 msgr2=0x7f1fa0004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 1 --2- 10.20.193.16:0/3654238760 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1fa0004900 0x7f1fa0004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 1 -- 10.20.193.16:0/3654238760 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1fa0003ba0 msgr2=0x7f1fa0003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 1 --2- 10.20.193.16:0/3654238760 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1fa0003ba0 0x7f1fa0003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 1 monclient: found mon.a 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 1 -- 10.20.193.16:0/3654238760 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa0003620 con 0x7f1fa0005310 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb8e05640 5 monclient: authenticate success, global_id 4505 2026-02-17T05:56:47.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb8e05640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f8363961640 1 librados: starting msgr at 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f8363961640 1 librados: starting objecter 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f8363961640 1 Processor -- start 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f8363961640 1 -- start start 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f8363961640 1 librados: setting wanted keys 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f8363961640 1 librados: calling monclient init 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f8363961640 10 monclient: init 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f020e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f020e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f020e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f1fb2d76640 1 -- 10.20.193.16:0/3654238760 reap_dead start 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.140+0000 7f46f7fff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f46f8155e00 0x7f46f80ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57050/0 (socket says 10.20.193.16:57050) 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f46f7fff640 1 -- 10.20.193.16:0/2691600672 learned_addr learned my addr 10.20.193.16:0/2691600672 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f46f7fff640 10 monclient(hunting): get_auth_request con 0x7f46f8155e00 auth_method 0 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f46f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f46f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f46f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f835c074dc0 0x7f835c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 10 monclient(hunting): picked mon.noname-b con 0x7f835c074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c155e00 0x7f835c143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 10 monclient(hunting): picked mon.noname-c con 0x7f835c155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f835c1561e0 0x7f835c14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 10 monclient(hunting): picked mon.noname-a con 0x7f835c1561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0c9360 con 0x7f835c1561e0 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0ca1e0 con 0x7f835c074dc0 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0c8c20 con 0x7f835c155e00 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7f8363961640 10 monclient(hunting): authenticate will time out at 1059.710088s 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 1 -- 10.20.193.16:0/161933326 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9000052d0 con 0x7fe90c156380 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 10 monclient: dump: 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.165 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 1 -- 10.20.193.16:0/161933326 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe900005470 con 0x7fe90c156380 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 1 -- 10.20.193.16:0/161933326 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9000038d0 con 0x7fe90c156380 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe913314640 5 monclient: authenticate success, global_id 4422 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe913314640 10 monclient: _renew_subs 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8fb7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe913314640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe913314640 1 -- 10.20.193.16:0/161933326 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90c1f6af0 con 0x7fe90c156380 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe8f9ffb640 10 monclient: dump: 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe913314640 10 client.4422.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe913314640 10 monclient: _renew_subs 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe913314640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe913314640 1 -- 10.20.193.16:0/161933326 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90c1f70b0 con 0x7fe90c156380 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.141+0000 7fe913314640 1 librados: init done 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f97f27fc640 1 --2- 10.20.193.16:0/2026972037 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f41561e0 0x7f97f40ca700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0098a0 tx=0x7f97dc02ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc241ff61f035e7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7fbfbdd74640 1 --2- 10.20.193.16:0/2833506156 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbfc0156380 0x7fbfc00740a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009970 tx=0x7fbfa802efb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e28a60824b662616 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7fe913314640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7fe913314640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7fe913314640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f30edf2a640 1 --2- 10.20.193.16:0/1979355478 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e815fa10 0x7f30e80ca700 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009d90 tx=0x7f30e402f190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a33f10e875e49159 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7fe913314640 1 -- 10.20.193.16:0/161933326 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90c0d2970 con 0x7fe90c156380 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f1fb2d76640 1 --2- 10.20.193.16:0/3654238760 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fa0005310 0x7f1fa000bac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4004a20 tx=0x7f1fa402f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2b450566d54d3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f020e7fc640 10 monclient(hunting): handle_auth_done global_id 4529 payload 274 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f020e7fc640 1 -- 10.20.193.16:0/170626629 >> v1:10.20.193.128:6789/0 conn(0x7f0210074dc0 legacy=0x7f021015f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f020e7fc640 1 -- 10.20.193.16:0/170626629 >> v1:10.20.193.16:6789/0 conn(0x7f0210155e00 legacy=0x7f0210074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f46f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f46f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f46f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f020e7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f020e7fc640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f020e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f020e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.142+0000 7f020e7fc640 1 -- 10.20.193.16:0/170626629 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02100c9c20 con 0x7f02101561e0 2026-02-17T05:56:47.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7f46f77fe640 1 --2- 10.20.193.16:0/2691600672 >> v2:10.20.193.16:3300/0 conn(0x7f46f81561e0 0x7f46f80cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 1 -- 10.20.193.16:0/2973515293 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8403c070 con 0x7efc8c155fa0 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 10 monclient: dump: 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 1 -- 10.20.193.16:0/2973515293 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc8402fbc0 con 0x7efc8c155fa0 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 1 -- 10.20.193.16:0/2973515293 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84037440 con 0x7efc8c155fa0 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc948b0640 5 monclient: authenticate success, global_id 4419 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc948b0640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc948b0640 4 set_mon_vals no callback set 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.143+0000 7efc91623640 10 monclient: dump: 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.168 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7ffba160c640 1 --2- 10.20.193.16:0/771121767 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffb9c14fec0 0x7ffb9c150310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009e50 tx=0x7ffb9802f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd6f768ccbe53b43 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 10 monclient: shutdown 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 1 -- 10.20.193.16:0/2973515293 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc8c155fa0 msgr2=0x7efc8c0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 1 --2- 10.20.193.16:0/2973515293 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc8c155fa0 0x7efc8c0cc1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efc8c08e730 tx=0x7efc8402ef90 comp rx=0 tx=0).stop 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7f46f7fff640 10 monclient(hunting): handle_auth_done global_id 4532 payload 274 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7f46f7fff640 1 -- 10.20.193.16:0/2691600672 >> v2:10.20.193.16:3300/0 conn(0x7f46f81561e0 msgr2=0x7f46f80cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7f46f7fff640 1 --2- 10.20.193.16:0/2691600672 >> v2:10.20.193.16:3300/0 conn(0x7f46f81561e0 0x7f46f80cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7f46f7fff640 1 -- 10.20.193.16:0/2691600672 >> v1:10.20.193.128:6789/0 conn(0x7f46f8074dc0 legacy=0x7f46f8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7f46f7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7f46f7fff640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7f46f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7f46f7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7f46f7fff640 1 -- 10.20.193.16:0/2691600672 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f80c9c20 con 0x7f46f8155e00 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 1 -- 10.20.193.16:0/2973515293 shutdown_connections 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 1 --2- 10.20.193.16:0/2973515293 >> v2:10.20.193.128:3300/0 conn(0x7efc8c15d360 0x7efc8c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 1 --2- 10.20.193.16:0/2973515293 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc8c155fa0 0x7efc8c0cc1a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 1 -- 10.20.193.16:0/2973515293 >> 10.20.193.16:0/2973515293 conn(0x7efc8c0ce6c0 msgr2=0x7efc8c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 1 -- 10.20.193.16:0/2973515293 shutdown_connections 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 1 -- 10.20.193.16:0/2973515293 wait complete. 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.144+0000 7efc948b0640 10 monclient: monmap: 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 librados: starting msgr at 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 librados: starting objecter 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 Processor -- start 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 -- start start 2026-02-17T05:56:47.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7f478affd640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f478c155e00 0x7f478c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 librados: setting wanted keys 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 librados: calling monclient init 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 10 monclient: init 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efc8c155fa0 0x7efc8c1f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 10 monclient(hunting): picked mon.noname-c con 0x7efc8c155fa0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc8c156380 0x7efc8c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 10 monclient(hunting): picked mon.noname-a con 0x7efc8c156380 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc8c15d360 0x7efc8c1f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 10 monclient(hunting): picked mon.noname-b con 0x7efc8c15d360 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c0747b0 con 0x7efc8c156380 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c0cd660 con 0x7efc8c15d360 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c15dca0 con 0x7efc8c155fa0 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.145+0000 7efc948b0640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.146+0000 7efc948b0640 10 monclient(hunting): authenticate will time out at 1059.715022s 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.146+0000 7f478affd640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f478c155e00 0x7f478c0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50374/0 (socket says 10.20.193.16:50374) 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.146+0000 7f478affd640 1 -- 10.20.193.16:0/2708375620 learned_addr learned my addr 10.20.193.16:0/2708375620 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.146+0000 7f478affd640 10 monclient(hunting): get_auth_request con 0x7f478c155e00 auth_method 0 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.146+0000 7f478affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.146+0000 7f478affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.146+0000 7f478affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.147+0000 7f020e7fc640 1 --2- 10.20.193.16:0/170626629 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f02101561e0 0x7f02100ca700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009970 tx=0x7f01f802eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=186e25e0ed3970c0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.147+0000 7f478affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.147+0000 7f478affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.147+0000 7f478affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7fed7922b640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fed741561e0 0x7fed74074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46546/0 (socket says 10.20.193.16:46546) 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7fed7922b640 1 -- 10.20.193.16:0/1934827957 learned_addr learned my addr 10.20.193.16:0/1934827957 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7f46f7fff640 1 --2- 10.20.193.16:0/2691600672 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f46f8155e00 0x7f46f80ca700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f46f806ce20 tx=0x7f46e402eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=251e2a992a88587 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7f478affd640 10 monclient(hunting): handle_auth_done global_id 4492 payload 274 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7f478affd640 1 -- 10.20.193.16:0/2708375620 >> v2:10.20.193.16:3300/0 conn(0x7f478c1561e0 msgr2=0x7f478c0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7f478affd640 1 --2- 10.20.193.16:0/2708375620 >> v2:10.20.193.16:3300/0 conn(0x7f478c1561e0 0x7f478c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7f478affd640 1 -- 10.20.193.16:0/2708375620 >> v1:10.20.193.128:6789/0 conn(0x7f478c074dc0 legacy=0x7f478c074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7f478affd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7f478affd640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7f478affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7f478affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.148+0000 7f478affd640 1 -- 10.20.193.16:0/2708375620 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c0c9c20 con 0x7f478c155e00 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.150+0000 7f73f7fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7400155e00 0x7f74001e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7f478affd640 1 --2- 10.20.193.16:0/2708375620 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c155e00 0x7f478c0ca700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f478c06ce20 tx=0x7f478002eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da4093f62267a80 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7f73f7fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7400155e00 0x7f74001e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57052/0 (socket says 10.20.193.16:57052) 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7f73f7fff640 1 -- 10.20.193.16:0/2132433469 learned_addr learned my addr 10.20.193.16:0/2132433469 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7f73f7fff640 10 monclient(hunting): get_auth_request con 0x7f7400155e00 auth_method 0 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7f73f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7f73f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7f73f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 1 -- 10.20.193.16:0/2833506156 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa803c070 con 0x7fbfc0156380 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 10 monclient: dump: 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 1 -- 10.20.193.16:0/2833506156 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa802fbe0 con 0x7fbfc0156380 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 1 -- 10.20.193.16:0/2833506156 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8037440 con 0x7fbfc0156380 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfc4d49640 5 monclient: authenticate success, global_id 4480 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfc4d49640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfc4d49640 4 set_mon_vals no callback set 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.151+0000 7fbfbd573640 10 monclient: dump: 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.173 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 10 monclient: shutdown 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 1 -- 10.20.193.16:0/2833506156 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbfc0156380 msgr2=0x7fbfc00740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 1 --2- 10.20.193.16:0/2833506156 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbfc0156380 0x7fbfc00740a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009970 tx=0x7fbfa802efb0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 1 -- 10.20.193.16:0/2833506156 shutdown_connections 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 1 --2- 10.20.193.16:0/2833506156 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbfc0156380 0x7fbfc00740a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 1 -- 10.20.193.16:0/2833506156 >> 10.20.193.16:0/2833506156 conn(0x7fbfc00ce6c0 msgr2=0x7fbfc00cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 1 -- 10.20.193.16:0/2833506156 shutdown_connections 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 1 -- 10.20.193.16:0/2833506156 wait complete. 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 1 -- 10.20.193.16:0/2026972037 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc03c070 con 0x7f97f41561e0 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 10 monclient: dump: 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 1 -- 10.20.193.16:0/771121767 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9802fd90 con 0x7ffb9c14fec0 2026-02-17T05:56:47.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 10 monclient: dump: 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 1 -- 10.20.193.16:0/2026972037 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97dc02fb70 con 0x7f97f41561e0 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 1 -- 10.20.193.16:0/771121767 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb980373f0 con 0x7ffb9c14fec0 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 1 -- 10.20.193.16:0/2026972037 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc037440 con 0x7f97f41561e0 2026-02-17T05:56:47.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 1 -- 10.20.193.16:0/771121767 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb980377c0 con 0x7ffb9c14fec0 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffb937fe640 10 monclient: dump: 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffba3096640 5 monclient: authenticate success, global_id 4486 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffba3096640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7ffba3096640 4 set_mon_vals no callback set 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f1ffb640 10 monclient: dump: 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f9379640 5 monclient: authenticate success, global_id 4517 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f9379640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7f97f9379640 4 set_mon_vals no callback set 2026-02-17T05:56:47.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7ffba3096640 10 monclient: shutdown 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7f97f9379640 10 monclient: shutdown 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7f97f9379640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7f97f9379640 1 -- 10.20.193.16:0/2026972037 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f41561e0 msgr2=0x7f97f40ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7f97f9379640 1 --2- 10.20.193.16:0/2026972037 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f41561e0 0x7f97f40ca700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0098a0 tx=0x7f97dc02ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7ffba3096640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7ffba3096640 1 -- 10.20.193.16:0/771121767 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffb9c14fec0 msgr2=0x7ffb9c150310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7ffba3096640 1 --2- 10.20.193.16:0/771121767 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffb9c14fec0 0x7ffb9c150310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009e50 tx=0x7ffb9802f330 comp rx=0 tx=0).stop 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.152+0000 7fbfc4d49640 10 monclient: monmap: 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 librados: starting msgr at 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 librados: starting objecter 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 Processor -- start 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 -- start start 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 librados: setting wanted keys 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 librados: calling monclient init 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient: init 2026-02-17T05:56:47.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7f97f9379640 1 -- 10.20.193.16:0/2026972037 shutdown_connections 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7f97f9379640 1 --2- 10.20.193.16:0/2026972037 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f41561e0 0x7f97f40ca700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7f97f9379640 1 -- 10.20.193.16:0/2026972037 >> 10.20.193.16:0/2026972037 conn(0x7f97f40ce6c0 msgr2=0x7f97f40cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7f97f9379640 1 -- 10.20.193.16:0/2026972037 shutdown_connections 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7f74052bf640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f74001561e0 0x7f740013eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbfc00745e0 0x7fbfc00c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient(hunting): picked mon.noname-c con 0x7fbfc00745e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbfc0155fa0 0x7fbfc0149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient(hunting): picked mon.noname-b con 0x7fbfc0155fa0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbfc0156380 0x7fbfc015fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient(hunting): picked mon.noname-a con 0x7fbfc0156380 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc00cd660 con 0x7fbfc0156380 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc00cd9d0 con 0x7fbfc0155fa0 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0074d50 con 0x7fbfc00745e0 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7fbfc4d49640 10 monclient(hunting): authenticate will time out at 1059.722923s 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7ffba3096640 1 -- 10.20.193.16:0/771121767 shutdown_connections 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7ffba3096640 1 --2- 10.20.193.16:0/771121767 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffb9c14fec0 0x7ffb9c150310 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7ffba3096640 1 -- 10.20.193.16:0/771121767 >> 10.20.193.16:0/771121767 conn(0x7ffb9c0cd770 msgr2=0x7ffb9c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.153+0000 7f97f9379640 1 -- 10.20.193.16:0/2026972037 wait complete. 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 1 -- 10.20.193.16:0/771121767 shutdown_connections 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 1 -- 10.20.193.16:0/771121767 wait complete. 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient: monmap: 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 librados: starting msgr at 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 librados: starting objecter 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 Processor -- start 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f74052bf640 10 monclient(hunting): get_auth_request con 0x7f74001561e0 auth_method 0 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f74052bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f74052bf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f74052bf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 10 monclient: monmap: 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 1 librados: starting msgr at 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 1 librados: starting objecter 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 1 Processor -- start 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 1 -- 10.20.193.16:0/1476379032 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c017070 con 0x7faa6015d360 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 10 monclient: dump: 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 1 -- 10.20.193.16:0/1476379032 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa5c00ad70 con 0x7faa6015d360 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 1 -- 10.20.193.16:0/1476379032 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c012640 con 0x7faa6015d360 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa649be640 4 set_mon_vals no callback set 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa46ffd640 10 monclient: dump: 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 1 -- start start 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 1 librados: setting wanted keys 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 1 librados: calling monclient init 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7ffba3096640 10 monclient: init 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 -- start start 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 librados: setting wanted keys 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 librados: calling monclient init 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient: init 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f4074dc0 0x7f97f4073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient(hunting): picked mon.noname-c con 0x7f97f4074dc0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97f4155e00 0x7f97f414cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient(hunting): picked mon.noname-a con 0x7f97f4155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97f41561e0 0x7f97f414d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient(hunting): picked mon.noname-b con 0x7f97f41561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97f40c9360 con 0x7f97f4155e00 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa6844c640 5 monclient: authenticate success, global_id 4477 2026-02-17T05:56:47.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f97f40ca1e0 con 0x7f97f41561e0 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97f415fa10 con 0x7f97f4074dc0 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7f97f9379640 10 monclient(hunting): authenticate will time out at 1059.723887s 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa6844c640 10 monclient: _renew_subs 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa6844c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.154+0000 7faa6844c640 1 -- 10.20.193.16:0/1476379032 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa600c8de0 con 0x7faa6015d360 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7faa6844c640 10 client.4477.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7faa6844c640 10 monclient: _renew_subs 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7faa6844c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7faa6844c640 1 -- 10.20.193.16:0/1476379032 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa600c93a0 con 0x7faa6015d360 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7faa6844c640 1 librados: init done 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f73f7fff640 10 monclient(hunting): handle_auth_done global_id 4535 payload 824 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f73f7fff640 1 -- 10.20.193.16:0/2132433469 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f74001561e0 msgr2=0x7f740013eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f73f7fff640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f74001561e0 0x7f740013eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f73f7fff640 1 -- 10.20.193.16:0/2132433469 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7400074dc0 msgr2=0x7f74001500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f73f7fff640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7400074dc0 0x7f74001500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f73f7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f73f7fff640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f73f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f73f7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f73f7fff640 1 -- 10.20.193.16:0/2132433469 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74001ea2f0 con 0x7f7400155e00 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffb9c0c6fc0 0x7ffb9c158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 10 monclient(hunting): picked mon.noname-b con 0x7ffb9c0c6fc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffb9c0c73a0 0x7ffb9c14c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 10 monclient(hunting): picked mon.noname-a con 0x7ffb9c0c73a0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffb9c14fec0 0x7ffb9c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 10 monclient(hunting): picked mon.noname-c con 0x7ffb9c14fec0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c0c5300 con 0x7ffb9c0c73a0 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c0c5020 con 0x7ffb9c0c6fc0 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c150850 con 0x7ffb9c14fec0 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7efc92e26640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc8c15d360 0x7efc8c1f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7ffba3096640 10 monclient(hunting): authenticate will time out at 1059.724624s 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 1 -- 10.20.193.16:0/1979355478 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e403c070 con 0x7f30e815fa10 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 10 monclient: dump: 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.183 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 1 -- 10.20.193.16:0/1979355478 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e402fe10 con 0x7f30e815fa10 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 1 -- 10.20.193.16:0/1979355478 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4037660 con 0x7f30e815fa10 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30d7fff640 10 monclient: dump: 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30ef9b4640 5 monclient: authenticate success, global_id 4489 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30ef9b4640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7f30ef9b4640 4 set_mon_vals no callback set 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7faa6844c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7faa6844c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7faa6844c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.155+0000 7faa6844c640 1 -- 10.20.193.16:0/1476379032 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa60076e00 con 0x7faa6015d360 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 1 -- 10.20.193.16:0/170626629 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f803c070 con 0x7f02101561e0 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 10 monclient: dump: 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 1 -- 10.20.193.16:0/170626629 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f802fb60 con 0x7f02101561e0 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 5 monclient: authenticate success, global_id 4529 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 1 -- 10.20.193.16:0/170626629 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8037440 con 0x7f02101561e0 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f020dffb640 10 monclient: dump: 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.185 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 4 set_mon_vals no callback set 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 10 monclient: shutdown 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 1 -- 10.20.193.16:0/1979355478 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e815fa10 msgr2=0x7f30e80ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 1 --2- 10.20.193.16:0/1979355478 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e815fa10 0x7f30e80ca700 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009d90 tx=0x7f30e402f190 comp rx=0 tx=0).stop 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 1 -- 10.20.193.16:0/1979355478 shutdown_connections 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 1 --2- 10.20.193.16:0/1979355478 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e815fa10 0x7f30e80ca700 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 1 -- 10.20.193.16:0/1979355478 >> 10.20.193.16:0/1979355478 conn(0x7f30e80ce6c0 msgr2=0x7f30e80cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 1 -- 10.20.193.16:0/1979355478 shutdown_connections 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 1 -- 10.20.193.16:0/1979355478 wait complete. 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7efc92e26640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc8c15d360 0x7efc8c1f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50400/0 (socket says 10.20.193.16:50400) 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7efc92e26640 1 -- 10.20.193.16:0/1090734837 learned_addr learned my addr 10.20.193.16:0/1090734837 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7efc92e26640 10 monclient(hunting): get_auth_request con 0x7efc8c15d360 auth_method 0 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7efc92e26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7efc92e26640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7efc92e26640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 10 monclient: shutdown 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 1 -- 10.20.193.16:0/170626629 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f02101561e0 msgr2=0x7f02100ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 1 --2- 10.20.193.16:0/170626629 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f02101561e0 0x7f02100ca700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009970 tx=0x7f01f802eef0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f1947fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1950142130 0x7f19501f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 1 -- 10.20.193.16:0/170626629 shutdown_connections 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 1 --2- 10.20.193.16:0/170626629 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f02101561e0 0x7f02100ca700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 1 -- 10.20.193.16:0/170626629 >> 10.20.193.16:0/170626629 conn(0x7f02100ce6c0 msgr2=0x7f02100cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 1 -- 10.20.193.16:0/170626629 shutdown_connections 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f021586f640 1 -- 10.20.193.16:0/170626629 wait complete. 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 10 monclient: monmap: 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 1 librados: starting msgr at 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 1 librados: starting objecter 2026-02-17T05:56:47.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.156+0000 7f30ef9b4640 1 Processor -- start 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 1 -- start start 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 1 librados: setting wanted keys 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 1 librados: calling monclient init 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient: init 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 10 monclient: monmap: 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 librados: starting msgr at 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 librados: starting objecter 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 Processor -- start 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 -- start start 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 librados: setting wanted keys 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 librados: calling monclient init 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 10 monclient: init 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f73f7fff640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7400155e00 0x7f74001e9db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009a40 tx=0x7f73e80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f73f57fa640 10 client.?.objecter ms_handle_connect 0x7f7400155e00 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f73f57fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30e8155e00 0x7f30e81fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient(hunting): picked mon.noname-c con 0x7f30e8155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e81561e0 0x7f30e814c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient(hunting): picked mon.noname-b con 0x7f30e81561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e815fa10 0x7f30e8147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient(hunting): picked mon.noname-a con 0x7f30e815fa10 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30e80c9360 con 0x7f30e815fa10 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f30e80ca1e0 con 0x7f30e81561e0 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30e8074dc0 con 0x7f30e8155e00 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f30ef9b4640 10 monclient(hunting): authenticate will time out at 1059.726549s 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0210074dc0 0x7f021014e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 10 monclient(hunting): picked mon.noname-b con 0x7f0210074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0210155e00 0x7f0210153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 10 monclient(hunting): picked mon.noname-a con 0x7f0210155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f02101561e0 0x7f02100c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 10 monclient(hunting): picked mon.noname-c con 0x7f02101561e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02100c9360 con 0x7f0210155e00 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.157+0000 7f021586f640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f02100ca1e0 con 0x7f0210074dc0 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.158+0000 7f021586f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.158+0000 7f021586f640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f021015fa10 con 0x7f02101561e0 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.158+0000 7f021586f640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.158+0000 7f021586f640 10 monclient(hunting): authenticate will time out at 1059.727048s 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.158+0000 7f19548ee640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c0052c0 0x7f19501f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7f19548ee640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c0052c0 0x7f19501f78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57054/0 (socket says 10.20.193.16:57054) 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7f19548ee640 1 -- 10.20.193.16:0/3399919245 learned_addr learned my addr 10.20.193.16:0/3399919245 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7f19548ee640 10 monclient(hunting): get_auth_request con 0x7f193c0052c0 auth_method 0 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7f19548ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7f19548ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7f19548ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7fed78a2a640 1 -- 10.20.193.16:0/1934827957 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4020886091 0 0) 0x7fed68002920 con 0x7fed741561e0 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7fed78a2a640 10 monclient(hunting): _negotiate 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7fed78a2a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7fed78a2a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7fed78a2a640 10 monclient(hunting): my global_id is 4538 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.159+0000 7fed78a2a640 1 -- 10.20.193.16:0/1934827957 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed5c003620 con 0x7fed741561e0 2026-02-17T05:56:47.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.160+0000 7efc92625640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efc8c155fa0 0x7efc8c1f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.160+0000 7f19548ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.160+0000 7f19548ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.160+0000 7f19548ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.160+0000 7efc92625640 10 monclient(hunting): get_auth_request con 0x7efc8c155fa0 auth_method 0 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.160+0000 7efc92625640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.160+0000 7efc92625640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.160+0000 7efc92625640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.162+0000 7f19548ee640 10 monclient(hunting): handle_auth_done global_id 4541 payload 824 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.162+0000 7f19548ee640 1 -- 10.20.193.16:0/3399919245 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1950142130 msgr2=0x7f19501f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.162+0000 7f19548ee640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1950142130 0x7f19501f39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.162+0000 7f19548ee640 1 -- 10.20.193.16:0/3399919245 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f195013e2f0 msgr2=0x7f19500c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.162+0000 7f19548ee640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f195013e2f0 0x7f19500c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.162+0000 7f19548ee640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.162+0000 7f19548ee640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.162+0000 7f19548ee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.162+0000 7f19548ee640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.162+0000 7f19548ee640 1 -- 10.20.193.16:0/3399919245 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19501f48a0 con 0x7f193c0052c0 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7efc92625640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7efc92625640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7efc92625640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 1 -- 10.20.193.16:0/2691600672 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e403c070 con 0x7f46f8155e00 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 10 monclient: dump: 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 1 -- 10.20.193.16:0/2691600672 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e402fb40 con 0x7f46f8155e00 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 1 -- 10.20.193.16:0/2691600672 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4037440 con 0x7f46f8155e00 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 5 monclient: authenticate success, global_id 4532 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 4 set_mon_vals no callback set 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46f6ffd640 10 monclient: dump: 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7ffba160c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffb9c14fec0 0x7ffb9c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 10 monclient: shutdown 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 1 -- 10.20.193.16:0/2691600672 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f46f8155e00 msgr2=0x7f46f80ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 1 --2- 10.20.193.16:0/2691600672 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f46f8155e00 0x7f46f80ca700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f46f806ce20 tx=0x7f46e402eed0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 1 -- 10.20.193.16:0/2691600672 shutdown_connections 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 1 --2- 10.20.193.16:0/2691600672 >> v2:10.20.193.16:3300/0 conn(0x7f46f81561e0 0x7f46f80cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 1 --2- 10.20.193.16:0/2691600672 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f46f8155e00 0x7f46f80ca700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 1 -- 10.20.193.16:0/2691600672 >> 10.20.193.16:0/2691600672 conn(0x7f46f80ce6c0 msgr2=0x7f46f80cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 1 -- 10.20.193.16:0/2691600672 shutdown_connections 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.163+0000 7f46fe751640 1 -- 10.20.193.16:0/2691600672 wait complete. 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.164+0000 7f46fe751640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.164+0000 7f46fe751640 10 monclient: monmap: 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.193 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.164+0000 7f46fe751640 1 librados: starting msgr at 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.164+0000 7f46fe751640 1 librados: starting objecter 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.164+0000 7f46fe751640 1 Processor -- start 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.164+0000 7f46fe751640 1 -- start start 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.164+0000 7f46fe751640 1 librados: setting wanted keys 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.164+0000 7f46fe751640 1 librados: calling monclient init 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.164+0000 7f46fe751640 10 monclient: init 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f46f8074dc0 0x7f46f81f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 10 monclient(hunting): picked mon.noname-c con 0x7f46f8074dc0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f46f8155e00 0x7f46f80d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 10 monclient(hunting): picked mon.noname-b con 0x7f46f8155e00 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f46f81561e0 0x7f46f814d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 10 monclient(hunting): picked mon.noname-a con 0x7f46f81561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46f80c9360 con 0x7f46f81561e0 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f46f80ca1e0 con 0x7f46f8155e00 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46f80c8c20 con 0x7f46f8074dc0 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7f46fe751640 10 monclient(hunting): authenticate will time out at 1059.734226s 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7efc92625640 10 monclient(hunting): handle_auth_done global_id 4544 payload 824 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7efc92625640 1 -- 10.20.193.16:0/1090734837 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc8c15d360 msgr2=0x7efc8c1f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7efc92625640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc8c15d360 0x7efc8c1f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7efc92625640 1 -- 10.20.193.16:0/1090734837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc8c156380 msgr2=0x7efc8c15f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7efc92625640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc8c156380 0x7efc8c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7efc92625640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7efc92625640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7efc92625640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7efc92625640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7efc92625640 1 -- 10.20.193.16:0/1090734837 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8c15f760 con 0x7efc8c155fa0 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7ffba160c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffb9c14fec0 0x7ffb9c1555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57110/0 (socket says 10.20.193.16:57110) 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7ffba160c640 1 -- 10.20.193.16:0/1443122777 learned_addr learned my addr 10.20.193.16:0/1443122777 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7ffba160c640 10 monclient(hunting): get_auth_request con 0x7ffb9c14fec0 auth_method 0 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7ffba160c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7ffba160c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.165+0000 7ffba160c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.166+0000 7f19548ee640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c0052c0 0x7f19501f78f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f19400119f0 tx=0x7f1940011ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.166+0000 7f19457fa640 10 client.?.objecter ms_handle_connect 0x7f193c0052c0 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.166+0000 7f19457fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.167+0000 7ffba160c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.167+0000 7ffba160c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.167+0000 7ffba160c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.167+0000 7f8360ed5640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c155e00 0x7f835c143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.169+0000 7fed79a2c640 1 --2- 10.20.193.16:0/1934827957 >> v2:10.20.193.16:3300/0 conn(0x7fed74155e00 0x7fed740ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.169+0000 7efc92625640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efc8c155fa0 0x7efc8c1f7000 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efc8c08f0e0 tx=0x7efc840047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.169+0000 7efc76ffd640 10 client.?.objecter ms_handle_connect 0x7efc8c155fa0 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.169+0000 7efc76ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7f8360ed5640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c155e00 0x7f835c143ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57068/0 (socket says 10.20.193.16:57068) 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7f8360ed5640 1 -- 10.20.193.16:0/944886098 learned_addr learned my addr 10.20.193.16:0/944886098 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7f8360ed5640 10 monclient(hunting): get_auth_request con 0x7f835c155e00 auth_method 0 2026-02-17T05:56:47.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7f8360ed5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7f8360ed5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7f8360ed5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7ffba160c640 10 monclient(hunting): handle_auth_done global_id 4547 payload 824 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7ffba160c640 1 -- 10.20.193.16:0/1443122777 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffb9c0c6fc0 msgr2=0x7ffb9c158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7ffba160c640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffb9c0c6fc0 0x7ffb9c158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7ffba160c640 1 -- 10.20.193.16:0/1443122777 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffb9c0c73a0 msgr2=0x7ffb9c14c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7ffba160c640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffb9c0c73a0 0x7ffb9c14c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7ffba160c640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7ffba160c640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7ffba160c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7ffba160c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.170+0000 7ffba160c640 1 -- 10.20.193.16:0/1443122777 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c158970 con 0x7ffb9c14fec0 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.171+0000 7f8360ed5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.171+0000 7f8360ed5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.171+0000 7f8360ed5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.171+0000 7f97f2ffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f4074dc0 0x7f97f4073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.173+0000 7f8360ed5640 10 monclient(hunting): handle_auth_done global_id 4550 payload 824 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.173+0000 7f8360ed5640 1 -- 10.20.193.16:0/944886098 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f835c074dc0 msgr2=0x7f835c151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.173+0000 7f8360ed5640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f835c074dc0 0x7f835c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.173+0000 7f8360ed5640 1 -- 10.20.193.16:0/944886098 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f835c1561e0 msgr2=0x7f835c14d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.173+0000 7f8360ed5640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f835c1561e0 0x7f835c14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.173+0000 7f8360ed5640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.173+0000 7f8360ed5640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.173+0000 7f8360ed5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.173+0000 7f8360ed5640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.173+0000 7f8360ed5640 1 -- 10.20.193.16:0/944886098 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c1f4980 con 0x7f835c155e00 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.174+0000 7f97f2ffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f4074dc0 0x7f97f4073b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57094/0 (socket says 10.20.193.16:57094) 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.174+0000 7f97f2ffd640 1 -- 10.20.193.16:0/1259493090 learned_addr learned my addr 10.20.193.16:0/1259493090 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.174+0000 7f97f2ffd640 10 monclient(hunting): get_auth_request con 0x7f97f4074dc0 auth_method 0 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.174+0000 7f97f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.174+0000 7f97f2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.174+0000 7f97f2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.174+0000 7ffba160c640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffb9c14fec0 0x7ffb9c1555c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb98039040 tx=0x7ffb98004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.174+0000 7ffb917fa640 10 client.?.objecter ms_handle_connect 0x7ffb9c14fec0 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.174+0000 7ffb917fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.175+0000 7f97f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.175+0000 7f97f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.175+0000 7f97f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.176+0000 7f97f2ffd640 10 monclient(hunting): handle_auth_done global_id 4553 payload 824 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.176+0000 7f97f2ffd640 1 -- 10.20.193.16:0/1259493090 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97f41561e0 msgr2=0x7f97f414d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.176+0000 7f97f2ffd640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97f41561e0 0x7f97f414d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.176+0000 7f97f2ffd640 1 -- 10.20.193.16:0/1259493090 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97f4155e00 msgr2=0x7f97f414cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.176+0000 7f97f2ffd640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97f4155e00 0x7f97f414cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.176+0000 7f97f2ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.176+0000 7f97f2ffd640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.176+0000 7f97f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.176+0000 7f97f2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.176+0000 7f97f2ffd640 1 -- 10.20.193.16:0/1259493090 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f41f8db0 con 0x7f97f4074dc0 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.177+0000 7f8360ed5640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c155e00 0x7f835c143ac0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f835000b500 tx=0x7f835000b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.177+0000 7f8349ffb640 10 client.?.objecter ms_handle_connect 0x7f835c155e00 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.177+0000 7f8349ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.178+0000 7fed79a2c640 10 monclient(hunting): get_auth_request con 0x7fed74155e00 auth_method 0 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.178+0000 7fed79a2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.178+0000 7fed79a2c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.178+0000 7fed79a2c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.178+0000 7fed79a2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.178+0000 7fed79a2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.178+0000 7fed79a2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 1 -- 10.20.193.16:0/3225517002 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c012070 con 0x7fd01c0c8050 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 10 monclient: dump: 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.198 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 1 -- 10.20.193.16:0/3225517002 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd00c0044a0 con 0x7fd01c0c8050 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 1 -- 10.20.193.16:0/3225517002 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0029c0 con 0x7fd01c0c8050 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd0236fa640 5 monclient: authenticate success, global_id 4428 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd0236fa640 10 monclient: _renew_subs 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd0236fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd00b7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd0236fa640 1 -- 10.20.193.16:0/3225517002 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01c152b20 con 0x7fd01c0c8050 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd009ffb640 10 monclient: dump: 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd0236fa640 10 client.4428.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd0236fa640 10 monclient: _renew_subs 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd0236fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd0236fa640 1 -- 10.20.193.16:0/3225517002 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01c1530e0 con 0x7fd01c0c8050 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fd0236fa640 1 librados: init done 2026-02-17T05:56:47.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 1 -- 10.20.193.16:0/2132433469 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8045020 con 0x7f7400155e00 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 10 monclient: dump: 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 1 -- 10.20.193.16:0/2132433469 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e803d040 con 0x7f7400155e00 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f7406d49640 5 monclient: authenticate success, global_id 4535 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f7406d49640 10 monclient: _renew_subs 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f6ffd640 4 set_mon_vals no callback set 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f7406d49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f7406d49640 1 -- 10.20.193.16:0/2132433469 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74001f5ba0 con 0x7f7400155e00 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f7406d49640 10 client.4535.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f7406d49640 10 monclient: _renew_subs 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f7406d49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 1 -- 10.20.193.16:0/2132433469 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e803c070 con 0x7f7400155e00 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f7406d49640 1 -- 10.20.193.16:0/2132433469 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740014dba0 con 0x7f7400155e00 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f7406d49640 1 librados: init done 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f73f57fa640 10 monclient: dump: 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.200 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fbfbe575640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbfc00745e0 0x7fbfc00c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fed79a2c640 10 monclient(hunting): handle_auth_done global_id 4431 payload 274 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fed79a2c640 1 -- 10.20.193.16:0/1934827957 >> v1:10.20.193.148:6789/0 conn(0x7fed741561e0 legacy=0x7fed74074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fed79a2c640 1 -- 10.20.193.16:0/1934827957 >> v1:10.20.193.16:6789/0 conn(0x7fed74074dc0 legacy=0x7fed7415f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7fed79a2c640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fed79a2c640 1 monclient: found mon.noname-a 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fed79a2c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fed79a2c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fed79a2c640 1 -- 10.20.193.16:0/1934827957 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed740c9c20 con 0x7fed74155e00 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f97f2ffd640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f4074dc0 0x7f97f4073b00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97e800c730 tx=0x7f97e800cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f97d37fe640 10 client.?.objecter ms_handle_connect 0x7f97f4074dc0 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.179+0000 7f97d37fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fd0236fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fd0236fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fd0236fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fd0236fa640 1 -- 10.20.193.16:0/3225517002 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd01c0c9850 con 0x7fd01c0c8050 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7f7406d49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7f7406d49640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7f7406d49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7f7406d49640 1 -- 10.20.193.16:0/2132433469 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f740006ccc0 con 0x7f7400155e00 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fbfbe575640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbfc00745e0 0x7fbfc00c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57078/0 (socket says 10.20.193.16:57078) 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fbfbe575640 1 -- 10.20.193.16:0/3509700266 learned_addr learned my addr 10.20.193.16:0/3509700266 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fbfbe575640 10 monclient(hunting): get_auth_request con 0x7fbfc00745e0 auth_method 0 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fbfbe575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fbfbe575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.180+0000 7fbfbe575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.181+0000 7fbfbe575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.181+0000 7fbfbe575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.181+0000 7fbfbe575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.184+0000 7fbfbe575640 10 monclient(hunting): handle_auth_done global_id 4556 payload 824 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.184+0000 7fbfbe575640 1 -- 10.20.193.16:0/3509700266 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbfc0155fa0 msgr2=0x7fbfc0149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.184+0000 7fbfbe575640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbfc0155fa0 0x7fbfc0149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.184+0000 7fbfbe575640 1 -- 10.20.193.16:0/3509700266 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbfc0156380 msgr2=0x7fbfc015fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.184+0000 7fbfbe575640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbfc0156380 0x7fbfc015fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.184+0000 7fbfbe575640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.184+0000 7fbfbe575640 1 monclient: found mon.noname-c 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.184+0000 7fbfbe575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.184+0000 7fbfbe575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.184+0000 7fbfbe575640 1 -- 10.20.193.16:0/3509700266 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc0200c20 con 0x7fbfc00745e0 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 1 -- 10.20.193.16:0/1443122777 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98046020 con 0x7ffb9c14fec0 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 10 monclient: dump: 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 1 -- 10.20.193.16:0/1443122777 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb98041450 con 0x7ffb9c14fec0 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffba3096640 5 monclient: authenticate success, global_id 4547 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffba3096640 10 monclient: _renew_subs 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffba3096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffba3096640 1 -- 10.20.193.16:0/1443122777 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9c158c50 con 0x7ffb9c14fec0 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 1 -- 10.20.193.16:0/1443122777 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98041820 con 0x7ffb9c14fec0 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb917fa640 10 monclient: dump: 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.203 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffb92ffd640 4 set_mon_vals no callback set 2026-02-17T05:56:47.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffba3096640 10 client.4547.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffba3096640 10 monclient: _renew_subs 2026-02-17T05:56:47.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffba3096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffba3096640 1 -- 10.20.193.16:0/1443122777 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9c159210 con 0x7ffb9c14fec0 2026-02-17T05:56:47.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.185+0000 7ffba3096640 1 librados: init done 2026-02-17T05:56:47.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.186+0000 7ffba3096640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.186+0000 7ffba3096640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.186+0000 7ffba3096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.186+0000 7ffba3096640 1 -- 10.20.193.16:0/1443122777 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb9c0698c0 con 0x7ffb9c14fec0 2026-02-17T05:56:47.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.250+0000 7fed79a2c640 1 --2- 10.20.193.16:0/1934827957 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 0x7fed740ca700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed640098a0 tx=0x7fed6402ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa40197603bc8ee server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 1 -- 10.20.193.16:0/2708375620 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478003c070 con 0x7f478c155e00 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 10 monclient: dump: 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 1 -- 10.20.193.16:0/2708375620 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f478002fb60 con 0x7f478c155e00 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 1 -- 10.20.193.16:0/2708375620 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780037440 con 0x7f478c155e00 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f4789ffb640 10 monclient: dump: 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.258 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f479167e640 5 monclient: authenticate success, global_id 4492 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f479167e640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.257+0000 7f479167e640 4 set_mon_vals no callback set 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.258+0000 7f479167e640 10 monclient: shutdown 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.258+0000 7f479167e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.258+0000 7f479167e640 1 -- 10.20.193.16:0/2708375620 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c155e00 msgr2=0x7f478c0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.258+0000 7f479167e640 1 --2- 10.20.193.16:0/2708375620 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c155e00 0x7f478c0ca700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f478c06ce20 tx=0x7f478002eef0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.258+0000 7f479167e640 1 -- 10.20.193.16:0/2708375620 shutdown_connections 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.258+0000 7f479167e640 1 --2- 10.20.193.16:0/2708375620 >> v2:10.20.193.16:3300/0 conn(0x7f478c1561e0 0x7f478c0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.258+0000 7f479167e640 1 --2- 10.20.193.16:0/2708375620 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c155e00 0x7f478c0ca700 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.258+0000 7f479167e640 1 -- 10.20.193.16:0/2708375620 >> 10.20.193.16:0/2708375620 conn(0x7f478c0ce6c0 msgr2=0x7f478c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.258+0000 7f479167e640 1 -- 10.20.193.16:0/2708375620 shutdown_connections 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.258+0000 7f479167e640 1 -- 10.20.193.16:0/2708375620 wait complete. 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient: monmap: 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 librados: starting msgr at 2026-02-17T05:56:47.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 librados: starting objecter 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7fe8f9ffb640 1 -- 10.20.193.16:0/161933326 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fe900003a70 con 0x7fe90c156380 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 Processor -- start 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 -- start start 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 librados: setting wanted keys 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 librados: calling monclient init 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient: init 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7fe8f9ffb640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe8dc079190 0x7fe8dc07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f478c074dc0 0x7f478c14d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient(hunting): picked mon.noname-c con 0x7f478c074dc0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f478c155e00 0x7f478c14d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient(hunting): picked mon.noname-a con 0x7f478c155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c1561e0 0x7f478c143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient(hunting): picked mon.noname-b con 0x7f478c1561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f478c0c9360 con 0x7f478c155e00 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f478c0ca1e0 con 0x7f478c1561e0 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f478c0c8c20 con 0x7f478c074dc0 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7f479167e640 10 monclient(hunting): authenticate will time out at 1059.828887s 2026-02-17T05:56:47.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.259+0000 7fe911089640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe8dc079190 0x7fe8dc07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.260+0000 7fe911089640 10 monclient: get_auth_request con 0x7fe8dc079190 auth_method 0 2026-02-17T05:56:47.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.260+0000 7fe8f9ffb640 10 client.4422.objecter ms_handle_connect 0x7fe8dc079190 2026-02-17T05:56:47.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.260+0000 7fe911089640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe8dc079190 0x7fe8dc07b650 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc039000 tx=0x7fe8fc031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.265+0000 7faa46ffd640 1 -- 10.20.193.16:0/1476379032 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7faa5c01b3b0 con 0x7faa6015d360 2026-02-17T05:56:47.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.266+0000 7faa46ffd640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faa34079190 0x7faa3407b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.266+0000 7faa661c1640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faa34079190 0x7faa3407b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.266+0000 7faa661c1640 10 monclient: get_auth_request con 0x7faa34079190 auth_method 0 2026-02-17T05:56:47.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.266+0000 7faa661c1640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faa34079190 0x7faa3407b650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa50039000 tx=0x7faa50031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.266+0000 7faa46ffd640 10 client.4477.objecter ms_handle_connect 0x7faa34079190 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7faa46ffd640 1 -- 10.20.193.16:0/1476379032 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7faa5c05df30 con 0x7faa6015d360 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7faa46ffd640 10 client.4477.objecter ms_dispatch2 0x7faa60000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7faa46ffd640 3 client.4477.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7faa46ffd640 3 client.4477.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7faa46ffd640 20 client.4477.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7fe8f9ffb640 1 -- 10.20.193.16:0/161933326 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fe90005e360 con 0x7fe90c156380 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7fe8f9ffb640 10 client.4422.objecter ms_dispatch2 0x7fe90c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7fe8f9ffb640 3 client.4422.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7fe8f9ffb640 3 client.4422.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7fe8f9ffb640 20 client.4422.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7fe8f9ffb640 1 -- 10.20.193.16:0/161933326 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe900062a20 con 0x7fe90c156380 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7fe8f9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.268+0000 7fe8f9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.274 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.274+0000 7faa46ffd640 1 -- 10.20.193.16:0/1476379032 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7faa60076e00 con 0x7faa6015d360 2026-02-17T05:56:47.274 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.274+0000 7faa46ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.274+0000 7faa46ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f020effd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0210074dc0 0x7f021014e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f30ecf28640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e81561e0 0x7f30e814c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f30ecf28640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e81561e0 0x7f30e814c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50444/0 (socket says 10.20.193.16:50444) 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f30ecf28640 1 -- 10.20.193.16:0/770118070 learned_addr learned my addr 10.20.193.16:0/770118070 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f30ecf28640 10 monclient(hunting): get_auth_request con 0x7f30e81561e0 auth_method 0 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f30ecf28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f30ecf28640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f30ecf28640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f020effd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0210074dc0 0x7f021014e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50456/0 (socket says 10.20.193.16:50456) 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f020effd640 1 -- 10.20.193.16:0/2582501118 learned_addr learned my addr 10.20.193.16:0/2582501118 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f020effd640 10 monclient(hunting): get_auth_request con 0x7f0210074dc0 auth_method 0 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f020effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f020effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.278+0000 7f020effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.279+0000 7f30ecf28640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.279+0000 7f30ecf28640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.279+0000 7f30ecf28640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.279+0000 7f020effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.279+0000 7f020effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.279+0000 7f020effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.280+0000 7f30ecf28640 10 monclient(hunting): handle_auth_done global_id 4501 payload 824 2026-02-17T05:56:47.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.280+0000 7f30ecf28640 1 -- 10.20.193.16:0/770118070 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30e8155e00 msgr2=0x7f30e81fc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.280+0000 7f30ecf28640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30e8155e00 0x7f30e81fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.280+0000 7f30ecf28640 1 -- 10.20.193.16:0/770118070 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e815fa10 msgr2=0x7f30e8147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.280+0000 7f30ecf28640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e815fa10 0x7f30e8147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.280+0000 7f30ecf28640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.280+0000 7f30ecf28640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.280+0000 7f30ecf28640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.280+0000 7f30ecf28640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.280+0000 7f30ecf28640 1 -- 10.20.193.16:0/770118070 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e8150f00 con 0x7f30e81561e0 2026-02-17T05:56:47.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.283+0000 7f020effd640 10 monclient(hunting): handle_auth_done global_id 4504 payload 824 2026-02-17T05:56:47.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.283+0000 7f020effd640 1 -- 10.20.193.16:0/2582501118 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f02101561e0 msgr2=0x7f02100c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.283+0000 7f020effd640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f02101561e0 0x7f02100c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.283+0000 7f020effd640 1 -- 10.20.193.16:0/2582501118 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0210155e00 msgr2=0x7f0210153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.283+0000 7f020effd640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0210155e00 0x7f0210153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.283+0000 7f020effd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.283+0000 7f020effd640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.283+0000 7f020effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.283+0000 7f020effd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.283+0000 7f020effd640 1 -- 10.20.193.16:0/2582501118 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021014e650 con 0x7f0210074dc0 2026-02-17T05:56:47.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.284+0000 7f30ecf28640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e81561e0 0x7f30e814c420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00b570 tx=0x7f30dc00ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.284+0000 7f30d5ffb640 10 client.?.objecter ms_handle_connect 0x7f30e81561e0 2026-02-17T05:56:47.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.284+0000 7f30d5ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.285+0000 7f478b7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c1561e0 0x7f478c143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.286+0000 7f020effd640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0210074dc0 0x7f021014e110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f020400b570 tx=0x7f020400ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.286+0000 7f01e7fff640 10 client.?.objecter ms_handle_connect 0x7f0210074dc0 2026-02-17T05:56:47.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.286+0000 7f01e7fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.287+0000 7f46f77fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f46f8155e00 0x7f46f80d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.287+0000 7f46f77fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f46f8155e00 0x7f46f80d0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50470/0 (socket says 10.20.193.16:50470) 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.287+0000 7f46f77fe640 1 -- 10.20.193.16:0/3024630734 learned_addr learned my addr 10.20.193.16:0/3024630734 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.287+0000 7f46f77fe640 10 monclient(hunting): get_auth_request con 0x7f46f8155e00 auth_method 0 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.287+0000 7f46f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.287+0000 7f46f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.287+0000 7f46f77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.288+0000 7f478b7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c1561e0 0x7f478c143ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50478/0 (socket says 10.20.193.16:50478) 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.288+0000 7f478b7fe640 1 -- 10.20.193.16:0/4188371468 learned_addr learned my addr 10.20.193.16:0/4188371468 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.288+0000 7f478b7fe640 10 monclient(hunting): get_auth_request con 0x7f478c1561e0 auth_method 0 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.288+0000 7f478b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.288+0000 7f478b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.288+0000 7f478b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.288+0000 7fbfbe575640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbfc00745e0 0x7fbfc00c37f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400b570 tx=0x7fbfb400ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.288+0000 7fbfa6ffd640 10 client.?.objecter ms_handle_connect 0x7fbfc00745e0 2026-02-17T05:56:47.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.288+0000 7fbfa6ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f46f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f46f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f46f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f478b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f478b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f478b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 1 -- 10.20.193.16:0/2582501118 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204013020 con 0x7f0210074dc0 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 10 monclient: dump: 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 1 -- 10.20.193.16:0/2582501118 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02040044a0 con 0x7f0210074dc0 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 1 -- 10.20.193.16:0/2582501118 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02040029c0 con 0x7f0210074dc0 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f021586f640 5 monclient: authenticate success, global_id 4504 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f020d7fa640 4 set_mon_vals no callback set 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f021586f640 10 monclient: _renew_subs 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f021586f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f021586f640 1 -- 10.20.193.16:0/2582501118 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02101f3bf0 con 0x7f0210074dc0 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.289+0000 7f01e7fff640 10 monclient: dump: 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.292 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.290+0000 7f021586f640 10 client.4504.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.290+0000 7f021586f640 10 monclient: _renew_subs 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.290+0000 7f021586f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.290+0000 7f021586f640 1 -- 10.20.193.16:0/2582501118 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02101f41b0 con 0x7f0210074dc0 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.290+0000 7f021586f640 1 librados: init done 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.290+0000 7f021586f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.290+0000 7f021586f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.290+0000 7f021586f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.290+0000 7f021586f640 1 -- 10.20.193.16:0/2582501118 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02100c58b0 con 0x7f0210074dc0 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f46f77fe640 10 monclient(hunting): handle_auth_done global_id 4507 payload 824 2026-02-17T05:56:47.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f46f77fe640 1 -- 10.20.193.16:0/3024630734 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f46f8074dc0 msgr2=0x7f46f81f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f46f77fe640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f46f8074dc0 0x7f46f81f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f46f77fe640 1 -- 10.20.193.16:0/3024630734 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f46f81561e0 msgr2=0x7f46f814d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f46f77fe640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f46f81561e0 0x7f46f814d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f46f77fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f46f77fe640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f46f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f46f77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f46f77fe640 1 -- 10.20.193.16:0/3024630734 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f80c5cc0 con 0x7f46f8155e00 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f478b7fe640 10 monclient(hunting): handle_auth_done global_id 4510 payload 824 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f478b7fe640 1 -- 10.20.193.16:0/4188371468 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f478c074dc0 msgr2=0x7f478c14d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f478b7fe640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f478c074dc0 0x7f478c14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f478b7fe640 1 -- 10.20.193.16:0/4188371468 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f478c155e00 msgr2=0x7f478c14d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f478b7fe640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f478c155e00 0x7f478c14d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f478b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f478b7fe640 1 monclient: found mon.noname-b 2026-02-17T05:56:47.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f478b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f478b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.292+0000 7f478b7fe640 1 -- 10.20.193.16:0/4188371468 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c14dc40 con 0x7f478c1561e0 2026-02-17T05:56:47.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.294+0000 7f46f77fe640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f46f8155e00 0x7f46f80d0ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c7f0 tx=0x7f46e800ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.294+0000 7f46f4ff9640 10 client.?.objecter ms_handle_connect 0x7f46f8155e00 2026-02-17T05:56:47.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.294+0000 7f46f4ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.298+0000 7f478b7fe640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c1561e0 0x7f478c143ee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f477c00cc40 tx=0x7f477c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.298+0000 7f476b7fe640 10 client.?.objecter ms_handle_connect 0x7f478c1561e0 2026-02-17T05:56:47.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.298+0000 7f476b7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.353+0000 7fe913314640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-17T05:56:47.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.353+0000 7fe913314640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-17T05:56:47.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.353+0000 7fe913314640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.353+0000 7fe913314640 1 -- 10.20.193.16:0/161933326 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fe90c14cb10 con 0x7fe90c156380 2026-02-17T05:56:47.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.363+0000 7faa6844c640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:56:47.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.363+0000 7faa6844c640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:56:47.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.363+0000 7faa6844c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.363+0000 7faa6844c640 1 -- 10.20.193.16:0/1476379032 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7faa600740a0 con 0x7faa6015d360 2026-02-17T05:56:47.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4004620 con 0x7f1fa0005310 2026-02-17T05:56:47.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb1573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:47.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb1573640 10 monclient: dump: 2026-02-17T05:56:47.379 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.379 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.379 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.379 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa402fe40 con 0x7f1fa0005310 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb1573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb1573640 1 -- 10.20.193.16:0/3654238760 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4037680 con 0x7f1fa0005310 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb1573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:47.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb1573640 10 monclient: dump: 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb8e05640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.378+0000 7f1fb8e05640 4 set_mon_vals no callback set 2026-02-17T05:56:47.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7f1fb8e05640 10 monclient: shutdown 2026-02-17T05:56:47.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7f1fb8e05640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7f1fb8e05640 1 -- 10.20.193.16:0/3654238760 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fa0005310 msgr2=0x7f1fa000bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7f1fb8e05640 1 --2- 10.20.193.16:0/3654238760 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fa0005310 0x7f1fa000bac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4004a20 tx=0x7f1fa402f3c0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7f1fb8e05640 1 -- 10.20.193.16:0/3654238760 shutdown_connections 2026-02-17T05:56:47.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7f1fb8e05640 1 --2- 10.20.193.16:0/3654238760 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fa0005310 0x7f1fa000bac0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7f1fb8e05640 1 -- 10.20.193.16:0/3654238760 >> 10.20.193.16:0/3654238760 conn(0x7f1fb40ca5b0 msgr2=0x7f1fb40ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7f1fb8e05640 1 -- 10.20.193.16:0/3654238760 shutdown_connections 2026-02-17T05:56:47.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7f1fb8e05640 1 -- 10.20.193.16:0/3654238760 wait complete. 2026-02-17T05:56:47.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 1 -- 10.20.193.16:0/1934827957 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6403c070 con 0x7fed74155e00 2026-02-17T05:56:47.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:47.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 10 monclient: dump: 2026-02-17T05:56:47.384 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.384 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.384 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.384 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.384 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.384 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 1 -- 10.20.193.16:0/1934827957 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6402fad0 con 0x7fed74155e00 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 1 -- 10.20.193.16:0/1934827957 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64037440 con 0x7fed74155e00 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed78a2a640 10 monclient: dump: 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed7bcb7640 5 monclient: authenticate success, global_id 4431 2026-02-17T05:56:47.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed7bcb7640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:47.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.379+0000 7fed7bcb7640 4 set_mon_vals no callback set 2026-02-17T05:56:47.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient: monmap: 2026-02-17T05:56:47.387 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.387 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.387 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.388 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.388 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.388 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.388 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.388 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.388 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.388 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 10 monclient: shutdown 2026-02-17T05:56:47.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 1 -- 10.20.193.16:0/1934827957 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 msgr2=0x7fed740ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 1 --2- 10.20.193.16:0/1934827957 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 0x7fed740ca700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed640098a0 tx=0x7fed6402ee10 comp rx=0 tx=0).stop 2026-02-17T05:56:47.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 1 -- 10.20.193.16:0/1934827957 shutdown_connections 2026-02-17T05:56:47.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 1 --2- 10.20.193.16:0/1934827957 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 0x7fed740ca700 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 1 -- 10.20.193.16:0/1934827957 >> 10.20.193.16:0/1934827957 conn(0x7fed740ce6c0 msgr2=0x7fed740cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 1 -- 10.20.193.16:0/1934827957 shutdown_connections 2026-02-17T05:56:47.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 1 -- 10.20.193.16:0/1934827957 wait complete. 2026-02-17T05:56:47.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 librados: starting msgr at 2026-02-17T05:56:47.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 librados: starting objecter 2026-02-17T05:56:47.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 Processor -- start 2026-02-17T05:56:47.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 -- start start 2026-02-17T05:56:47.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 librados: setting wanted keys 2026-02-17T05:56:47.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 librados: calling monclient init 2026-02-17T05:56:47.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient: init 2026-02-17T05:56:47.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 10 monclient: build_initial_monmap 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 10 monclient: monmap: 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:47.392 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:47.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1fa0005310 0x7f1fb41f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient(hunting): picked mon.noname-c con 0x7f1fa0005310 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1fb40c6eb0 0x7f1fb41f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient(hunting): picked mon.noname-b con 0x7f1fb40c6eb0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fb4073560 0x7f1fb4146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient(hunting): picked mon.noname-a con 0x7f1fb4073560 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb40c9cb0 con 0x7f1fb4073560 2026-02-17T05:56:47.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb40c94b0 con 0x7f1fb40c6eb0 2026-02-17T05:56:47.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb40c9e30 con 0x7f1fa0005310 2026-02-17T05:56:47.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7f1fb8e05640 10 monclient(hunting): authenticate will time out at 1059.949933s 2026-02-17T05:56:47.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 1 librados: starting msgr at 2026-02-17T05:56:47.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.380+0000 7fed7bcb7640 1 librados: starting objecter 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 1 Processor -- start 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 1 -- start start 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 1 librados: setting wanted keys 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 1 librados: calling monclient init 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient: init 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fed74074dc0 0x7fed741f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient(hunting): picked mon.noname-c con 0x7fed74074dc0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 0x7fed7415e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient(hunting): picked mon.noname-a con 0x7fed74155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:47.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fed741561e0 0x7fed740c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient(hunting): picked mon.noname-b con 0x7fed741561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:47.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed740c9360 con 0x7fed74155e00 2026-02-17T05:56:47.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fed740ca1e0 con 0x7fed741561e0 2026-02-17T05:56:47.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:47.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed7415fa10 con 0x7fed74074dc0 2026-02-17T05:56:47.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:47.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.381+0000 7fed7bcb7640 10 monclient(hunting): authenticate will time out at 1059.950594s 2026-02-17T05:56:47.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.387+0000 7fe8f9ffb640 1 -- 10.20.193.16:0/161933326 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fe900062320 con 0x7fe90c156380 2026-02-17T05:56:47.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.387+0000 7fe8f9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-17T05:56:47.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.387+0000 7fe8f9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.399 INFO:teuthology.orchestra.run.trial016.stdout:98784247812 2026-02-17T05:56:47.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 10 librados: watch_flush enter 2026-02-17T05:56:47.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 10 librados: watch_flush exit 2026-02-17T05:56:47.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 20 client.4422.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 10 client.4422.objecter successfully canceled tick 2026-02-17T05:56:47.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 -- 10.20.193.16:0/161933326 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe8dc079190 msgr2=0x7fe8dc07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe8dc079190 0x7fe8dc07b650 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc039000 tx=0x7fe8fc031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 10 monclient: shutdown 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 -- 10.20.193.16:0/161933326 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe90c156380 msgr2=0x7fe90c1495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe90c156380 0x7fe90c1495a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe90000c730 tx=0x7fe90000cc00 comp rx=0 tx=0).stop 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 -- 10.20.193.16:0/161933326 shutdown_connections 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe8dc079190 0x7fe8dc07b650 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe90c15d360 0x7fe90c149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe90c156380 0x7fe90c1495a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 --2- 10.20.193.16:0/161933326 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe90c155fa0 0x7fe90c148430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 -- 10.20.193.16:0/161933326 >> 10.20.193.16:0/161933326 conn(0x7fe90c0ce6c0 msgr2=0x7fe90c14d030 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 -- 10.20.193.16:0/161933326 shutdown_connections 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 -- 10.20.193.16:0/161933326 wait complete. 2026-02-17T05:56:47.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7fe913314640 1 librados: shutdown 2026-02-17T05:56:47.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.389+0000 7f1fb2d76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fb4073560 0x7f1fb4146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.390+0000 7f1fb2d76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fb4073560 0x7f1fb4146880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59950/0 (socket says 10.20.193.16:59950) 2026-02-17T05:56:47.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.390+0000 7f1fb2d76640 1 -- 10.20.193.16:0/2467955324 learned_addr learned my addr 10.20.193.16:0/2467955324 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.390+0000 7f1fb2d76640 10 monclient(hunting): get_auth_request con 0x7f1fb4073560 auth_method 0 2026-02-17T05:56:47.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.390+0000 7f1fb2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.390+0000 7f1fb2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.390+0000 7f1fb2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.390+0000 7f1fb2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.390+0000 7f1fb2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.390+0000 7f1fb2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.11 2026-02-17T05:56:47.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.398+0000 7fd009ffb640 1 -- 10.20.193.16:0/3225517002 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fd00c021480 con 0x7fd01c0c8050 2026-02-17T05:56:47.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7fd009ffb640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcff8079190 0x7fcff807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7fd020c6e640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcff8079190 0x7fcff807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7fd020c6e640 10 monclient: get_auth_request con 0x7fcff8079190 auth_method 0 2026-02-17T05:56:47.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7f73f57fa640 1 -- 10.20.193.16:0/2132433469 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f73e8041a10 con 0x7f7400155e00 2026-02-17T05:56:47.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7fd009ffb640 10 client.4428.objecter ms_handle_connect 0x7fcff8079190 2026-02-17T05:56:47.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7fd020c6e640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcff8079190 0x7fcff807b650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd01c14b1b0 tx=0x7fd010005c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7f73f57fa640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f73d0079190 0x7f73d007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7f7404abe640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f73d0079190 0x7f73d007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7f7404abe640 10 monclient: get_auth_request con 0x7f73d0079190 auth_method 0 2026-02-17T05:56:47.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7f7404abe640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f73d0079190 0x7f73d007b650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f73f00098a0 tx=0x7f73f0006d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.399+0000 7f73f57fa640 10 client.4535.objecter ms_handle_connect 0x7f73d0079190 2026-02-17T05:56:47.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.400+0000 7f01e7fff640 1 -- 10.20.193.16:0/2582501118 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f0204002b60 con 0x7f0210074dc0 2026-02-17T05:56:47.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.401+0000 7f01e7fff640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01e0079190 0x7f01e007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.401+0000 7f020e7fc640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01e0079190 0x7f01e007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.401+0000 7f020e7fc640 10 monclient: get_auth_request con 0x7f01e0079190 auth_method 0 2026-02-17T05:56:47.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.401+0000 7f01e7fff640 10 client.4504.objecter ms_handle_connect 0x7f01e0079190 2026-02-17T05:56:47.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.401+0000 7f020e7fc640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01e0079190 0x7f01e007b650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f021006ce20 tx=0x7f01f8031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.401+0000 7f73f57fa640 1 -- 10.20.193.16:0/2132433469 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f73e802fac0 con 0x7f7400155e00 2026-02-17T05:56:47.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.401+0000 7f73f57fa640 10 client.4535.objecter ms_dispatch2 0x7f7400000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.401+0000 7f73f57fa640 3 client.4535.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.401+0000 7f73f57fa640 3 client.4535.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.401+0000 7f73f57fa640 20 client.4535.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f01e7fff640 1 -- 10.20.193.16:0/2582501118 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f020405e2e0 con 0x7f0210074dc0 2026-02-17T05:56:47.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f01e7fff640 10 client.4504.objecter ms_dispatch2 0x7f0210000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f01e7fff640 3 client.4504.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f01e7fff640 3 client.4504.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f01e7fff640 20 client.4504.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 1 -- 10.20.193.16:0/770118070 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc013020 con 0x7f30e81561e0 2026-02-17T05:56:47.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 10 monclient: dump: 2026-02-17T05:56:47.411 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.411 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.411 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.411 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.411 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 1 -- 10.20.193.16:0/770118070 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30dc0044a0 con 0x7f30e81561e0 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 1 -- 10.20.193.16:0/770118070 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0029c0 con 0x7f30e81561e0 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30ef9b4640 5 monclient: authenticate success, global_id 4501 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30ef9b4640 10 monclient: _renew_subs 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30ef9b4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30ef9b4640 1 -- 10.20.193.16:0/770118070 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e81511e0 con 0x7f30e81561e0 2026-02-17T05:56:47.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d77fe640 4 set_mon_vals no callback set 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30d5ffb640 10 monclient: dump: 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30ef9b4640 10 client.4501.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30ef9b4640 10 monclient: _renew_subs 2026-02-17T05:56:47.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30ef9b4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30ef9b4640 1 -- 10.20.193.16:0/770118070 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e81517a0 con 0x7f30e81561e0 2026-02-17T05:56:47.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.402+0000 7f30ef9b4640 1 librados: init done 2026-02-17T05:56:47.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.403+0000 7f30ef9b4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.403+0000 7f30ef9b4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.403+0000 7f30ef9b4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.403+0000 7f30ef9b4640 1 -- 10.20.193.16:0/770118070 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e80ca700 con 0x7f30e81561e0 2026-02-17T05:56:47.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 1 -- 10.20.193.16:0/1090734837 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84044070 con 0x7efc8c155fa0 2026-02-17T05:56:47.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 10 monclient: dump: 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 1 -- 10.20.193.16:0/1090734837 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc84040610 con 0x7efc8c155fa0 2026-02-17T05:56:47.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 1 -- 10.20.193.16:0/1090734837 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8403c040 con 0x7efc8c155fa0 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 5 monclient: authenticate success, global_id 4544 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 10 monclient: _renew_subs 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc90e22640 4 set_mon_vals no callback set 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 1 -- 10.20.193.16:0/1090734837 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8c1f7ec0 con 0x7efc8c155fa0 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc76ffd640 10 monclient: dump: 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.416 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 10 client.4544.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 10 monclient: _renew_subs 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 1 -- 10.20.193.16:0/1090734837 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8c1f7900 con 0x7efc8c155fa0 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 1 librados: init done 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7fd009ffb640 1 -- 10.20.193.16:0/3225517002 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fd00c05dbf0 con 0x7fd01c0c8050 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7fd009ffb640 10 client.4428.objecter ms_dispatch2 0x7fd01c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7fd009ffb640 3 client.4428.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7fd009ffb640 3 client.4428.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7fd009ffb640 20 client.4428.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7fd009ffb640 1 -- 10.20.193.16:0/3225517002 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd00c061bb0 con 0x7fd01c0c8050 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7fd009ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7fd009ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7efc948b0640 1 -- 10.20.193.16:0/1090734837 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc8c074d50 con 0x7efc8c155fa0 2026-02-17T05:56:47.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7f73f57fa640 1 -- 10.20.193.16:0/2132433469 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f740006ccc0 con 0x7f7400155e00 2026-02-17T05:56:47.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7f73f57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.408+0000 7f73f57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f01e7fff640 1 -- 10.20.193.16:0/2582501118 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f020401d020 con 0x7f0210074dc0 2026-02-17T05:56:47.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f01e7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f01e7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 1 -- 10.20.193.16:0/1259493090 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e80052d0 con 0x7f97f4074dc0 2026-02-17T05:56:47.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 10 monclient: dump: 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 1 -- 10.20.193.16:0/1259493090 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e8005490 con 0x7f97f4074dc0 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 1 -- 10.20.193.16:0/1259493090 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e80038b0 con 0x7f97f4074dc0 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97d37fe640 10 monclient: dump: 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97f17fa640 4 set_mon_vals no callback set 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97f9379640 5 monclient: authenticate success, global_id 4553 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97f9379640 10 monclient: _renew_subs 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97f9379640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97f9379640 1 -- 10.20.193.16:0/1259493090 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f41f9090 con 0x7f97f4074dc0 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97f9379640 10 client.4553.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97f9379640 10 monclient: _renew_subs 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97f9379640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97f9379640 1 -- 10.20.193.16:0/1259493090 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f41f9650 con 0x7f97f4074dc0 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.409+0000 7f97f9379640 1 librados: init done 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f97f9379640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f97f9379640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f97f9379640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f97f9379640 1 -- 10.20.193.16:0/1259493090 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f40cbb60 con 0x7f97f4074dc0 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f1fb2d76640 10 monclient(hunting): handle_auth_done global_id 4434 payload 824 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f1fb2d76640 1 -- 10.20.193.16:0/2467955324 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1fa0005310 msgr2=0x7f1fb41f05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f1fb2d76640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1fa0005310 0x7f1fb41f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f1fb2d76640 1 -- 10.20.193.16:0/2467955324 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1fb40c6eb0 msgr2=0x7f1fb41f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f1fb2d76640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1fb40c6eb0 0x7f1fb41f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f1fb2d76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f1fb2d76640 1 monclient: found mon.noname-a 2026-02-17T05:56:47.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f1fb2d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f1fb2d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.410+0000 7f1fb2d76640 1 -- 10.20.193.16:0/2467955324 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb40c5e10 con 0x7f1fb4073560 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 1 -- 10.20.193.16:0/3399919245 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194000fb50 con 0x7f193c0052c0 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 10 monclient: dump: 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 1 -- 10.20.193.16:0/3399919245 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194000fcf0 con 0x7f193c0052c0 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 1 -- 10.20.193.16:0/3399919245 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194000fea0 con 0x7f193c0052c0 2026-02-17T05:56:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1956b79640 5 monclient: authenticate success, global_id 4541 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1946ffd640 4 set_mon_vals no callback set 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f19457fa640 10 monclient: dump: 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1956b79640 10 monclient: _renew_subs 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1956b79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1956b79640 1 -- 10.20.193.16:0/3399919245 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19501f42e0 con 0x7f193c0052c0 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1956b79640 10 client.4541.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1956b79640 10 monclient: _renew_subs 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1956b79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1956b79640 1 -- 10.20.193.16:0/3399919245 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19500c58a0 con 0x7f193c0052c0 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1956b79640 1 librados: init done 2026-02-17T05:56:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.412+0000 7f1956b79640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.413+0000 7f1956b79640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.413+0000 7f1956b79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.413+0000 7f1956b79640 1 -- 10.20.193.16:0/3399919245 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19500caa30 con 0x7f193c0052c0 2026-02-17T05:56:47.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.413+0000 7f1fb2d76640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fb4073560 0x7f1fb4146880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4002790 tx=0x7f1fa4003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.413+0000 7f1f9affd640 10 client.?.objecter ms_handle_connect 0x7f1fb4073560 2026-02-17T05:56:47.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.413+0000 7f1f9affd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.415+0000 7fed7922b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 0x7fed7415e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.415+0000 7fed7922b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 0x7fed7415e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59964/0 (socket says 10.20.193.16:59964) 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.415+0000 7fed7922b640 1 -- 10.20.193.16:0/2807428913 learned_addr learned my addr 10.20.193.16:0/2807428913 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.415+0000 7fed7922b640 10 monclient(hunting): get_auth_request con 0x7fed74155e00 auth_method 0 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.415+0000 7fed7922b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.415+0000 7fed7922b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.415+0000 7fed7922b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.415+0000 7fed7922b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.415+0000 7fed7922b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.415+0000 7fed7922b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.417+0000 7fed7922b640 10 monclient(hunting): handle_auth_done global_id 4437 payload 824 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.417+0000 7fed7922b640 1 -- 10.20.193.16:0/2807428913 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fed74074dc0 msgr2=0x7fed741f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.417+0000 7fed7922b640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fed74074dc0 0x7fed741f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.417+0000 7fed7922b640 1 -- 10.20.193.16:0/2807428913 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fed741561e0 msgr2=0x7fed740c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.417+0000 7fed7922b640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fed741561e0 0x7fed740c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.417+0000 7fed7922b640 10 monclient: _finish_hunting 0 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.417+0000 7fed7922b640 1 monclient: found mon.noname-a 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.417+0000 7fed7922b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.417+0000 7fed7922b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.417+0000 7fed7922b640 1 -- 10.20.193.16:0/2807428913 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7415d350 con 0x7fed74155e00 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.418+0000 7fed7922b640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 0x7fed7415e6d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c730 tx=0x7fed6800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.418+0000 7fed627fc640 10 client.?.objecter ms_handle_connect 0x7fed74155e00 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.418+0000 7fed627fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 1 -- 10.20.193.16:0/944886098 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350004480 con 0x7f835c155e00 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 10 monclient: dump: 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 1 -- 10.20.193.16:0/944886098 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8350014070 con 0x7f835c155e00 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 1 -- 10.20.193.16:0/944886098 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350002920 con 0x7f835c155e00 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8363961640 5 monclient: authenticate success, global_id 4550 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8363961640 10 monclient: _renew_subs 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f834b7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8363961640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8363961640 1 -- 10.20.193.16:0/944886098 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c150140 con 0x7f835c155e00 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 1 -- 10.20.193.16:0/2467955324 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4003f70 con 0x7f1fb4073560 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8349ffb640 10 monclient: dump: 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.430 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 10 monclient: dump: 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 1 -- 10.20.193.16:0/2467955324 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa4047070 con 0x7f1fb4073560 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8363961640 10 client.4550.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8363961640 10 monclient: _renew_subs 2026-02-17T05:56:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8363961640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 1 -- 10.20.193.16:0/2467955324 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4037e60 con 0x7f1fb4073560 2026-02-17T05:56:47.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8363961640 1 -- 10.20.193.16:0/944886098 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c150700 con 0x7f835c155e00 2026-02-17T05:56:47.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f8363961640 1 librados: init done 2026-02-17T05:56:47.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1fb8e05640 5 monclient: authenticate success, global_id 4434 2026-02-17T05:56:47.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1fb0d72640 4 set_mon_vals no callback set 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1f9affd640 10 monclient: dump: 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1fb8e05640 10 monclient: _renew_subs 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1fb8e05640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1fb8e05640 1 -- 10.20.193.16:0/2467955324 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb40c60f0 con 0x7f1fb4073560 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7ffb917fa640 1 -- 10.20.193.16:0/1443122777 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7ffb980419c0 con 0x7ffb9c14fec0 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1fb8e05640 10 client.4434.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1fb8e05640 10 monclient: _renew_subs 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1fb8e05640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1fb8e05640 1 -- 10.20.193.16:0/2467955324 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb40c66b0 con 0x7f1fb4073560 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.419+0000 7f1fb8e05640 1 librados: init done 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 1 -- 10.20.193.16:0/2807428913 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68017070 con 0x7fed74155e00 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 10 monclient: dump: 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.434 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 1 -- 10.20.193.16:0/2807428913 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed68005bb0 con 0x7fed74155e00 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 1 -- 10.20.193.16:0/2807428913 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6800a640 con 0x7fed74155e00 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed627fc640 10 monclient: dump: 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed63fff640 4 set_mon_vals no callback set 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed7bcb7640 5 monclient: authenticate success, global_id 4437 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed7bcb7640 10 monclient: _renew_subs 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed7bcb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed7bcb7640 1 -- 10.20.193.16:0/2807428913 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7415d630 con 0x7fed74155e00 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed7bcb7640 10 client.4437.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed7bcb7640 10 monclient: _renew_subs 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed7bcb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed7bcb7640 1 -- 10.20.193.16:0/2807428913 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7415dbf0 con 0x7fed74155e00 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7fed7bcb7640 1 librados: init done 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7ffb917fa640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffb6c079140 0x7ffb6c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7f8363961640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7f8363961640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7f8363961640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7f8363961640 1 -- 10.20.193.16:0/944886098 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835c0c8f50 con 0x7f835c155e00 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7ffba0e0b640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffb6c079140 0x7ffb6c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7ffba0e0b640 10 monclient: get_auth_request con 0x7ffb6c079140 auth_method 0 2026-02-17T05:56:47.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7f1fb8e05640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7f1fb8e05640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7f1fb8e05640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7f1fb8e05640 1 -- 10.20.193.16:0/2467955324 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb40c5280 con 0x7f1fb4073560 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7ffba0e0b640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffb6c079140 0x7ffb6c07b600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009750 tx=0x7ffb8c006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.420+0000 7ffb917fa640 10 client.4547.objecter ms_handle_connect 0x7ffb6c079140 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.421+0000 7fed7bcb7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.421+0000 7fed7bcb7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.421+0000 7fed7bcb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.421+0000 7fed7bcb7640 1 -- 10.20.193.16:0/2807428913 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed740ca700 con 0x7fed74155e00 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.422+0000 7ffb917fa640 1 -- 10.20.193.16:0/1443122777 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7ffb98083dc0 con 0x7ffb9c14fec0 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.422+0000 7ffb917fa640 10 client.4547.objecter ms_dispatch2 0x7ffb9c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.422+0000 7ffb917fa640 3 client.4547.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.422+0000 7ffb917fa640 3 client.4547.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.422+0000 7ffb917fa640 20 client.4547.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7ffb917fa640 1 -- 10.20.193.16:0/1443122777 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ffb9c0698c0 con 0x7ffb9c14fec0 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7ffb917fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7ffb917fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 1 -- 10.20.193.16:0/3509700266 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4013020 con 0x7fbfc00745e0 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 10 monclient: dump: 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.438 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 1 -- 10.20.193.16:0/3509700266 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb40044a0 con 0x7fbfc00745e0 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 1 -- 10.20.193.16:0/3509700266 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40029c0 con 0x7fbfc00745e0 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfc4d49640 5 monclient: authenticate success, global_id 4556 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfbcd72640 4 set_mon_vals no callback set 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfa6ffd640 10 monclient: dump: 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfc4d49640 10 monclient: _renew_subs 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfc4d49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfc4d49640 1 -- 10.20.193.16:0/3509700266 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc0200f00 con 0x7fbfc00745e0 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfc4d49640 10 client.4556.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfc4d49640 10 monclient: _renew_subs 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfc4d49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfc4d49640 1 -- 10.20.193.16:0/3509700266 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc02014c0 con 0x7fbfc00745e0 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.429+0000 7fbfc4d49640 1 librados: init done 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.430+0000 7fbfc4d49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.430+0000 7fbfc4d49640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.430+0000 7fbfc4d49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.430+0000 7fbfc4d49640 1 -- 10.20.193.16:0/3509700266 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc00740a0 con 0x7fbfc00745e0 2026-02-17T05:56:47.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7fd0236fa640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:56:47.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7fd0236fa640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:56:47.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7fd0236fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7fd0236fa640 1 -- 10.20.193.16:0/3225517002 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd01c159160 con 0x7fd01c0c8050 2026-02-17T05:56:47.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7f7406d49640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:56:47.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7f7406d49640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:56:47.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7f7406d49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7f7406d49640 1 -- 10.20.193.16:0/2132433469 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7400076e00 con 0x7f7400155e00 2026-02-17T05:56:47.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7f021586f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:56:47.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7f021586f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:56:47.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7f021586f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.499+0000 7f021586f640 1 -- 10.20.193.16:0/2582501118 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0210074140 con 0x7f0210074dc0 2026-02-17T05:56:47.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 1 -- 10.20.193.16:0/3024630734 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8015070 con 0x7f46f8155e00 2026-02-17T05:56:47.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 10 monclient: dump: 2026-02-17T05:56:47.514 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.514 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.514 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.514 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 1 -- 10.20.193.16:0/3024630734 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e8004220 con 0x7f46f8155e00 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 1 -- 10.20.193.16:0/3024630734 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8007830 con 0x7f46f8155e00 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f4ff9640 10 monclient: dump: 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46f67fc640 4 set_mon_vals no callback set 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46fe751640 5 monclient: authenticate success, global_id 4507 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46fe751640 10 monclient: _renew_subs 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46fe751640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.513+0000 7f46fe751640 1 -- 10.20.193.16:0/3024630734 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f80cfb30 con 0x7f46f8155e00 2026-02-17T05:56:47.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f46fe751640 10 client.4507.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f46fe751640 10 monclient: _renew_subs 2026-02-17T05:56:47.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f46fe751640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f46fe751640 1 -- 10.20.193.16:0/3024630734 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f80d00f0 con 0x7f46f8155e00 2026-02-17T05:56:47.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f46fe751640 1 librados: init done 2026-02-17T05:56:47.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7faa46ffd640 1 -- 10.20.193.16:0/1476379032 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faa5c021020 con 0x7faa6015d360 2026-02-17T05:56:47.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7faa46ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:56:47.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7faa46ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.517 INFO:teuthology.orchestra.run.trial016.stdout:120259084294 2026-02-17T05:56:47.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 1 -- 10.20.193.16:0/4188371468 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c016020 con 0x7f478c1561e0 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 10 monclient: dump: 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 1 -- 10.20.193.16:0/4188371468 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477c012420 con 0x7f478c1561e0 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 1 -- 10.20.193.16:0/4188371468 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c00ce60 con 0x7f478c1561e0 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f476b7fe640 10 monclient: dump: 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:47.520 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f47897fa640 4 set_mon_vals no callback set 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f479167e640 5 monclient: authenticate success, global_id 4510 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f479167e640 10 monclient: _renew_subs 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f479167e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f479167e640 1 -- 10.20.193.16:0/4188371468 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478c14df20 con 0x7f478c1561e0 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f479167e640 10 client.4510.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f479167e640 10 monclient: _renew_subs 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f479167e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f479167e640 1 -- 10.20.193.16:0/4188371468 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f478c14e4e0 con 0x7f478c1561e0 2026-02-17T05:56:47.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f479167e640 1 librados: init done 2026-02-17T05:56:47.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f46fe751640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f46fe751640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f46fe751640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.514+0000 7f46fe751640 1 -- 10.20.193.16:0/3024630734 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f80c8fa0 con 0x7f46f8155e00 2026-02-17T05:56:47.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.515+0000 7f479167e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.515+0000 7f479167e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.515+0000 7f479167e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.515+0000 7f479167e640 1 -- 10.20.193.16:0/4188371468 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f478c0c8fc0 con 0x7f478c1561e0 2026-02-17T05:56:47.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 10 librados: watch_flush enter 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 10 librados: watch_flush exit 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 20 client.4477.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 10 client.4477.objecter successfully canceled tick 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 -- 10.20.193.16:0/1476379032 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faa34079190 msgr2=0x7faa3407b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faa34079190 0x7faa3407b650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa50039000 tx=0x7faa50031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 10 monclient: shutdown 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 -- 10.20.193.16:0/1476379032 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa6015d360 msgr2=0x7faa601f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa6015d360 0x7faa601f4c90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00cbb0 tx=0x7faa5c00a430 comp rx=0 tx=0).stop 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 -- 10.20.193.16:0/1476379032 shutdown_connections 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7faa34079190 0x7faa3407b650 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7faa6015d360 0x7faa601f4c90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7faa60156380 0x7faa6014bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 --2- 10.20.193.16:0/1476379032 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7faa60155fa0 0x7faa600ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 -- 10.20.193.16:0/1476379032 >> 10.20.193.16:0/1476379032 conn(0x7faa600ce6c0 msgr2=0x7faa601537c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 -- 10.20.193.16:0/1476379032 shutdown_connections 2026-02-17T05:56:47.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 -- 10.20.193.16:0/1476379032 wait complete. 2026-02-17T05:56:47.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.516+0000 7faa6844c640 1 librados: shutdown 2026-02-17T05:56:47.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.517+0000 7ffba3096640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-17T05:56:47.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.517+0000 7ffba3096640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-17T05:56:47.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.517+0000 7ffba3096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.517+0000 7ffba3096640 1 -- 10.20.193.16:0/1443122777 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ffb9c1e7960 con 0x7ffb9c14fec0 2026-02-17T05:56:47.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.521+0000 7f30d5ffb640 1 -- 10.20.193.16:0/770118070 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f30dc002b60 con 0x7f30e81561e0 2026-02-17T05:56:47.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.522+0000 7f30d5ffb640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b8079190 0x7f30b807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.522+0000 7f30ed729640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b8079190 0x7f30b807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.522+0000 7f30ed729640 10 monclient: get_auth_request con 0x7f30b8079190 auth_method 0 2026-02-17T05:56:47.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.522+0000 7f30d5ffb640 10 client.4501.objecter ms_handle_connect 0x7f30b8079190 2026-02-17T05:56:47.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.522+0000 7f30ed729640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b8079190 0x7f30b807b650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30d8007fd0 tx=0x7f30d8007b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.523+0000 7f30d5ffb640 1 -- 10.20.193.16:0/770118070 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f30dc05e270 con 0x7f30e81561e0 2026-02-17T05:56:47.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.523+0000 7f30d5ffb640 10 client.4501.objecter ms_dispatch2 0x7f30e8000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.523+0000 7f30d5ffb640 3 client.4501.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.523+0000 7f30d5ffb640 3 client.4501.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.523+0000 7f30d5ffb640 20 client.4501.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084295 got 120259084294 for osd.0 2026-02-17T05:56:47.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.530+0000 7f30d5ffb640 1 -- 10.20.193.16:0/770118070 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f30dc01d020 con 0x7f30e81561e0 2026-02-17T05:56:47.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.530+0000 7f30d5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.530+0000 7f30d5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.535+0000 7f01e7fff640 1 -- 10.20.193.16:0/2582501118 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02040629a0 con 0x7f0210074dc0 2026-02-17T05:56:47.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.535+0000 7f01e7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:56:47.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.535+0000 7f01e7fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.536 INFO:teuthology.orchestra.run.trial016.stdout:98784247811 2026-02-17T05:56:47.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f1f9affd640 1 -- 10.20.193.16:0/2467955324 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1fa40428d0 con 0x7f1fb4073560 2026-02-17T05:56:47.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 10 librados: watch_flush enter 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 10 librados: watch_flush exit 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 20 client.4504.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 10 client.4504.objecter successfully canceled tick 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 1 -- 10.20.193.16:0/2582501118 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01e0079190 msgr2=0x7f01e007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01e0079190 0x7f01e007b650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f021006ce20 tx=0x7f01f8031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 10 monclient: shutdown 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 1 -- 10.20.193.16:0/2582501118 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0210074dc0 msgr2=0x7f021014e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0210074dc0 0x7f021014e110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f020400b570 tx=0x7f020400ba40 comp rx=0 tx=0).stop 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 1 -- 10.20.193.16:0/2582501118 shutdown_connections 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01e0079190 0x7f01e007b650 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f02101561e0 0x7f02100c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0210155e00 0x7f0210153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 1 --2- 10.20.193.16:0/2582501118 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0210074dc0 0x7f021014e110 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.537+0000 7f021586f640 1 -- 10.20.193.16:0/2582501118 >> 10.20.193.16:0/2582501118 conn(0x7f02100ce6c0 msgr2=0x7f02100c8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.538+0000 7f021586f640 1 -- 10.20.193.16:0/2582501118 shutdown_connections 2026-02-17T05:56:47.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.538+0000 7f021586f640 1 -- 10.20.193.16:0/2582501118 wait complete. 2026-02-17T05:56:47.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.538+0000 7f021586f640 1 librados: shutdown 2026-02-17T05:56:47.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.538+0000 7f1f9affd640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f800791c0 0x7f1f8007b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.538+0000 7f1fb2575640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f800791c0 0x7f1f8007b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.538+0000 7f1fb2575640 10 monclient: get_auth_request con 0x7f1f800791c0 auth_method 0 2026-02-17T05:56:47.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.538+0000 7f1f9affd640 10 client.4434.objecter ms_handle_connect 0x7f1f800791c0 2026-02-17T05:56:47.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.538+0000 7f1fb2575640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f800791c0 0x7f1f8007b680 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009b20 tx=0x7f1fa80095e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.541+0000 7f1f9affd640 1 -- 10.20.193.16:0/2467955324 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f1fa4083ed0 con 0x7f1fb4073560 2026-02-17T05:56:47.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.541+0000 7f1f9affd640 10 client.4434.objecter ms_dispatch2 0x7f1fb4000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.541+0000 7f1f9affd640 3 client.4434.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.541+0000 7f1f9affd640 3 client.4434.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.541+0000 7f1f9affd640 20 client.4434.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.546 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.3 2026-02-17T05:56:47.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.549+0000 7fed627fc640 1 -- 10.20.193.16:0/2807428913 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fed6800a7e0 con 0x7fed74155e00 2026-02-17T05:56:47.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.549+0000 7fed627fc640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fed44079190 0x7fed4407b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.549+0000 7fed79a2c640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fed44079190 0x7fed4407b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.549+0000 7fed79a2c640 10 monclient: get_auth_request con 0x7fed44079190 auth_method 0 2026-02-17T05:56:47.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.550+0000 7fed627fc640 10 client.4437.objecter ms_handle_connect 0x7fed44079190 2026-02-17T05:56:47.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.550+0000 7fed79a2c640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fed44079190 0x7fed4407b650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed64039000 tx=0x7fed64031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.557+0000 7fed627fc640 1 -- 10.20.193.16:0/2807428913 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fed6805d6a0 con 0x7fed74155e00 2026-02-17T05:56:47.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.557+0000 7fed627fc640 10 client.4437.objecter ms_dispatch2 0x7fed74000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.557+0000 7fed627fc640 3 client.4437.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.557+0000 7fed627fc640 3 client.4437.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.557+0000 7fed627fc640 20 client.4437.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.557+0000 7fed627fc640 1 -- 10.20.193.16:0/2807428913 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fed68021020 con 0x7fed74155e00 2026-02-17T05:56:47.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.557+0000 7fed627fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.557+0000 7fed627fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.616+0000 7f30ef9b4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-17T05:56:47.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.616+0000 7f30ef9b4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-17T05:56:47.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.616+0000 7f30ef9b4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.616+0000 7f30ef9b4640 1 -- 10.20.193.16:0/770118070 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 12} v 0) -- 0x7f30e8159730 con 0x7f30e81561e0 2026-02-17T05:56:47.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.645+0000 7fed7bcb7640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-17T05:56:47.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.645+0000 7fed7bcb7640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-17T05:56:47.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.645+0000 7fed7bcb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.645+0000 7fed7bcb7640 1 -- 10.20.193.16:0/2807428913 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7fed741601e0 con 0x7fed74155e00 2026-02-17T05:56:47.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.664+0000 7fd009ffb640 1 -- 10.20.193.16:0/3225517002 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd00c065eb0 con 0x7fd01c0c8050 2026-02-17T05:56:47.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.664+0000 7fd009ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:56:47.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.664+0000 7fd009ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.665 INFO:teuthology.orchestra.run.trial016.stdout:115964116999 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 10 librados: watch_flush enter 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 10 librados: watch_flush exit 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 20 client.4428.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 10 client.4428.objecter successfully canceled tick 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 -- 10.20.193.16:0/3225517002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcff8079190 msgr2=0x7fcff807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcff8079190 0x7fcff807b650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd01c14b1b0 tx=0x7fd010005c40 comp rx=0 tx=0).stop 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 10 monclient: shutdown 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 -- 10.20.193.16:0/3225517002 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd01c0c8050 msgr2=0x7fd01c1f4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd01c0c8050 0x7fd01c1f4cd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00b5a0 tx=0x7fd00c00ba70 comp rx=0 tx=0).stop 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 -- 10.20.193.16:0/3225517002 shutdown_connections 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcff8079190 0x7fcff807b650 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd01c14e480 0x7fd01c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd01c14e860 0x7fd01c0701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 --2- 10.20.193.16:0/3225517002 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd01c0c8050 0x7fd01c1f4cd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 -- 10.20.193.16:0/3225517002 >> 10.20.193.16:0/3225517002 conn(0x7fd01c0ca150 msgr2=0x7fd01c159960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 -- 10.20.193.16:0/3225517002 shutdown_connections 2026-02-17T05:56:47.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 -- 10.20.193.16:0/3225517002 wait complete. 2026-02-17T05:56:47.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.666+0000 7fd0236fa640 1 librados: shutdown 2026-02-17T05:56:47.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.671+0000 7f1f9affd640 1 -- 10.20.193.16:0/2467955324 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1fb40c5280 con 0x7f1fb4073560 2026-02-17T05:56:47.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.671+0000 7f1f9affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.671+0000 7f1f9affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.675+0000 7fed627fc640 1 -- 10.20.193.16:0/2807428913 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fed74074860 con 0x7fed74155e00 2026-02-17T05:56:47.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.675+0000 7fed627fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-17T05:56:47.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.675+0000 7fed627fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.675 INFO:teuthology.orchestra.run.trial016.stdout:94489280517 2026-02-17T05:56:47.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964117001 got 115964116999 for osd.5 2026-02-17T05:56:47.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 10 librados: watch_flush enter 2026-02-17T05:56:47.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 10 librados: watch_flush exit 2026-02-17T05:56:47.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 20 client.4437.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 10 client.4437.objecter successfully canceled tick 2026-02-17T05:56:47.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 -- 10.20.193.16:0/2807428913 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fed44079190 msgr2=0x7fed4407b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fed44079190 0x7fed4407b650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed64039000 tx=0x7fed64031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 10 monclient: shutdown 2026-02-17T05:56:47.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 -- 10.20.193.16:0/2807428913 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 msgr2=0x7fed7415e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 0x7fed7415e6d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c730 tx=0x7fed6800cc00 comp rx=0 tx=0).stop 2026-02-17T05:56:47.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 -- 10.20.193.16:0/2807428913 shutdown_connections 2026-02-17T05:56:47.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fed44079190 0x7fed4407b650 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fed741561e0 0x7fed740c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fed74155e00 0x7fed7415e6d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 --2- 10.20.193.16:0/2807428913 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fed74074dc0 0x7fed741f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 -- 10.20.193.16:0/2807428913 >> 10.20.193.16:0/2807428913 conn(0x7fed740ce6c0 msgr2=0x7fed740cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 -- 10.20.193.16:0/2807428913 shutdown_connections 2026-02-17T05:56:47.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 -- 10.20.193.16:0/2807428913 wait complete. 2026-02-17T05:56:47.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.677+0000 7fed7bcb7640 1 librados: shutdown 2026-02-17T05:56:47.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280519 got 94489280517 for osd.15 2026-02-17T05:56:47.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.741+0000 7f46f4ff9640 1 -- 10.20.193.16:0/3024630734 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f46e80079d0 con 0x7f46f8155e00 2026-02-17T05:56:47.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.742+0000 7f46f4ff9640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f46c8079190 0x7f46c807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.742+0000 7f46f7fff640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f46c8079190 0x7f46c807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.742+0000 7f46f7fff640 10 monclient: get_auth_request con 0x7f46c8079190 auth_method 0 2026-02-17T05:56:47.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.742+0000 7f46f7fff640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f46c8079190 0x7f46c807b650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46f806cbb0 tx=0x7f46e4031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.742+0000 7f46f4ff9640 10 client.4507.objecter ms_handle_connect 0x7f46c8079190 2026-02-17T05:56:47.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.744+0000 7f46f4ff9640 1 -- 10.20.193.16:0/3024630734 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f46e8007cf0 con 0x7f46f8155e00 2026-02-17T05:56:47.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.744+0000 7f46f4ff9640 10 client.4507.objecter ms_dispatch2 0x7f46f8000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.744+0000 7f46f4ff9640 3 client.4507.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.744+0000 7f46f4ff9640 3 client.4507.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.744+0000 7f46f4ff9640 20 client.4507.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.756+0000 7f1fb8e05640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:56:47.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.756+0000 7f1fb8e05640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:56:47.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.756+0000 7f1fb8e05640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:47.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.756+0000 7f1fb8e05640 1 -- 10.20.193.16:0/2467955324 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1fb4074a00 con 0x7f1fb4073560 2026-02-17T05:56:47.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.757+0000 7f46f4ff9640 1 -- 10.20.193.16:0/3024630734 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f46e8062660 con 0x7f46f8155e00 2026-02-17T05:56:47.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.757+0000 7f46f4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.757+0000 7f46f4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.758+0000 7f476b7fe640 1 -- 10.20.193.16:0/4188371468 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f477c012810 con 0x7f478c1561e0 2026-02-17T05:56:47.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.758+0000 7f476b7fe640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4758079190 0x7f475807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.759+0000 7f478affd640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4758079190 0x7f475807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.759+0000 7f478affd640 10 monclient: get_auth_request con 0x7f4758079190 auth_method 0 2026-02-17T05:56:47.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.759+0000 7f478affd640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4758079190 0x7f475807b650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f478c06cbb0 tx=0x7f4780031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.759+0000 7f476b7fe640 10 client.4510.objecter ms_handle_connect 0x7f4758079190 2026-02-17T05:56:47.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.760+0000 7f476b7fe640 1 -- 10.20.193.16:0/4188371468 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f477c05ec10 con 0x7f478c1561e0 2026-02-17T05:56:47.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.760+0000 7f476b7fe640 10 client.4510.objecter ms_dispatch2 0x7f478c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.760+0000 7f476b7fe640 3 client.4510.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.760+0000 7f476b7fe640 3 client.4510.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.760+0000 7f476b7fe640 20 client.4510.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.761+0000 7f1f9affd640 1 -- 10.20.193.16:0/2467955324 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1fb4074a00 con 0x7f1fb4073560 2026-02-17T05:56:47.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.761+0000 7f1f9affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:56:47.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.761+0000 7f1f9affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.762 INFO:teuthology.orchestra.run.trial016.stdout:90194313222 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.762+0000 7f1fb8e05640 10 librados: watch_flush enter 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 10 librados: watch_flush exit 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 20 client.4434.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 10 client.4434.objecter successfully canceled tick 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 -- 10.20.193.16:0/2467955324 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f800791c0 msgr2=0x7f1f8007b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f800791c0 0x7f1f8007b680 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009b20 tx=0x7f1fa80095e0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 10 monclient: shutdown 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 -- 10.20.193.16:0/2467955324 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fb4073560 msgr2=0x7f1fb4146880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fb4073560 0x7f1fb4146880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4002790 tx=0x7f1fa4003b70 comp rx=0 tx=0).stop 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 -- 10.20.193.16:0/2467955324 shutdown_connections 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f800791c0 0x7f1f8007b680 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1fb40c6eb0 0x7f1fb41f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1fb4073560 0x7f1fb4146880 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 --2- 10.20.193.16:0/2467955324 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1fa0005310 0x7f1fb41f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 -- 10.20.193.16:0/2467955324 >> 10.20.193.16:0/2467955324 conn(0x7f1fb40ca5b0 msgr2=0x7f1fb414c170 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 -- 10.20.193.16:0/2467955324 shutdown_connections 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 -- 10.20.193.16:0/2467955324 wait complete. 2026-02-17T05:56:47.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.763+0000 7f1fb8e05640 1 librados: shutdown 2026-02-17T05:56:47.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.766+0000 7f476b7fe640 1 -- 10.20.193.16:0/4188371468 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f477c020020 con 0x7f478c1561e0 2026-02-17T05:56:47.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.766+0000 7f476b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.766+0000 7f476b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.770+0000 7f30d5ffb640 1 -- 10.20.193.16:0/770118070 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 12}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f30dc062930 con 0x7f30e81561e0 2026-02-17T05:56:47.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.770+0000 7f30d5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-17T05:56:47.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.770+0000 7f30d5ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.770 INFO:teuthology.orchestra.run.trial016.stdout:120259084295 2026-02-17T05:56:47.772 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313223 got 90194313222 for osd.2 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 10 librados: watch_flush enter 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 10 librados: watch_flush exit 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 20 client.4501.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 10 client.4501.objecter successfully canceled tick 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 -- 10.20.193.16:0/770118070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b8079190 msgr2=0x7f30b807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b8079190 0x7f30b807b650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30d8007fd0 tx=0x7f30d8007b20 comp rx=0 tx=0).stop 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 10 monclient: shutdown 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 -- 10.20.193.16:0/770118070 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e81561e0 msgr2=0x7f30e814c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e81561e0 0x7f30e814c420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00b570 tx=0x7f30dc00ba40 comp rx=0 tx=0).stop 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 -- 10.20.193.16:0/770118070 shutdown_connections 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b8079190 0x7f30b807b650 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e81561e0 0x7f30e814c420 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e815fa10 0x7f30e8147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 --2- 10.20.193.16:0/770118070 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30e8155e00 0x7f30e81fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 -- 10.20.193.16:0/770118070 >> 10.20.193.16:0/770118070 conn(0x7f30e80ce6c0 msgr2=0x7f30e8153820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 -- 10.20.193.16:0/770118070 shutdown_connections 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.772+0000 7f30ef9b4640 1 -- 10.20.193.16:0/770118070 wait complete. 2026-02-17T05:56:47.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.773+0000 7f30ef9b4640 1 librados: shutdown 2026-02-17T05:56:47.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084296 got 120259084295 for osd.12 2026-02-17T05:56:47.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.826+0000 7f19457fa640 1 -- 10.20.193.16:0/3399919245 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f194001d400 con 0x7f193c0052c0 2026-02-17T05:56:47.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.826+0000 7f19457fa640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1920079140 0x7f192007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.826+0000 7f1947fff640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1920079140 0x7f192007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.826+0000 7f1947fff640 10 monclient: get_auth_request con 0x7f1920079140 auth_method 0 2026-02-17T05:56:47.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.827+0000 7f19457fa640 10 client.4541.objecter ms_handle_connect 0x7f1920079140 2026-02-17T05:56:47.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.827+0000 7f1947fff640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1920079140 0x7f192007b600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f19500cdd80 tx=0x7f1938009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.828+0000 7f19457fa640 1 -- 10.20.193.16:0/3399919245 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f194005eb10 con 0x7f193c0052c0 2026-02-17T05:56:47.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.828+0000 7f19457fa640 10 client.4541.objecter ms_dispatch2 0x7f1950000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.828+0000 7f19457fa640 3 client.4541.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.828+0000 7f19457fa640 3 client.4541.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.828+0000 7f19457fa640 20 client.4541.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.838+0000 7f19457fa640 1 -- 10.20.193.16:0/3399919245 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f194002c080 con 0x7f193c0052c0 2026-02-17T05:56:47.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.838+0000 7f19457fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.838+0000 7f19457fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.844+0000 7f46fe751640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-17T05:56:47.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.844+0000 7f46fe751640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-17T05:56:47.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.844+0000 7f46fe751640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.844+0000 7f46fe751640 1 -- 10.20.193.16:0/3024630734 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 13} v 0) -- 0x7f46f80c5e90 con 0x7f46f8155e00 2026-02-17T05:56:47.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.846+0000 7efc76ffd640 1 -- 10.20.193.16:0/1090734837 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7efc8403f050 con 0x7efc8c155fa0 2026-02-17T05:56:47.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.846+0000 7f46f4ff9640 1 -- 10.20.193.16:0/3024630734 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 13}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f46e8061f60 con 0x7f46f8155e00 2026-02-17T05:56:47.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.846+0000 7f46f4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-17T05:56:47.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.846+0000 7f46f4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.848 INFO:teuthology.orchestra.run.trial016.stdout:115964116996 2026-02-17T05:56:47.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.847+0000 7efc76ffd640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc5c079190 0x7efc5c07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.847+0000 7efc91e24640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc5c079190 0x7efc5c07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.847+0000 7efc91e24640 10 monclient: get_auth_request con 0x7efc5c079190 auth_method 0 2026-02-17T05:56:47.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.847+0000 7efc76ffd640 10 client.4544.objecter ms_handle_connect 0x7efc5c079190 2026-02-17T05:56:47.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.847+0000 7efc91e24640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc5c079190 0x7efc5c07b650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc78006fd0 tx=0x7efc78006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 10 librados: watch_flush enter 2026-02-17T05:56:47.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 10 librados: watch_flush exit 2026-02-17T05:56:47.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 20 client.4507.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 10 client.4507.objecter successfully canceled tick 2026-02-17T05:56:47.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 -- 10.20.193.16:0/3024630734 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f46c8079190 msgr2=0x7f46c807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f46c8079190 0x7f46c807b650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46f806cbb0 tx=0x7f46e4031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 10 monclient: shutdown 2026-02-17T05:56:47.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 -- 10.20.193.16:0/3024630734 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f46f8155e00 msgr2=0x7f46f80d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f46f8155e00 0x7f46f80d0ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c7f0 tx=0x7f46e800ccc0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 -- 10.20.193.16:0/3024630734 shutdown_connections 2026-02-17T05:56:47.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f46c8079190 0x7f46c807b650 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f46f8155e00 0x7f46f80d0ed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f46f81561e0 0x7f46f814d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 --2- 10.20.193.16:0/3024630734 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f46f8074dc0 0x7f46f81f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 -- 10.20.193.16:0/3024630734 >> 10.20.193.16:0/3024630734 conn(0x7f46f80ce6c0 msgr2=0x7f46f80ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 -- 10.20.193.16:0/3024630734 shutdown_connections 2026-02-17T05:56:47.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 -- 10.20.193.16:0/3024630734 wait complete. 2026-02-17T05:56:47.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.848+0000 7f46fe751640 1 librados: shutdown 2026-02-17T05:56:47.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.853+0000 7f479167e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-17T05:56:47.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.853+0000 7f479167e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-17T05:56:47.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.853+0000 7f479167e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:47.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.853+0000 7f479167e640 1 -- 10.20.193.16:0/4188371468 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f478c0cc520 con 0x7f478c1561e0 2026-02-17T05:56:47.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.855+0000 7f476b7fe640 1 -- 10.20.193.16:0/4188371468 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f477c0102e0 con 0x7f478c1561e0 2026-02-17T05:56:47.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.855+0000 7f476b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-17T05:56:47.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.855+0000 7f476b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.856 INFO:teuthology.orchestra.run.trial016.stdout:90194313219 2026-02-17T05:56:47.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116998 got 115964116996 for osd.13 2026-02-17T05:56:47.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 10 librados: watch_flush enter 2026-02-17T05:56:47.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 10 librados: watch_flush exit 2026-02-17T05:56:47.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 20 client.4510.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 10 client.4510.objecter successfully canceled tick 2026-02-17T05:56:47.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 -- 10.20.193.16:0/4188371468 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4758079190 msgr2=0x7f475807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4758079190 0x7f475807b650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f478c06cbb0 tx=0x7f4780031000 comp rx=0 tx=0).stop 2026-02-17T05:56:47.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 10 monclient: shutdown 2026-02-17T05:56:47.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 -- 10.20.193.16:0/4188371468 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c1561e0 msgr2=0x7f478c143ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c1561e0 0x7f478c143ee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f477c00cc40 tx=0x7f477c00a430 comp rx=0 tx=0).stop 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 -- 10.20.193.16:0/4188371468 shutdown_connections 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4758079190 0x7f475807b650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f478c1561e0 0x7f478c143ee0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f478c155e00 0x7f478c14d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 --2- 10.20.193.16:0/4188371468 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f478c074dc0 0x7f478c14d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 -- 10.20.193.16:0/4188371468 >> 10.20.193.16:0/4188371468 conn(0x7f478c0ce6c0 msgr2=0x7f478c06d470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 -- 10.20.193.16:0/4188371468 shutdown_connections 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 -- 10.20.193.16:0/4188371468 wait complete. 2026-02-17T05:56:47.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.857+0000 7f479167e640 1 librados: shutdown 2026-02-17T05:56:47.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313219 for osd.6 2026-02-17T05:56:47.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.922+0000 7f1956b79640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-17T05:56:47.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.923+0000 7f1956b79640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-17T05:56:47.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.923+0000 7f1956b79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:47.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.923+0000 7f1956b79640 1 -- 10.20.193.16:0/3399919245 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 14} v 0) -- 0x7f1950159730 con 0x7f193c0052c0 2026-02-17T05:56:47.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.944+0000 7efc76ffd640 1 -- 10.20.193.16:0/1090734837 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7efc84083050 con 0x7efc8c155fa0 2026-02-17T05:56:47.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.944+0000 7efc76ffd640 10 client.4544.objecter ms_dispatch2 0x7efc8c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.944+0000 7efc76ffd640 3 client.4544.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.944+0000 7efc76ffd640 3 client.4544.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.944+0000 7efc76ffd640 20 client.4544.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.951+0000 7efc76ffd640 1 -- 10.20.193.16:0/1090734837 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efc84087710 con 0x7efc8c155fa0 2026-02-17T05:56:47.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.951+0000 7efc76ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.951+0000 7efc76ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.959+0000 7f8349ffb640 1 -- 10.20.193.16:0/944886098 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f8350019460 con 0x7f835c155e00 2026-02-17T05:56:47.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.959+0000 7f8349ffb640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f832c079190 0x7f832c07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.960+0000 7f83616d6640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f832c079190 0x7f832c07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.960+0000 7f83616d6640 10 monclient: get_auth_request con 0x7f832c079190 auth_method 0 2026-02-17T05:56:47.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.960+0000 7f83616d6640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f832c079190 0x7f832c07b650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f834c006fd0 tx=0x7f834c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.960+0000 7f8349ffb640 10 client.4550.objecter ms_handle_connect 0x7f832c079190 2026-02-17T05:56:47.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.966+0000 7f97d37fe640 1 -- 10.20.193.16:0/1259493090 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f97e8003a50 con 0x7f97f4074dc0 2026-02-17T05:56:47.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.966+0000 7f97d37fe640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97c00791c0 0x7f97c007b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:47.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.966+0000 7f97f27fc640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97c00791c0 0x7f97c007b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:47.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.966+0000 7f97f27fc640 10 monclient: get_auth_request con 0x7f97c00791c0 auth_method 0 2026-02-17T05:56:47.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.966+0000 7f97d37fe640 10 client.4553.objecter ms_handle_connect 0x7f97c00791c0 2026-02-17T05:56:47.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.966+0000 7f97f27fc640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97c00791c0 0x7f97c007b680 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97dc039000 tx=0x7f97dc031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:47.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.968+0000 7f8349ffb640 1 -- 10.20.193.16:0/944886098 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f835005e070 con 0x7f835c155e00 2026-02-17T05:56:47.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.968+0000 7f8349ffb640 10 client.4550.objecter ms_dispatch2 0x7f835c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.968+0000 7f8349ffb640 3 client.4550.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.968+0000 7f8349ffb640 3 client.4550.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.968+0000 7f8349ffb640 20 client.4550.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.977+0000 7f97d37fe640 1 -- 10.20.193.16:0/1259493090 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f97e805e3d0 con 0x7f97f4074dc0 2026-02-17T05:56:47.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.977+0000 7f97d37fe640 10 client.4553.objecter ms_dispatch2 0x7f97f4000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:47.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.977+0000 7f97d37fe640 3 client.4553.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:47.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.977+0000 7f97d37fe640 3 client.4553.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:47.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.977+0000 7f97d37fe640 20 client.4553.objecter dump_active .. 0 homeless 2026-02-17T05:56:47.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.980+0000 7f8349ffb640 1 -- 10.20.193.16:0/944886098 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8350062730 con 0x7f835c155e00 2026-02-17T05:56:47.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.980+0000 7f8349ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.980+0000 7f8349ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.987+0000 7f73f57fa640 1 -- 10.20.193.16:0/2132433469 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7400076e00 con 0x7f7400155e00 2026-02-17T05:56:47.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.987+0000 7f73f57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:56:47.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.987+0000 7f73f57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.988 INFO:teuthology.orchestra.run.trial016.stdout:98784247812 2026-02-17T05:56:47.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.987+0000 7f97d37fe640 1 -- 10.20.193.16:0/1259493090 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f97f40cbb60 con 0x7f97f4074dc0 2026-02-17T05:56:47.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.987+0000 7f97d37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:47.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.987+0000 7f97d37fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:47.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.988+0000 7ffb917fa640 1 -- 10.20.193.16:0/1443122777 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7ffb98050030 con 0x7ffb9c14fec0 2026-02-17T05:56:47.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.988+0000 7ffb917fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-17T05:56:47.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.988+0000 7ffb917fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:47.990 INFO:teuthology.orchestra.run.trial016.stdout:90194313222 2026-02-17T05:56:47.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 10 librados: watch_flush enter 2026-02-17T05:56:47.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 10 librados: watch_flush exit 2026-02-17T05:56:47.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 20 client.4535.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 10 client.4535.objecter successfully canceled tick 2026-02-17T05:56:47.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 -- 10.20.193.16:0/2132433469 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f73d0079190 msgr2=0x7f73d007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f73d0079190 0x7f73d007b650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f73f00098a0 tx=0x7f73f0006d90 comp rx=0 tx=0).stop 2026-02-17T05:56:47.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 10 monclient: shutdown 2026-02-17T05:56:47.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 -- 10.20.193.16:0/2132433469 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7400155e00 msgr2=0x7f74001e9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7400155e00 0x7f74001e9db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009a40 tx=0x7f73e80047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 -- 10.20.193.16:0/2132433469 shutdown_connections 2026-02-17T05:56:47.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f73d0079190 0x7f73d007b650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f74001561e0 0x7f740013eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7400155e00 0x7f74001e9db0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 --2- 10.20.193.16:0/2132433469 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7400074dc0 0x7f74001500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 -- 10.20.193.16:0/2132433469 >> 10.20.193.16:0/2132433469 conn(0x7f74000ce6c0 msgr2=0x7f740006dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 -- 10.20.193.16:0/2132433469 shutdown_connections 2026-02-17T05:56:47.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 -- 10.20.193.16:0/2132433469 wait complete. 2026-02-17T05:56:47.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.989+0000 7f7406d49640 1 librados: shutdown 2026-02-17T05:56:47.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 10 librados: watch_flush enter 2026-02-17T05:56:47.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 10 librados: watch_flush exit 2026-02-17T05:56:47.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 20 client.4547.objecter shutdown clearing up homeless session... 2026-02-17T05:56:47.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 10 client.4547.objecter successfully canceled tick 2026-02-17T05:56:47.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 -- 10.20.193.16:0/1443122777 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffb6c079140 msgr2=0x7ffb6c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffb6c079140 0x7ffb6c07b600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009750 tx=0x7ffb8c006ca0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 10 monclient: shutdown 2026-02-17T05:56:47.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:47.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 -- 10.20.193.16:0/1443122777 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffb9c14fec0 msgr2=0x7ffb9c1555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:47.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffb9c14fec0 0x7ffb9c1555c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb98039040 tx=0x7ffb98004b40 comp rx=0 tx=0).stop 2026-02-17T05:56:47.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 -- 10.20.193.16:0/1443122777 shutdown_connections 2026-02-17T05:56:47.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ffb6c079140 0x7ffb6c07b600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffb9c14fec0 0x7ffb9c1555c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffb9c0c73a0 0x7ffb9c14c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 --2- 10.20.193.16:0/1443122777 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffb9c0c6fc0 0x7ffb9c158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:47.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 -- 10.20.193.16:0/1443122777 >> 10.20.193.16:0/1443122777 conn(0x7ffb9c0cd770 msgr2=0x7ffb9c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:47.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 -- 10.20.193.16:0/1443122777 shutdown_connections 2026-02-17T05:56:47.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 -- 10.20.193.16:0/1443122777 wait complete. 2026-02-17T05:56:47.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:47.990+0000 7ffba3096640 1 librados: shutdown 2026-02-17T05:56:47.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.1 2026-02-17T05:56:48.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313223 got 90194313222 for osd.10 2026-02-17T05:56:48.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.004+0000 7fbfa6ffd640 1 -- 10.20.193.16:0/3509700266 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fbfb4002b60 con 0x7fbfc00745e0 2026-02-17T05:56:48.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.004+0000 7fbfa6ffd640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbf8c079190 0x7fbf8c07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.005+0000 7fbfbdd74640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbf8c079190 0x7fbf8c07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.005+0000 7fbfbdd74640 10 monclient: get_auth_request con 0x7fbf8c079190 auth_method 0 2026-02-17T05:56:48.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.005+0000 7fbfbdd74640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbf8c079190 0x7fbf8c07b650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbfc008e730 tx=0x7fbfa8031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.005+0000 7fbfa6ffd640 10 client.4556.objecter ms_handle_connect 0x7fbf8c079190 2026-02-17T05:56:48.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.006+0000 7fbfa6ffd640 1 -- 10.20.193.16:0/3509700266 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fbfb405e330 con 0x7fbfc00745e0 2026-02-17T05:56:48.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.006+0000 7fbfa6ffd640 10 client.4556.objecter ms_dispatch2 0x7fbfc0000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:48.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.006+0000 7fbfa6ffd640 3 client.4556.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:48.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.006+0000 7fbfa6ffd640 3 client.4556.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:48.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.006+0000 7fbfa6ffd640 20 client.4556.objecter dump_active .. 0 homeless 2026-02-17T05:56:48.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.012+0000 7fbfa6ffd640 1 -- 10.20.193.16:0/3509700266 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbfb401d020 con 0x7fbfc00745e0 2026-02-17T05:56:48.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.012+0000 7fbfa6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.012+0000 7fbfa6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:48.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.018+0000 7f19457fa640 1 -- 10.20.193.16:0/3399919245 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 14}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1940019030 con 0x7f193c0052c0 2026-02-17T05:56:48.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.018+0000 7f19457fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-17T05:56:48.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.018+0000 7f19457fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:48.019 INFO:teuthology.orchestra.run.trial016.stdout:107374182408 2026-02-17T05:56:48.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.020+0000 7f1956b79640 10 librados: watch_flush enter 2026-02-17T05:56:48.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.020+0000 7f1956b79640 10 librados: watch_flush exit 2026-02-17T05:56:48.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.020+0000 7f1956b79640 20 client.4541.objecter shutdown clearing up homeless session... 2026-02-17T05:56:48.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.020+0000 7f1956b79640 10 client.4541.objecter successfully canceled tick 2026-02-17T05:56:48.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.020+0000 7f1956b79640 1 -- 10.20.193.16:0/3399919245 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1920079140 msgr2=0x7f192007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.020+0000 7f1956b79640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1920079140 0x7f192007b600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f19500cdd80 tx=0x7f1938009040 comp rx=0 tx=0).stop 2026-02-17T05:56:48.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 10 monclient: shutdown 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 -- 10.20.193.16:0/3399919245 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c0052c0 msgr2=0x7f19501f78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c0052c0 0x7f19501f78f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f19400119f0 tx=0x7f1940011ec0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 -- 10.20.193.16:0/3399919245 shutdown_connections 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1920079140 0x7f192007b600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1950142130 0x7f19501f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f195013e2f0 0x7f19500c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 --2- 10.20.193.16:0/3399919245 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f193c0052c0 0x7f19501f78f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 -- 10.20.193.16:0/3399919245 >> 10.20.193.16:0/3399919245 conn(0x7f19500ce6c0 msgr2=0x7f19500cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 -- 10.20.193.16:0/3399919245 shutdown_connections 2026-02-17T05:56:48.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 -- 10.20.193.16:0/3399919245 wait complete. 2026-02-17T05:56:48.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.021+0000 7f1956b79640 1 librados: shutdown 2026-02-17T05:56:48.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182409 got 107374182408 for osd.14 2026-02-17T05:56:48.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.039+0000 7efc948b0640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-17T05:56:48.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.039+0000 7efc948b0640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-17T05:56:48.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.039+0000 7efc948b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.039+0000 7efc948b0640 1 -- 10.20.193.16:0/1090734837 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7efc8c0c4f80 con 0x7efc8c155fa0 2026-02-17T05:56:48.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.057+0000 7efc76ffd640 1 -- 10.20.193.16:0/1090734837 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc8c0c4f80 con 0x7efc8c155fa0 2026-02-17T05:56:48.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.057+0000 7efc76ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-17T05:56:48.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.057+0000 7efc76ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:48.059 INFO:teuthology.orchestra.run.trial016.stdout:98784247811 2026-02-17T05:56:48.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 10 librados: watch_flush enter 2026-02-17T05:56:48.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 10 librados: watch_flush exit 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 20 client.4544.objecter shutdown clearing up homeless session... 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 10 client.4544.objecter successfully canceled tick 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 -- 10.20.193.16:0/1090734837 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc5c079190 msgr2=0x7efc5c07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc5c079190 0x7efc5c07b650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc78006fd0 tx=0x7efc78006d40 comp rx=0 tx=0).stop 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 10 monclient: shutdown 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 -- 10.20.193.16:0/1090734837 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efc8c155fa0 msgr2=0x7efc8c1f7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efc8c155fa0 0x7efc8c1f7000 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efc8c08f0e0 tx=0x7efc840047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 -- 10.20.193.16:0/1090734837 shutdown_connections 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc5c079190 0x7efc5c07b650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc8c15d360 0x7efc8c1f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc8c156380 0x7efc8c15f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 --2- 10.20.193.16:0/1090734837 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efc8c155fa0 0x7efc8c1f7000 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 -- 10.20.193.16:0/1090734837 >> 10.20.193.16:0/1090734837 conn(0x7efc8c0ce6c0 msgr2=0x7efc8c0ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 -- 10.20.193.16:0/1090734837 shutdown_connections 2026-02-17T05:56:48.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 -- 10.20.193.16:0/1090734837 wait complete. 2026-02-17T05:56:48.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.060+0000 7efc948b0640 1 librados: shutdown 2026-02-17T05:56:48.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.9 2026-02-17T05:56:48.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.071+0000 7f8363961640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:56:48.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.071+0000 7f8363961640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:56:48.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.071+0000 7f8363961640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.071+0000 7f8363961640 1 -- 10.20.193.16:0/944886098 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f835c06e120 con 0x7f835c155e00 2026-02-17T05:56:48.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.073+0000 7f8349ffb640 1 -- 10.20.193.16:0/944886098 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8350062030 con 0x7f835c155e00 2026-02-17T05:56:48.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.074+0000 7f8349ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:56:48.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.074+0000 7f8349ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:48.074 INFO:teuthology.orchestra.run.trial016.stdout:107374182404 2026-02-17T05:56:48.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.075+0000 7f8363961640 10 librados: watch_flush enter 2026-02-17T05:56:48.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.075+0000 7f8363961640 10 librados: watch_flush exit 2026-02-17T05:56:48.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.075+0000 7f8363961640 20 client.4550.objecter shutdown clearing up homeless session... 2026-02-17T05:56:48.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.075+0000 7f8363961640 10 client.4550.objecter successfully canceled tick 2026-02-17T05:56:48.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.075+0000 7f8363961640 1 -- 10.20.193.16:0/944886098 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f832c079190 msgr2=0x7f832c07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.075+0000 7f8363961640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f832c079190 0x7f832c07b650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f834c006fd0 tx=0x7f834c008040 comp rx=0 tx=0).stop 2026-02-17T05:56:48.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 10 monclient: shutdown 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 -- 10.20.193.16:0/944886098 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c155e00 msgr2=0x7f835c143ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c155e00 0x7f835c143ac0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f835000b500 tx=0x7f835000b9d0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 -- 10.20.193.16:0/944886098 shutdown_connections 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f832c079190 0x7f832c07b650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f835c1561e0 0x7f835c14d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f835c155e00 0x7f835c143ac0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 --2- 10.20.193.16:0/944886098 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f835c074dc0 0x7f835c151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 -- 10.20.193.16:0/944886098 >> 10.20.193.16:0/944886098 conn(0x7f835c0ce6c0 msgr2=0x7f835c0cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 -- 10.20.193.16:0/944886098 shutdown_connections 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 -- 10.20.193.16:0/944886098 wait complete. 2026-02-17T05:56:48.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.076+0000 7f8363961640 1 librados: shutdown 2026-02-17T05:56:48.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.078+0000 7f97f9379640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-17T05:56:48.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.078+0000 7f97f9379640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-17T05:56:48.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.078+0000 7f97f9379640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.078+0000 7f97f9379640 1 -- 10.20.193.16:0/1259493090 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f97f4076e00 con 0x7f97f4074dc0 2026-02-17T05:56:48.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.080+0000 7f97d37fe640 1 -- 10.20.193.16:0/1259493090 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f97f4076e00 con 0x7f97f4074dc0 2026-02-17T05:56:48.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.080+0000 7f97d37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-17T05:56:48.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.080+0000 7f97d37fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:48.081 INFO:teuthology.orchestra.run.trial016.stdout:107374182404 2026-02-17T05:56:48.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 10 librados: watch_flush enter 2026-02-17T05:56:48.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 10 librados: watch_flush exit 2026-02-17T05:56:48.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 20 client.4553.objecter shutdown clearing up homeless session... 2026-02-17T05:56:48.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 10 client.4553.objecter successfully canceled tick 2026-02-17T05:56:48.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 -- 10.20.193.16:0/1259493090 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97c00791c0 msgr2=0x7f97c007b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97c00791c0 0x7f97c007b680 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97dc039000 tx=0x7f97dc031000 comp rx=0 tx=0).stop 2026-02-17T05:56:48.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 10 monclient: shutdown 2026-02-17T05:56:48.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 -- 10.20.193.16:0/1259493090 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f4074dc0 msgr2=0x7f97f4073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f4074dc0 0x7f97f4073b00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97e800c730 tx=0x7f97e800cc00 comp rx=0 tx=0).stop 2026-02-17T05:56:48.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 -- 10.20.193.16:0/1259493090 shutdown_connections 2026-02-17T05:56:48.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97c00791c0 0x7f97c007b680 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97f41561e0 0x7f97f414d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97f4155e00 0x7f97f414cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 --2- 10.20.193.16:0/1259493090 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97f4074dc0 0x7f97f4073b00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 -- 10.20.193.16:0/1259493090 >> 10.20.193.16:0/1259493090 conn(0x7f97f40ce6c0 msgr2=0x7f97f40ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 -- 10.20.193.16:0/1259493090 shutdown_connections 2026-02-17T05:56:48.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 -- 10.20.193.16:0/1259493090 wait complete. 2026-02-17T05:56:48.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.082+0000 7f97f9379640 1 librados: shutdown 2026-02-17T05:56:48.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182405 got 107374182404 for osd.4 2026-02-17T05:56:48.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182405 got 107374182404 for osd.8 2026-02-17T05:56:48.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.102+0000 7fbfc4d49640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-17T05:56:48.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.102+0000 7fbfc4d49640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-17T05:56:48.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.102+0000 7fbfc4d49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.102+0000 7fbfc4d49640 1 -- 10.20.193.16:0/3509700266 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbfc00d0c70 con 0x7fbfc00745e0 2026-02-17T05:56:48.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.104+0000 7fbfa6ffd640 1 -- 10.20.193.16:0/3509700266 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbfb40629f0 con 0x7fbfc00745e0 2026-02-17T05:56:48.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.104+0000 7fbfa6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-17T05:56:48.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.104+0000 7fbfa6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:48.106 INFO:teuthology.orchestra.run.trial016.stdout:115964116999 2026-02-17T05:56:48.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 10 librados: watch_flush enter 2026-02-17T05:56:48.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 10 librados: watch_flush exit 2026-02-17T05:56:48.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 20 client.4556.objecter shutdown clearing up homeless session... 2026-02-17T05:56:48.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 10 client.4556.objecter successfully canceled tick 2026-02-17T05:56:48.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 -- 10.20.193.16:0/3509700266 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbf8c079190 msgr2=0x7fbf8c07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbf8c079190 0x7fbf8c07b650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbfc008e730 tx=0x7fbfa8031000 comp rx=0 tx=0).stop 2026-02-17T05:56:48.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 10 monclient: shutdown 2026-02-17T05:56:48.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 -- 10.20.193.16:0/3509700266 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbfc00745e0 msgr2=0x7fbfc00c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbfc00745e0 0x7fbfc00c37f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400b570 tx=0x7fbfb400ba40 comp rx=0 tx=0).stop 2026-02-17T05:56:48.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 -- 10.20.193.16:0/3509700266 shutdown_connections 2026-02-17T05:56:48.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbf8c079190 0x7fbf8c07b650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbfc0155fa0 0x7fbfc0149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbfc0156380 0x7fbfc015fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 --2- 10.20.193.16:0/3509700266 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbfc00745e0 0x7fbfc00c37f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 -- 10.20.193.16:0/3509700266 >> 10.20.193.16:0/3509700266 conn(0x7fbfc00ce6c0 msgr2=0x7fbfc01512a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 -- 10.20.193.16:0/3509700266 shutdown_connections 2026-02-17T05:56:48.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 -- 10.20.193.16:0/3509700266 wait complete. 2026-02-17T05:56:48.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.106+0000 7fbfc4d49640 1 librados: shutdown 2026-02-17T05:56:48.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964117001 got 115964116999 for osd.7 2026-02-17T05:56:48.406 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-17T05:56:48.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.475+0000 7f97d6387640 10 monclient: get_monmap_and_config 2026-02-17T05:56:48.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.475+0000 7f97d6387640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.475+0000 7f97d6387640 10 monclient: monmap: 2026-02-17T05:56:48.477 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.477 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.477 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:48.476721+0000 2026-02-17T05:56:48.477 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:48.476721+0000 2026-02-17T05:56:48.477 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.477 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.477 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:48.477 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:48.477 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:48.478 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:48.478 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:48.478 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:48.478 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 1 Processor -- start 2026-02-17T05:56:48.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 1 -- start start 2026-02-17T05:56:48.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 10 monclient: init 2026-02-17T05:56:48.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 10 monclient: _add_conns ranks=[1,2,0,3,4,5] 2026-02-17T05:56:48.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f97d00700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:48.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f97d00c86f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:48.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f97d0153e00 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:48.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d00c4820 con 0x7f97d0153e00 2026-02-17T05:56:48.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d00c4b00 con 0x7f97d00700f0 2026-02-17T05:56:48.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d00c4de0 con 0x7f97d00c86f0 2026-02-17T05:56:48.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.476+0000 7f97d6387640 10 monclient(hunting): authenticate will time out at 1061.045735s 2026-02-17T05:56:48.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.477+0000 7f97d48fd640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f97d0153e00 0x7f97d0164350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:56162/0 (socket says 10.20.193.16:56162) 2026-02-17T05:56:48.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.477+0000 7f97d48fd640 1 -- 10.20.193.16:0/1894564857 learned_addr learned my addr 10.20.193.16:0/1894564857 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:48.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1553439388 0 0) 0x7f97d00c4820 con 0x7f97d0153e00 2026-02-17T05:56:48.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 10 monclient(hunting): _negotiate 2026-02-17T05:56:48.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 10 monclient(hunting): my global_id is 4440 2026-02-17T05:56:48.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97b4003620 con 0x7f97d0153e00 2026-02-17T05:56:48.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2984449938 0 0) 0x7f97d00c4de0 con 0x7f97d00c86f0 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 10 monclient(hunting): _negotiate 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 10 monclient(hunting): my global_id is 4559 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.478+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97d00c4820 con 0x7f97d00c86f0 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.479+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584894683 0 0) 0x7f97d00c4b00 con 0x7f97d00700f0 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.479+0000 7f97ceffd640 10 monclient(hunting): _negotiate 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.479+0000 7f97ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.479+0000 7f97ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.479+0000 7f97ceffd640 10 monclient(hunting): my global_id is 4513 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.479+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97d00c4de0 con 0x7f97d00700f0 2026-02-17T05:56:48.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 <== mon.0 v1:10.20.193.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1882450530 0 0) 0x7f97b4003620 con 0x7f97d0153e00 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 >> v1:10.20.193.148:6789/0 conn(0x7f97d00c86f0 legacy=0x7f97d01536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 >> v1:10.20.193.128:6789/0 conn(0x7f97d00700f0 legacy=0x7f97d00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.16:6789/0 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 --> v1:10.20.193.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d00c50c0 con 0x7f97d0153e00 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 <== mon.0 v1:10.20.193.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f97c4003280 con 0x7f97d0153e00 2026-02-17T05:56:48.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97d6387640 5 monclient: authenticate success, global_id 4440 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97d6387640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient: dump: 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.488 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] but i'm connected to v1:10.20.193.16:6789/0, reconnecting 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 >> v1:10.20.193.16:6789/0 conn(0x7f97d0153e00 legacy=0x7f97d0164350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b4003fe0 0x7f97b4004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient(hunting): picked mon.c con 0x7f97b4003fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b4004e40 0x7f97b40052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient(hunting): picked mon.a con 0x7f97b4004e40 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b4009ab0 0x7f97b400bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient(hunting): picked mon.b con 0x7f97b4009ab0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97b400c5c0 con 0x7f97b4004e40 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f97b400c770 con 0x7f97b4009ab0 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97b400c970 con 0x7f97b4003fe0 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97ceffd640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97d6387640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97cffff640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b4003fe0 0x7f97b4004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.480+0000 7f97cf7fe640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b4004e40 0x7f97b40052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cffff640 10 monclient(hunting): get_auth_request con 0x7f97b4003fe0 auth_method 0 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cf7fe640 10 monclient(hunting): get_auth_request con 0x7f97b4004e40 auth_method 0 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.481+0000 7f97cf7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:48.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97d48fd640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b4009ab0 0x7f97b400bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97d48fd640 10 monclient(hunting): get_auth_request con 0x7f97b4009ab0 auth_method 0 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97d48fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97d48fd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97d48fd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97d48fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97d48fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97d48fd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 10 monclient(hunting): handle_auth_done global_id 4440 payload 293 2026-02-17T05:56:48.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 1 -- 10.20.193.16:0/1894564857 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b4009ab0 msgr2=0x7f97b400bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b4009ab0 0x7f97b400bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 1 -- 10.20.193.16:0/1894564857 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b4004e40 msgr2=0x7f97b40052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b4004e40 0x7f97b40052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 1 monclient: found mon.c 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 1 -- 10.20.193.16:0/1894564857 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b4003620 con 0x7f97b4003fe0 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.482+0000 7f97d48fd640 1 -- 10.20.193.16:0/1894564857 reap_dead start 2026-02-17T05:56:48.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.483+0000 7f97cffff640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b4003fe0 0x7f97b4004900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0031a0 tx=0x7f97bc02ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90a5102146fa2e19 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc002dc0 con 0x7f97b4003fe0 2026-02-17T05:56:48.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:48.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97ceffd640 10 monclient: dump: 2026-02-17T05:56:48.500 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97bc0052d0 con 0x7f97b4003fe0 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97ceffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97ceffd640 1 -- 10.20.193.16:0/1894564857 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc02f720 con 0x7f97b4003fe0 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 4 set_mon_vals no callback set 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97ceffd640 10 monclient: dump: 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.502 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 10 monclient: shutdown 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 1 -- 10.20.193.16:0/1894564857 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b4003fe0 msgr2=0x7f97b4004900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b4003fe0 0x7f97b4004900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0031a0 tx=0x7f97bc02ef00 comp rx=0 tx=0).stop 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 1 -- 10.20.193.16:0/1894564857 shutdown_connections 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 1 --2- 10.20.193.16:0/1894564857 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b4003fe0 0x7f97b4004900 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 1 -- 10.20.193.16:0/1894564857 >> 10.20.193.16:0/1894564857 conn(0x7f97d00cd770 msgr2=0x7f97d00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 1 -- 10.20.193.16:0/1894564857 shutdown_connections 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.485+0000 7f97d6387640 1 -- 10.20.193.16:0/1894564857 wait complete. 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient: monmap: 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:48.504 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 librados: starting msgr at 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 librados: starting objecter 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 Processor -- start 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 -- start start 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 librados: setting wanted keys 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 librados: calling monclient init 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient: init 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b4003fe0 0x7f97d01f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient(hunting): picked mon.noname-b con 0x7f97b4003fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97d015b030 0x7f97d0073360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient(hunting): picked mon.noname-c con 0x7f97d015b030 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97d015b4b0 0x7f97d00738a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient(hunting): picked mon.noname-a con 0x7f97d015b4b0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0074c00 con 0x7f97d015b4b0 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f97d00c5680 con 0x7f97b4003fe0 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0074d80 con 0x7f97d015b030 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.486+0000 7f97d6387640 10 monclient(hunting): authenticate will time out at 1061.055882s 2026-02-17T05:56:48.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97d48fd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97d015b4b0 0x7f97d00738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cf7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97d015b030 0x7f97d0073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cffff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b4003fe0 0x7f97d01f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97d48fd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97d015b4b0 0x7f97d00738a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59970/0 (socket says 10.20.193.16:59970) 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97d48fd640 1 -- 10.20.193.16:0/1844242258 learned_addr learned my addr 10.20.193.16:0/1844242258 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97d48fd640 10 monclient(hunting): get_auth_request con 0x7f97d015b4b0 auth_method 0 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97d48fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97d48fd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97d48fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cf7fe640 10 monclient(hunting): get_auth_request con 0x7f97d015b030 auth_method 0 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cffff640 10 monclient(hunting): get_auth_request con 0x7f97b4003fe0 auth_method 0 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97d48fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97d48fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97d48fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.487+0000 7f97cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.488+0000 7f97d48fd640 10 monclient(hunting): handle_auth_done global_id 4443 payload 824 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.488+0000 7f97d48fd640 1 -- 10.20.193.16:0/1844242258 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97d015b030 msgr2=0x7f97d0073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.488+0000 7f97d48fd640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97d015b030 0x7f97d0073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.488+0000 7f97d48fd640 1 -- 10.20.193.16:0/1844242258 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b4003fe0 msgr2=0x7f97d01f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.488+0000 7f97d48fd640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b4003fe0 0x7f97d01f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.488+0000 7f97d48fd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.488+0000 7f97d48fd640 1 monclient: found mon.noname-a 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.488+0000 7f97d48fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.488+0000 7f97d48fd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.488+0000 7f97d48fd640 1 -- 10.20.193.16:0/1844242258 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d01490b0 con 0x7f97d015b4b0 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.489+0000 7f97d48fd640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97d015b4b0 0x7f97d00738a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f97d00c9440 tx=0x7f97c400a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.489+0000 7f97ccff9640 10 client.?.objecter ms_handle_connect 0x7f97d015b4b0 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.489+0000 7f97ccff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 1 -- 10.20.193.16:0/1844242258 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c40133f0 con 0x7f97d015b4b0 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 10 monclient: dump: 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.510 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 1 -- 10.20.193.16:0/1844242258 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c4019070 con 0x7f97d015b4b0 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 1 -- 10.20.193.16:0/1844242258 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4013790 con 0x7f97d015b4b0 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ce7fc640 4 set_mon_vals no callback set 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97ccff9640 10 monclient: dump: 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97d6387640 5 monclient: authenticate success, global_id 4443 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97d6387640 10 monclient: _renew_subs 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97d6387640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97d6387640 1 -- 10.20.193.16:0/1844242258 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d01f4440 con 0x7f97d015b4b0 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97d6387640 10 client.4443.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97d6387640 10 monclient: _renew_subs 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97d6387640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97d6387640 1 -- 10.20.193.16:0/1844242258 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d0072880 con 0x7f97d015b4b0 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.491+0000 7f97d6387640 1 librados: init done 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.492+0000 7f97d6387640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.492+0000 7f97d6387640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.492+0000 7f97d6387640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.492+0000 7f97d6387640 1 -- 10.20.193.16:0/1844242258 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d00704d0 con 0x7f97d015b4b0 2026-02-17T05:56:48.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.504+0000 7f97ccff9640 1 -- 10.20.193.16:0/1844242258 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f97c4013930 con 0x7f97d015b4b0 2026-02-17T05:56:48.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.504+0000 7f97ccff9640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97a0079140 0x7f97a007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.504+0000 7f97cffff640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97a0079140 0x7f97a007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.504+0000 7f97cffff640 10 monclient: get_auth_request con 0x7f97a0079140 auth_method 0 2026-02-17T05:56:48.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.505+0000 7f97ccff9640 10 client.4443.objecter ms_handle_connect 0x7f97a0079140 2026-02-17T05:56:48.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.505+0000 7f97cffff640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97a0079140 0x7f97a007b600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97bc037000 tx=0x7f97bc009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.506+0000 7f97ccff9640 1 -- 10.20.193.16:0/1844242258 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f97c405ed00 con 0x7f97d015b4b0 2026-02-17T05:56:48.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.506+0000 7f97ccff9640 10 client.4443.objecter ms_dispatch2 0x7f97d0000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:48.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.506+0000 7f97ccff9640 3 client.4443.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:48.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.506+0000 7f97ccff9640 3 client.4443.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:48.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.506+0000 7f97ccff9640 20 client.4443.objecter dump_active .. 0 homeless 2026-02-17T05:56:48.528 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-17T05:56:48.547 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-17T05:56:48.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.598+0000 7f043338f640 10 monclient: get_monmap_and_config 2026-02-17T05:56:48.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.598+0000 7f043338f640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.598+0000 7f043338f640 10 monclient: monmap: 2026-02-17T05:56:48.612 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.612 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.612 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:48.599164+0000 2026-02-17T05:56:48.612 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:48.599164+0000 2026-02-17T05:56:48.612 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.612 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.613 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:48.613 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:48.613 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:48.613 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:48.613 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:48.613 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:48.614 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.598+0000 7f043338f640 1 Processor -- start 2026-02-17T05:56:48.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.598+0000 7f043338f640 1 -- start start 2026-02-17T05:56:48.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.598+0000 7f043338f640 10 monclient: init 2026-02-17T05:56:48.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 10 monclient: _add_conns ranks=[1,2,5,0,3,4] 2026-02-17T05:56:48.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f042c1538c0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:48.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f042c153ca0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:48.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f042c15d810 0x7f042c164490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 10 monclient(hunting): picked mon.noname-c con 0x7f042c15d810 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:48.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042c0cbd90 con 0x7f042c1538c0 2026-02-17T05:56:48.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042c0ca300 con 0x7f042c153ca0 2026-02-17T05:56:48.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f042c0c7c30 con 0x7f042c15d810 2026-02-17T05:56:48.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f043338f640 10 monclient(hunting): authenticate will time out at 1061.168285s 2026-02-17T05:56:48.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f0431104640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f042c1538c0 0x7f042c0cb680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:53862/0 (socket says 10.20.193.16:53862) 2026-02-17T05:56:48.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f0431104640 1 -- 10.20.193.16:0/2835239535 learned_addr learned my addr 10.20.193.16:0/2835239535 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:48.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.599+0000 7f0431905640 1 --2- 10.20.193.16:0/2835239535 >> v2:10.20.193.148:3300/0 conn(0x7f042c15d810 0x7f042c164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f0431905640 10 monclient(hunting): get_auth_request con 0x7f042c15d810 auth_method 0 2026-02-17T05:56:48.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f0431905640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f0431905640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f0431905640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f0431905640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f0431905640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f0431905640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1806777561 0 0) 0x7f042c0cbd90 con 0x7f042c1538c0 2026-02-17T05:56:48.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f041bfff640 10 monclient(hunting): _negotiate 2026-02-17T05:56:48.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f041bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f041bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f041bfff640 10 monclient(hunting): my global_id is 4519 2026-02-17T05:56:48.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.600+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0414003620 con 0x7f042c1538c0 2026-02-17T05:56:48.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 <== mon.1 v1:10.20.193.128:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3320649958 0 0) 0x7f0414003620 con 0x7f042c1538c0 2026-02-17T05:56:48.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 >> v2:10.20.193.148:3300/0 conn(0x7f042c15d810 msgr2=0x7f042c164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 --2- 10.20.193.16:0/2835239535 >> v2:10.20.193.148:3300/0 conn(0x7f042c15d810 0x7f042c164490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 >> v1:10.20.193.148:6789/0 conn(0x7f042c153ca0 legacy=0x7f042c15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 monclient: found mon.noname-b-legacy 2026-02-17T05:56:48.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.128:6789/0 2026-02-17T05:56:48.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 --> v1:10.20.193.128:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c0ca5e0 con 0x7f042c1538c0 2026-02-17T05:56:48.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f043338f640 5 monclient: authenticate success, global_id 4519 2026-02-17T05:56:48.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 <== mon.1 v1:10.20.193.128:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f041c002f00 con 0x7f042c1538c0 2026-02-17T05:56:48.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f043338f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-17T05:56:48.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient: dump: 2026-02-17T05:56:48.622 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.622 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.622 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.622 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.623 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.623 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.623 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.623 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.623 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.623 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] but i'm connected to v1:10.20.193.128:6789/0, reconnecting 2026-02-17T05:56:48.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 >> v1:10.20.193.128:6789/0 conn(0x7f042c1538c0 legacy=0x7f042c0cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:48.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 --2- 10.20.193.16:0/2835239535 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 0x7f0414004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient(hunting): picked mon.b con 0x7f0414003bf0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 --2- 10.20.193.16:0/2835239535 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0414004900 0x7f0414004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient(hunting): picked mon.c con 0x7f0414004900 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 --2- 10.20.193.16:0/2835239535 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04140053e0 0x7f041400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient(hunting): picked mon.a con 0x7f04140053e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f042c0c7c30 con 0x7f04140053e0 2026-02-17T05:56:48.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f041400c2b0 con 0x7f0414003bf0 2026-02-17T05:56:48.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f041400c4b0 con 0x7f0414004900 2026-02-17T05:56:48.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f041bfff640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:48.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.602+0000 7f043338f640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.603+0000 7f0431104640 1 --2- 10.20.193.16:0/2835239535 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 0x7f0414004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0431104640 10 monclient(hunting): get_auth_request con 0x7f0414003bf0 auth_method 0 2026-02-17T05:56:48.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0431104640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0431104640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0431104640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:48.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0431104640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0431104640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0431104640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:48.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0430903640 1 --2- 10.20.193.16:0/2835239535 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0414004900 0x7f0414004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0430903640 10 monclient(hunting): get_auth_request con 0x7f0414004900 auth_method 0 2026-02-17T05:56:48.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0430903640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0430903640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.604+0000 7f0430903640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:48.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0430903640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0430903640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0430903640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:48.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 10 monclient(hunting): handle_auth_done global_id 4519 payload 293 2026-02-17T05:56:48.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 1 -- 10.20.193.16:0/2835239535 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0414004900 msgr2=0x7f0414004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 1 --2- 10.20.193.16:0/2835239535 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0414004900 0x7f0414004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 1 -- 10.20.193.16:0/2835239535 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04140053e0 msgr2=0x7f041400bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 1 --2- 10.20.193.16:0/2835239535 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04140053e0 0x7f041400bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 1 monclient: found mon.b 2026-02-17T05:56:48.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 1 -- 10.20.193.16:0/2835239535 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0414003620 con 0x7f0414003bf0 2026-02-17T05:56:48.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431905640 1 -- 10.20.193.16:0/2835239535 reap_dead start 2026-02-17T05:56:48.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.605+0000 7f0431104640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.606+0000 7f0431104640 1 --2- 10.20.193.16:0/2835239535 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 0x7f0414004010 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f041c0036c0 tx=0x7f041c02f550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90abffb85595b4c6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c007d80 con 0x7f0414003bf0 2026-02-17T05:56:48.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f041bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f041bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:48.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f041bfff640 10 monclient: dump: 2026-02-17T05:56:48.632 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.632 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.632 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.632 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.632 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.632 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.632 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.632 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.632 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041c005bd0 con 0x7f0414003bf0 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f041bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f041bfff640 1 -- 10.20.193.16:0/2835239535 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c007070 con 0x7f0414003bf0 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 4 set_mon_vals no callback set 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f041bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f041bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f041bfff640 10 monclient: dump: 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.633 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 10 monclient: shutdown 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 1 -- 10.20.193.16:0/2835239535 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 msgr2=0x7f0414004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 1 --2- 10.20.193.16:0/2835239535 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 0x7f0414004010 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f041c0036c0 tx=0x7f041c02f550 comp rx=0 tx=0).stop 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 1 -- 10.20.193.16:0/2835239535 shutdown_connections 2026-02-17T05:56:48.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 1 --2- 10.20.193.16:0/2835239535 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 0x7f0414004010 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 1 -- 10.20.193.16:0/2835239535 >> 10.20.193.16:0/2835239535 conn(0x7f042c0cf600 msgr2=0x7f042c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f97ccff9640 1 -- 10.20.193.16:0/1844242258 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f97c40633c0 con 0x7f97d015b4b0 2026-02-17T05:56:48.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f97ccff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f97ccff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 1 -- 10.20.193.16:0/2835239535 shutdown_connections 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.608+0000 7f043338f640 1 -- 10.20.193.16:0/2835239535 wait complete. 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.609+0000 7f043338f640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.609+0000 7f043338f640 10 monclient: monmap: 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.641 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.609+0000 7f043338f640 1 librados: starting msgr at 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.609+0000 7f043338f640 1 librados: starting objecter 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.609+0000 7f043338f640 1 Processor -- start 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.609+0000 7f043338f640 1 -- start start 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.609+0000 7f043338f640 1 librados: setting wanted keys 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.609+0000 7f043338f640 1 librados: calling monclient init 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.609+0000 7f043338f640 10 monclient: init 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 0x7f042c0c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 10 monclient(hunting): picked mon.noname-b con 0x7f0414003bf0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f042c1f8350 0x7f042c1e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 10 monclient(hunting): picked mon.noname-a con 0x7f042c1f8350 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f042c1e7210 0x7f042c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 10 monclient(hunting): picked mon.noname-c con 0x7f042c1e7210 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f042c0cae80 con 0x7f042c1f8350 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f042c0caba0 con 0x7f0414003bf0 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f042c0cb160 con 0x7f042c1e7210 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f043338f640 10 monclient(hunting): authenticate will time out at 1061.179075s 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0430903640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f042c1f8350 0x7f042c1e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431104640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 0x7f042c0c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0430903640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f042c1f8350 0x7f042c1e6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59988/0 (socket says 10.20.193.16:59988) 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0430903640 1 -- 10.20.193.16:0/165260077 learned_addr learned my addr 10.20.193.16:0/165260077 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0430903640 10 monclient(hunting): get_auth_request con 0x7f042c1f8350 auth_method 0 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0430903640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0430903640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0430903640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431905640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f042c1e7210 0x7f042c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431104640 10 monclient(hunting): get_auth_request con 0x7f0414003bf0 auth_method 0 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431104640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431104640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431104640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0430903640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0430903640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0430903640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431905640 10 monclient(hunting): get_auth_request con 0x7f042c1e7210 auth_method 0 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431905640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431905640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431905640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431104640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431104640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.610+0000 7f0431104640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.611+0000 7f0431905640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.611+0000 7f0431905640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.611+0000 7f0431905640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.612+0000 7f0431104640 10 monclient(hunting): handle_auth_done global_id 4522 payload 824 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.612+0000 7f0431104640 1 -- 10.20.193.16:0/165260077 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f042c1e7210 msgr2=0x7f042c15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.612+0000 7f0431104640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f042c1e7210 0x7f042c15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.612+0000 7f0431104640 1 -- 10.20.193.16:0/165260077 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f042c1f8350 msgr2=0x7f042c1e6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.612+0000 7f0431104640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f042c1f8350 0x7f042c1e6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.612+0000 7f0431104640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.612+0000 7f0431104640 1 monclient: found mon.noname-b 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.612+0000 7f0431104640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.612+0000 7f0431104640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.612+0000 7f0431104640 1 -- 10.20.193.16:0/165260077 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c06d710 con 0x7f0414003bf0 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.613+0000 7f0431104640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 0x7f042c0c5dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f041c033000 tx=0x7f042c0ca7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.613+0000 7f0419ffb640 10 client.?.objecter ms_handle_connect 0x7f0414003bf0 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.613+0000 7f0419ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 1 -- 10.20.193.16:0/165260077 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c0033c0 con 0x7f0414003bf0 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 10 monclient: dump: 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.645 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 1 -- 10.20.193.16:0/165260077 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041c004ae0 con 0x7f0414003bf0 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 5 monclient: authenticate success, global_id 4522 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 10 monclient: _renew_subs 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 1 -- 10.20.193.16:0/165260077 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042c06d9b0 con 0x7f0414003bf0 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 1 -- 10.20.193.16:0/165260077 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c007d60 con 0x7f0414003bf0 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f041b7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f0419ffb640 10 monclient: dump: 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.646 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 10 client.4522.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 10 monclient: _renew_subs 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 1 -- 10.20.193.16:0/165260077 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042c06df70 con 0x7f0414003bf0 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 1 librados: init done 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.615+0000 7f043338f640 1 -- 10.20.193.16:0/165260077 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042c15b000 con 0x7f0414003bf0 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.627+0000 7f0419ffb640 1 -- 10.20.193.16:0/165260077 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f041c04c030 con 0x7f0414003bf0 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.627+0000 7f0419ffb640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f03fc079170 0x7f03fc07b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.627+0000 7f0430903640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f03fc079170 0x7f03fc07b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.627+0000 7f0430903640 10 monclient: get_auth_request con 0x7f03fc079170 auth_method 0 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.627+0000 7f0419ffb640 10 client.4522.objecter ms_handle_connect 0x7f03fc079170 2026-02-17T05:56:48.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.627+0000 7f0430903640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f03fc079170 0x7f03fc07b630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0420002410 tx=0x7f0420007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.628+0000 7f0419ffb640 1 -- 10.20.193.16:0/165260077 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f041c082a70 con 0x7f0414003bf0 2026-02-17T05:56:48.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.628+0000 7f0419ffb640 10 client.4522.objecter ms_dispatch2 0x7f042c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:48.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.628+0000 7f0419ffb640 3 client.4522.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:48.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.628+0000 7f0419ffb640 3 client.4522.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:48.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.628+0000 7f0419ffb640 20 client.4522.objecter dump_active .. 0 homeless 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.638+0000 7f40f7d9e640 10 monclient: get_monmap_and_config 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.638+0000 7f40f7d9e640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.638+0000 7f40f7d9e640 10 monclient: monmap: 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:48.639275+0000 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:48.639275+0000 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:48.652 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.638+0000 7f40f7d9e640 1 Processor -- start 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.638+0000 7f40f7d9e640 1 -- start start 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.638+0000 7f40f7d9e640 10 monclient: init 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 10 monclient: _add_conns ranks=[5,2,1,0,4,3] 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f40f00700f0 0x7f40f00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 10 monclient(hunting): picked mon.noname-c con 0x7f40f00700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f40f00c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f40f01536f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f00c4820 con 0x7f40f01536f0 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f00c4b00 con 0x7f40f00c7fe0 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f40f00c4de0 con 0x7f40f00700f0 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f7d9e640 10 monclient(hunting): authenticate will time out at 1061.208227s 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f5b13640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f40f00700f0 0x7f40f00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f5b13640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f40f00700f0 0x7f40f00704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57226/0 (socket says 10.20.193.16:57226) 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f5b13640 1 -- 10.20.193.16:0/3186572859 learned_addr learned my addr 10.20.193.16:0/3186572859 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f5b13640 10 monclient(hunting): get_auth_request con 0x7f40f00700f0 auth_method 0 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f5b13640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f5b13640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.639+0000 7f40f5b13640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.640+0000 7f40f5b13640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.640+0000 7f40f5b13640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.640+0000 7f40f5b13640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.641+0000 7f40f5b13640 10 monclient(hunting): handle_auth_done global_id 4574 payload 274 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.641+0000 7f40f5b13640 1 -- 10.20.193.16:0/3186572859 >> v1:10.20.193.148:6789/0 conn(0x7f40f00c7fe0 legacy=0x7f40f00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.641+0000 7f40f5b13640 1 -- 10.20.193.16:0/3186572859 >> v1:10.20.193.128:6789/0 conn(0x7f40f01536f0 legacy=0x7f40f0153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.641+0000 7f40f5b13640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.641+0000 7f40f5b13640 1 monclient: found mon.noname-c 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.641+0000 7f40f5b13640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.641+0000 7f40f5b13640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.641+0000 7f40f5b13640 1 -- 10.20.193.16:0/3186572859 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f00c50c0 con 0x7f40f00700f0 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.642+0000 7f40f5b13640 1 --2- 10.20.193.16:0/3186572859 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f00700f0 0x7f40f00704d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009970 tx=0x7f40e002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea50f3b5281fd3ce server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.643+0000 7f40f4b11640 1 -- 10.20.193.16:0/3186572859 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e003c070 con 0x7f40f00700f0 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.643+0000 7f40f4b11640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.643+0000 7f40f4b11640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.643+0000 7f40f4b11640 10 monclient: dump: 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.654 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.643+0000 7f40f4b11640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.643+0000 7f40f4b11640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.643+0000 7f40f4b11640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.643+0000 7f40f7d9e640 5 monclient: authenticate success, global_id 4574 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.643+0000 7f40f7d9e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f4b11640 1 -- 10.20.193.16:0/3186572859 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e002fb10 con 0x7f40f00700f0 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f4b11640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f4b11640 1 -- 10.20.193.16:0/3186572859 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0037440 con 0x7f40f00700f0 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f4b11640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f4b11640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f4b11640 10 monclient: dump: 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 4 set_mon_vals no callback set 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 10 monclient: shutdown 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3186572859 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f00700f0 msgr2=0x7f40f00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 1 --2- 10.20.193.16:0/3186572859 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f00700f0 0x7f40f00704d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009970 tx=0x7f40e002ee90 comp rx=0 tx=0).stop 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3186572859 shutdown_connections 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 1 --2- 10.20.193.16:0/3186572859 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f00700f0 0x7f40f00704d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3186572859 >> 10.20.193.16:0/3186572859 conn(0x7f40f00cd770 msgr2=0x7f40f00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3186572859 shutdown_connections 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3186572859 wait complete. 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.644+0000 7f40f7d9e640 10 monclient: monmap: 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 librados: starting msgr at 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 librados: starting objecter 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 Processor -- start 2026-02-17T05:56:48.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 -- start start 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 librados: setting wanted keys 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 librados: calling monclient init 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient: init 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f40f00700f0 0x7f40f00c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient(hunting): picked mon.noname-b con 0x7f40f00700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f40f00c7fe0 0x7f40f013e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient(hunting): picked mon.noname-a con 0x7f40f00c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f01536f0 0x7f40f01e8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient(hunting): picked mon.noname-c con 0x7f40f01536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0074c00 con 0x7f40f00c7fe0 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f40f00c5680 con 0x7f40f00700f0 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0074d80 con 0x7f40f01536f0 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f7d9e640 10 monclient(hunting): authenticate will time out at 1061.214581s 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f6314640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f01536f0 0x7f40f01e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.645+0000 7f40f5312640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f40f00c7fe0 0x7f40f013e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f6314640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f01536f0 0x7f40f01e8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57236/0 (socket says 10.20.193.16:57236) 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f6314640 1 -- 10.20.193.16:0/3817797741 learned_addr learned my addr 10.20.193.16:0/3817797741 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f6314640 10 monclient(hunting): get_auth_request con 0x7f40f01536f0 auth_method 0 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f6314640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f6314640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f6314640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f5312640 10 monclient(hunting): get_auth_request con 0x7f40f00c7fe0 auth_method 0 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f5312640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f5312640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f5312640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f6314640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f6314640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f6314640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f5312640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f5312640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.646+0000 7f40f5312640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.647+0000 7f40f6314640 10 monclient(hunting): handle_auth_done global_id 4577 payload 824 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.647+0000 7f40f6314640 1 -- 10.20.193.16:0/3817797741 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f40f00700f0 msgr2=0x7f40f00c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.647+0000 7f40f6314640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f40f00700f0 0x7f40f00c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.647+0000 7f40f6314640 1 -- 10.20.193.16:0/3817797741 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f40f00c7fe0 msgr2=0x7f40f013e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.647+0000 7f40f6314640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f40f00c7fe0 0x7f40f013e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.647+0000 7f40f6314640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.647+0000 7f40f6314640 1 monclient: found mon.noname-c 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.647+0000 7f40f6314640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.647+0000 7f40f6314640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.647+0000 7f40f6314640 1 -- 10.20.193.16:0/3817797741 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f00c4260 con 0x7f40f01536f0 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.648+0000 7f40f6314640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f01536f0 0x7f40f01e8260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00b990 tx=0x7f40ec00be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.648+0000 7f40de7fc640 10 client.?.objecter ms_handle_connect 0x7f40f01536f0 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.648+0000 7f40de7fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 1 -- 10.20.193.16:0/3817797741 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00c790 con 0x7f40f01536f0 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 10 monclient: dump: 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 5 monclient: authenticate success, global_id 4577 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 1 -- 10.20.193.16:0/3817797741 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ec010070 con 0x7f40f01536f0 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 1 -- 10.20.193.16:0/3817797741 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00cbb0 con 0x7f40f01536f0 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40dffff640 4 set_mon_vals no callback set 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40de7fc640 10 monclient: dump: 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.659 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 10 monclient: _renew_subs 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3817797741 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f013de50 con 0x7f40f01536f0 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 10 client.4577.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 10 monclient: _renew_subs 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3817797741 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f01f3e40 con 0x7f40f01536f0 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 1 librados: init done 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.650+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3817797741 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f00704d0 con 0x7f40f01536f0 2026-02-17T05:56:48.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.662+0000 7f40de7fc640 1 -- 10.20.193.16:0/3817797741 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f40ec004070 con 0x7f40f01536f0 2026-02-17T05:56:48.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.662+0000 7f40de7fc640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f40c0079190 0x7f40c007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.662+0000 7f40f5b13640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f40c0079190 0x7f40c007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.662+0000 7f40f5b13640 10 monclient: get_auth_request con 0x7f40c0079190 auth_method 0 2026-02-17T05:56:48.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.662+0000 7f40de7fc640 10 client.4577.objecter ms_handle_connect 0x7f40c0079190 2026-02-17T05:56:48.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.662+0000 7f40f5b13640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f40c0079190 0x7f40c007b650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40f00c9540 tx=0x7f40e0031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.663+0000 7f40de7fc640 1 -- 10.20.193.16:0/3817797741 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f40ec00cd50 con 0x7f40f01536f0 2026-02-17T05:56:48.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.663+0000 7f40de7fc640 10 client.4577.objecter ms_dispatch2 0x7f40f0000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:48.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.663+0000 7f40de7fc640 3 client.4577.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:48.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.663+0000 7f40de7fc640 3 client.4577.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:48.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.663+0000 7f40de7fc640 20 client.4577.objecter dump_active .. 0 homeless 2026-02-17T05:56:48.677 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-17T05:56:48.687 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-17T05:56:48.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.693+0000 7f97d6387640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-17T05:56:48.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.694+0000 7f97d6387640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-17T05:56:48.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.694+0000 7f97d6387640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.694+0000 7f97d6387640 1 -- 10.20.193.16:0/1844242258 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f97d01573f0 con 0x7f97d015b4b0 2026-02-17T05:56:48.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.700+0000 7f97ccff9640 1 -- 10.20.193.16:0/1844242258 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f97c4062cc0 con 0x7f97d015b4b0 2026-02-17T05:56:48.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.700+0000 7f97ccff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-17T05:56:48.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.700+0000 7f97ccff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:48.701 INFO:teuthology.orchestra.run.trial016.stdout:98784247812 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 10 librados: watch_flush enter 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 10 librados: watch_flush exit 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 20 client.4443.objecter shutdown clearing up homeless session... 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 10 client.4443.objecter successfully canceled tick 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 -- 10.20.193.16:0/1844242258 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97a0079140 msgr2=0x7f97a007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97a0079140 0x7f97a007b600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97bc037000 tx=0x7f97bc009030 comp rx=0 tx=0).stop 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 10 monclient: shutdown 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 -- 10.20.193.16:0/1844242258 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97d015b4b0 msgr2=0x7f97d00738a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97d015b4b0 0x7f97d00738a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f97d00c9440 tx=0x7f97c400a600 comp rx=0 tx=0).stop 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 -- 10.20.193.16:0/1844242258 shutdown_connections 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f97a0079140 0x7f97a007b600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97d015b4b0 0x7f97d00738a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97d015b030 0x7f97d0073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 --2- 10.20.193.16:0/1844242258 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b4003fe0 0x7f97d01f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 -- 10.20.193.16:0/1844242258 >> 10.20.193.16:0/1844242258 conn(0x7f97d00cd770 msgr2=0x7f97d0149ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 -- 10.20.193.16:0/1844242258 shutdown_connections 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 -- 10.20.193.16:0/1844242258 wait complete. 2026-02-17T05:56:48.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.702+0000 7f97d6387640 1 librados: shutdown 2026-02-17T05:56:48.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.11 2026-02-17T05:56:48.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.730+0000 7f0419ffb640 1 -- 10.20.193.16:0/165260077 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f041c086a30 con 0x7f0414003bf0 2026-02-17T05:56:48.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.730+0000 7f0419ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.730+0000 7f0419ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:48.748 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.747+0000 7f9c8df85640 10 monclient: get_monmap_and_config 2026-02-17T05:56:48.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.747+0000 7f9c8df85640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.747+0000 7f9c8df85640 10 monclient: monmap: 2026-02-17T05:56:48.749 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.749 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.749 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:48.748860+0000 2026-02-17T05:56:48.749 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:48.748860+0000 2026-02-17T05:56:48.749 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.749 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.749 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:48.750 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:48.750 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:48.750 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:48.750 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:48.750 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:48.750 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 1 Processor -- start 2026-02-17T05:56:48.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 1 -- start start 2026-02-17T05:56:48.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 10 monclient: init 2026-02-17T05:56:48.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 10 monclient: _add_conns ranks=[0,1,5,4,2,3] 2026-02-17T05:56:48.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9c880700f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:48.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f9c880c86f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f9c88153e00 0x7f9c880704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 10 monclient(hunting): picked mon.noname-c con 0x7f9c88153e00 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c880c4820 con 0x7f9c880700f0 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c880c4b00 con 0x7f9c880c86f0 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f9c880c4de0 con 0x7f9c88153e00 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.748+0000 7f9c8df85640 10 monclient(hunting): authenticate will time out at 1061.317852s 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.749+0000 7f9c877fe640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f9c880700f0 0x7f9c880c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:56178/0 (socket says 10.20.193.16:56178) 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.749+0000 7f9c877fe640 1 -- 10.20.193.16:0/2241311540 learned_addr learned my addr 10.20.193.16:0/2241311540 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:48.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1570342494 0 0) 0x7f9c880c4820 con 0x7f9c880700f0 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 10 monclient(hunting): _negotiate 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 10 monclient(hunting): my global_id is 4452 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c6c003620 con 0x7f9c880700f0 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 679511866 0 0) 0x7f9c880c4b00 con 0x7f9c880c86f0 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 10 monclient(hunting): _negotiate 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 10 monclient(hunting): my global_id is 4525 2026-02-17T05:56:48.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.750+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c880c4820 con 0x7f9c880c86f0 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 <== mon.0 v1:10.20.193.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2099789305 0 0) 0x7f9c6c003620 con 0x7f9c880700f0 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 >> v2:10.20.193.148:3300/0 conn(0x7f9c88153e00 msgr2=0x7f9c880704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 1 --2- 10.20.193.16:0/2241311540 >> v2:10.20.193.148:3300/0 conn(0x7f9c88153e00 0x7f9c880704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 >> v1:10.20.193.128:6789/0 conn(0x7f9c880c86f0 legacy=0x7f9c881536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.16:6789/0 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 --> v1:10.20.193.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c880c50c0 con 0x7f9c880700f0 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 <== mon.0 v1:10.20.193.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f9c74002f00 con 0x7f9c880700f0 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 10 monclient: dump: 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.759 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] but i'm connected to v1:10.20.193.16:6789/0, reconnecting 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.751+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 >> v1:10.20.193.16:6789/0 conn(0x7f9c880700f0 legacy=0x7f9c880c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 1 --2- 10.20.193.16:0/2241311540 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c6c003dd0 0x7f9c6c0041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 10 monclient(hunting): picked mon.c con 0x7f9c6c003dd0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 1 --2- 10.20.193.16:0/2241311540 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c6c004950 0x7f9c6c004db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 10 monclient(hunting): picked mon.b con 0x7f9c6c004950 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 1 --2- 10.20.193.16:0/2241311540 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c6c009ab0 0x7f9c6c00bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 10 monclient(hunting): picked mon.a con 0x7f9c6c009ab0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c880c4de0 con 0x7f9c6c009ab0 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c00c5a0 con 0x7f9c6c004950 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c00c7a0 con 0x7f9c6c003dd0 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c867fc640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c87fff640 1 --2- 10.20.193.16:0/2241311540 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c6c009ab0 0x7f9c6c00bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.752+0000 7f9c86ffd640 1 --2- 10.20.193.16:0/2241311540 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c6c004950 0x7f9c6c004db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c86ffd640 10 monclient(hunting): get_auth_request con 0x7f9c6c004950 auth_method 0 2026-02-17T05:56:48.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c86ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c86ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c87fff640 10 monclient(hunting): get_auth_request con 0x7f9c6c009ab0 auth_method 0 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c87fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c87fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c87fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.753+0000 7f9c86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:48.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 10 monclient(hunting): handle_auth_done global_id 4452 payload 293 2026-02-17T05:56:48.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 1 -- 10.20.193.16:0/2241311540 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c6c003dd0 msgr2=0x7f9c6c0041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 1 --2- 10.20.193.16:0/2241311540 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c6c003dd0 0x7f9c6c0041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 1 -- 10.20.193.16:0/2241311540 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c6c004950 msgr2=0x7f9c6c004db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 1 --2- 10.20.193.16:0/2241311540 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c6c004950 0x7f9c6c004db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 1 monclient: found mon.a 2026-02-17T05:56:48.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 1 -- 10.20.193.16:0/2241311540 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c003620 con 0x7f9c6c009ab0 2026-02-17T05:56:48.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c8df85640 5 monclient: authenticate success, global_id 4452 2026-02-17T05:56:48.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c8df85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.754+0000 7f9c87fff640 1 -- 10.20.193.16:0/2241311540 reap_dead start 2026-02-17T05:56:48.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.755+0000 7f9c87fff640 1 --2- 10.20.193.16:0/2241311540 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c6c009ab0 0x7f9c6c00bea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009d90 tx=0x7f9c7c02f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efcd9a298761607 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c009a10 con 0x7f9c6c009ab0 2026-02-17T05:56:48.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c867fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:48.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c867fc640 10 monclient: dump: 2026-02-17T05:56:48.771 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.771 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.772 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.772 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.772 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.772 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.772 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.772 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.772 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.772 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c003cb0 con 0x7f9c6c009ab0 2026-02-17T05:56:48.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c867fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c867fc640 1 -- 10.20.193.16:0/2241311540 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c003e60 con 0x7f9c6c009ab0 2026-02-17T05:56:48.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c867fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:48.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c867fc640 10 monclient: dump: 2026-02-17T05:56:48.773 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.773 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.773 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.774 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.774 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.774 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.774 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.774 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.774 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.774 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 4 set_mon_vals no callback set 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 10 monclient: shutdown 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 1 -- 10.20.193.16:0/2241311540 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c6c009ab0 msgr2=0x7f9c6c00bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 1 --2- 10.20.193.16:0/2241311540 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c6c009ab0 0x7f9c6c00bea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009d90 tx=0x7f9c7c02f610 comp rx=0 tx=0).stop 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 1 -- 10.20.193.16:0/2241311540 shutdown_connections 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 1 --2- 10.20.193.16:0/2241311540 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c6c009ab0 0x7f9c6c00bea0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 1 -- 10.20.193.16:0/2241311540 >> 10.20.193.16:0/2241311540 conn(0x7f9c880cd770 msgr2=0x7f9c880cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 1 -- 10.20.193.16:0/2241311540 shutdown_connections 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.757+0000 7f9c8df85640 1 -- 10.20.193.16:0/2241311540 wait complete. 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient: monmap: 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.775 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 librados: starting msgr at 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 librados: starting objecter 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 Processor -- start 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 -- start start 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 librados: setting wanted keys 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 librados: calling monclient init 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient: init 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c6c009ab0 0x7f9c8813c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient(hunting): picked mon.noname-c con 0x7f9c6c009ab0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c8813cfb0 0x7f9c881f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient(hunting): picked mon.noname-a con 0x7f9c8813cfb0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c881542c0 0x7f9c8806d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient(hunting): picked mon.noname-b con 0x7f9c881542c0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88074c00 con 0x7f9c8813cfb0 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f9c880c5680 con 0x7f9c881542c0 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88074d80 con 0x7f9c6c009ab0 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.758+0000 7f9c8df85640 10 monclient(hunting): authenticate will time out at 1061.327812s 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c86ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c8813cfb0 0x7f9c881f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c87fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c881542c0 0x7f9c8806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c86ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c8813cfb0 0x7f9c881f00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:60016/0 (socket says 10.20.193.16:60016) 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c86ffd640 1 -- 10.20.193.16:0/3648377350 learned_addr learned my addr 10.20.193.16:0/3648377350 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c86ffd640 10 monclient(hunting): get_auth_request con 0x7f9c8813cfb0 auth_method 0 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c86ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c86ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c87fff640 10 monclient(hunting): get_auth_request con 0x7f9c881542c0 auth_method 0 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c87fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c87fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.759+0000 7f9c87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.760+0000 7f9c86ffd640 10 monclient(hunting): handle_auth_done global_id 4455 payload 824 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.760+0000 7f9c86ffd640 1 -- 10.20.193.16:0/3648377350 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c6c009ab0 msgr2=0x7f9c8813c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.760+0000 7f9c86ffd640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c6c009ab0 0x7f9c8813c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.760+0000 7f9c86ffd640 1 -- 10.20.193.16:0/3648377350 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c881542c0 msgr2=0x7f9c8806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.760+0000 7f9c86ffd640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c881542c0 0x7f9c8806d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.760+0000 7f9c86ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.760+0000 7f9c86ffd640 1 monclient: found mon.noname-a 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.760+0000 7f9c86ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.760+0000 7f9c86ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.760+0000 7f9c86ffd640 1 -- 10.20.193.16:0/3648377350 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c88073690 con 0x7f9c8813cfb0 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.761+0000 7f9c86ffd640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c8813cfb0 0x7f9c881f00a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800eb00 tx=0x7f9c780056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.761+0000 7f9c63fff640 10 client.?.objecter ms_handle_connect 0x7f9c8813cfb0 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.761+0000 7f9c63fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c63fff640 1 -- 10.20.193.16:0/3648377350 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800ca90 con 0x7f9c8813cfb0 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c63fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c63fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c63fff640 10 monclient: dump: 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.779 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c63fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c63fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c8df85640 5 monclient: authenticate success, global_id 4455 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c8df85640 10 monclient: _renew_subs 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c8df85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c8df85640 1 -- 10.20.193.16:0/3648377350 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c88073970 con 0x7f9c8813cfb0 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c8df85640 10 client.4455.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c8df85640 10 monclient: _renew_subs 2026-02-17T05:56:48.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c8df85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c8df85640 1 -- 10.20.193.16:0/3648377350 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c88073f30 con 0x7f9c8813cfb0 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.762+0000 7f9c8df85640 1 librados: init done 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c8df85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c8df85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c8df85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c8df85640 1 -- 10.20.193.16:0/3648377350 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c880704d0 con 0x7f9c8813cfb0 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c63fff640 1 -- 10.20.193.16:0/3648377350 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7800cc30 con 0x7f9c8813cfb0 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c63fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c63fff640 1 -- 10.20.193.16:0/3648377350 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800cde0 con 0x7f9c8813cfb0 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c63fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c63fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c85ffb640 4 set_mon_vals no callback set 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.763+0000 7f9c63fff640 10 monclient: dump: 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.781 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.764+0000 7fb5c8975640 10 monclient: get_monmap_and_config 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.764+0000 7fb5c8975640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.764+0000 7fb5c8975640 10 monclient: monmap: 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:48.765761+0000 2026-02-17T05:56:48.782 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:48.765761+0000 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.764+0000 7f40de7fc640 1 -- 10.20.193.16:0/3817797741 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f40ec062530 con 0x7f40f01536f0 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.764+0000 7f40de7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.764+0000 7f40de7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:48.785 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-17T05:56:48.785 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2026-02-17T05:56:48.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 1 Processor -- start 2026-02-17T05:56:48.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 1 -- start start 2026-02-17T05:56:48.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 10 monclient: init 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 10 monclient: _add_conns ranks=[3,1,5,2,0,4] 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb5c0155e00 0x7fb5c00ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 10 monclient(hunting): picked mon.noname-a con 0x7fb5c0155e00 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb5c01561e0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fb5c0074dc0 0x7fb5c00cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 10 monclient(hunting): picked mon.noname-c con 0x7fb5c0074dc0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c00c96d0 con 0x7fb5c01561e0 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c0074860 con 0x7fb5c0155e00 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c00c9940 con 0x7fb5c0074dc0 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.765+0000 7fb5c8975640 10 monclient(hunting): authenticate will time out at 1061.334813s 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.766+0000 7fb5c5ee9640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fb5c01561e0 0x7fb5c0074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:53890/0 (socket says 10.20.193.16:53890) 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.766+0000 7fb5c5ee9640 1 -- 10.20.193.16:0/2767893043 learned_addr learned my addr 10.20.193.16:0/2767893043 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.767+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573634313 0 0) 0x7fb5c00c96d0 con 0x7fb5c01561e0 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.767+0000 7fb5c56e8640 10 monclient(hunting): _negotiate 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.767+0000 7fb5c56e8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.767+0000 7fb5c56e8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.767+0000 7fb5c56e8640 10 monclient(hunting): my global_id is 4531 2026-02-17T05:56:48.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.767+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5b0003620 con 0x7fb5c01561e0 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c6eeb640 1 --2- 10.20.193.16:0/2767893043 >> v2:10.20.193.148:3300/0 conn(0x7fb5c0074dc0 0x7fb5c00cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c6eeb640 10 monclient(hunting): get_auth_request con 0x7fb5c0074dc0 auth_method 0 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c6eeb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c6eeb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c6eeb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 <== mon.1 v1:10.20.193.128:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1495201600 0 0) 0x7fb5b0003620 con 0x7fb5c01561e0 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 >> v2:10.20.193.148:3300/0 conn(0x7fb5c0074dc0 msgr2=0x7fb5c00cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 --2- 10.20.193.16:0/2767893043 >> v2:10.20.193.148:3300/0 conn(0x7fb5c0074dc0 0x7fb5c00cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 >> v2:10.20.193.16:3300/0 conn(0x7fb5c0155e00 msgr2=0x7fb5c00ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 --2- 10.20.193.16:0/2767893043 >> v2:10.20.193.16:3300/0 conn(0x7fb5c0155e00 0x7fb5c00ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 monclient: found mon.noname-b-legacy 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.128:6789/0 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 --> v1:10.20.193.128:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c00c9c20 con 0x7fb5c01561e0 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 <== mon.1 v1:10.20.193.128:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7fb5ac002f00 con 0x7fb5c01561e0 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient: dump: 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.791 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] but i'm connected to v1:10.20.193.128:6789/0, reconnecting 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 >> v1:10.20.193.128:6789/0 conn(0x7fb5c01561e0 legacy=0x7fb5c0074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 --2- 10.20.193.16:0/2767893043 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5b0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient(hunting): picked mon.c con 0x7fb5b0003bf0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 --2- 10.20.193.16:0/2767893043 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5b0004900 0x7fb5b0004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient(hunting): picked mon.a con 0x7fb5b0004900 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 --2- 10.20.193.16:0/2767893043 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5b00054a0 0x7fb5b000bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient(hunting): picked mon.b con 0x7fb5b00054a0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c0074860 con 0x7fb5b0004900 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00c9940 con 0x7fb5b00054a0 2026-02-17T05:56:48.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b000c460 con 0x7fb5b0003bf0 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.768+0000 7fb5c56e8640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.769+0000 7fb5c66ea640 1 --2- 10.20.193.16:0/2767893043 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5b0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.769+0000 7fb5c66ea640 10 monclient(hunting): get_auth_request con 0x7fb5b0003bf0 auth_method 0 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.769+0000 7fb5c66ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.769+0000 7fb5c66ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.769+0000 7fb5c66ea640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.769+0000 7fb5c66ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.769+0000 7fb5c66ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.769+0000 7fb5c66ea640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c6eeb640 1 --2- 10.20.193.16:0/2767893043 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5b00054a0 0x7fb5b000bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c6eeb640 10 monclient(hunting): get_auth_request con 0x7fb5b00054a0 auth_method 0 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c6eeb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c6eeb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c6eeb640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c6eeb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c6eeb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c6eeb640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 10 monclient(hunting): handle_auth_done global_id 4531 payload 293 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 1 -- 10.20.193.16:0/2767893043 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5b00054a0 msgr2=0x7fb5b000bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 1 --2- 10.20.193.16:0/2767893043 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5b00054a0 0x7fb5b000bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 1 -- 10.20.193.16:0/2767893043 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5b0004900 msgr2=0x7fb5b0004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 1 --2- 10.20.193.16:0/2767893043 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5b0004900 0x7fb5b0004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 1 monclient: found mon.c 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 1 -- 10.20.193.16:0/2767893043 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b0003620 con 0x7fb5b0003bf0 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c6eeb640 1 -- 10.20.193.16:0/2767893043 reap_dead start 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c66ea640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c8975640 5 monclient: authenticate success, global_id 4531 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.770+0000 7fb5c8975640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.771+0000 7fb5c66ea640 1 --2- 10.20.193.16:0/2767893043 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5b0004010 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009980 tx=0x7fb5b802ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a72ec1c540be70e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.772+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b803c070 con 0x7fb5b0003bf0 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.772+0000 7fb5c56e8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.772+0000 7fb5c56e8640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.772+0000 7fb5c56e8640 10 monclient: dump: 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.794 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.772+0000 7fb5c8975640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b802f770 con 0x7fb5b0003bf0 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c56e8640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c56e8640 1 -- 10.20.193.16:0/2767893043 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8009550 con 0x7fb5b0003bf0 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c56e8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c56e8640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c56e8640 10 monclient: dump: 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 4 set_mon_vals no callback set 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 10 monclient: shutdown 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 1 -- 10.20.193.16:0/2767893043 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 msgr2=0x7fb5b0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 1 --2- 10.20.193.16:0/2767893043 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5b0004010 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009980 tx=0x7fb5b802ed10 comp rx=0 tx=0).stop 2026-02-17T05:56:48.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 1 -- 10.20.193.16:0/2767893043 shutdown_connections 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 1 --2- 10.20.193.16:0/2767893043 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5b0004010 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 1 -- 10.20.193.16:0/2767893043 >> 10.20.193.16:0/2767893043 conn(0x7fb5c00ce6c0 msgr2=0x7fb5c00cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 1 -- 10.20.193.16:0/2767893043 shutdown_connections 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.773+0000 7fb5c8975640 1 -- 10.20.193.16:0/2767893043 wait complete. 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient: monmap: 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 librados: starting msgr at 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 librados: starting objecter 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 Processor -- start 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 -- start start 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 librados: setting wanted keys 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 librados: calling monclient init 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient: init 2026-02-17T05:56:48.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5c0144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient(hunting): picked mon.noname-c con 0x7fb5b0003bf0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5c01f3c30 0x7fb5c0149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient(hunting): picked mon.noname-b con 0x7fb5c01f3c30 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5c0149da0 0x7fb5c00ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient(hunting): picked mon.noname-a con 0x7fb5c0149da0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00c9360 con 0x7fb5c0149da0 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00ca1e0 con 0x7fb5c01f3c30 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00ca700 con 0x7fb5b0003bf0 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.774+0000 7fb5c8975640 10 monclient(hunting): authenticate will time out at 1061.343860s 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c66ea640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5c0144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c5ee9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5c01f3c30 0x7fb5c0149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c66ea640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5c0144350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57268/0 (socket says 10.20.193.16:57268) 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c66ea640 1 -- 10.20.193.16:0/2904989 learned_addr learned my addr 10.20.193.16:0/2904989 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c66ea640 10 monclient(hunting): get_auth_request con 0x7fb5b0003bf0 auth_method 0 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c66ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c66ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c66ea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c5ee9640 10 monclient(hunting): get_auth_request con 0x7fb5c01f3c30 auth_method 0 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c5ee9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c5ee9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c5ee9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c66ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c66ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c66ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c5ee9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c5ee9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.775+0000 7fb5c5ee9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.776+0000 7fb5c66ea640 10 monclient(hunting): handle_auth_done global_id 4583 payload 824 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.776+0000 7fb5c66ea640 1 -- 10.20.193.16:0/2904989 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5c01f3c30 msgr2=0x7fb5c0149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.776+0000 7fb5c66ea640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5c01f3c30 0x7fb5c0149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.776+0000 7fb5c66ea640 1 -- 10.20.193.16:0/2904989 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5c0149da0 msgr2=0x7fb5c00ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.776+0000 7fb5c66ea640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5c0149da0 0x7fb5c00ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.776+0000 7fb5c66ea640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.776+0000 7fb5c66ea640 1 monclient: found mon.noname-c 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.776+0000 7fb5c66ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.776+0000 7fb5c66ea640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.776+0000 7fb5c66ea640 1 -- 10.20.193.16:0/2904989 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c014a220 con 0x7fb5b0003bf0 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.777+0000 7fb5aaffd640 10 client.?.objecter ms_handle_connect 0x7fb5b0003bf0 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.777+0000 7fb5c66ea640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5c0144350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb5b803a040 tx=0x7fb5b80048d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.777+0000 7fb5aaffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5aaffd640 1 -- 10.20.193.16:0/2904989 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b803c050 con 0x7fb5b0003bf0 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5aaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5aaffd640 10 monclient: dump: 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5aaffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5aaffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5c8975640 5 monclient: authenticate success, global_id 4583 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5c8975640 10 monclient: _renew_subs 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5c8975640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5c8975640 1 -- 10.20.193.16:0/2904989 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c01490a0 con 0x7fb5b0003bf0 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5c8975640 10 client.4583.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5c8975640 10 monclient: _renew_subs 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5c8975640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5c8975640 1 -- 10.20.193.16:0/2904989 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c01f4440 con 0x7fb5b0003bf0 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.778+0000 7fb5c8975640 1 librados: init done 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5aaffd640 1 -- 10.20.193.16:0/2904989 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b8048070 con 0x7fb5b0003bf0 2026-02-17T05:56:48.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5aaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5aaffd640 1 -- 10.20.193.16:0/2904989 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8006e60 con 0x7fb5b0003bf0 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5aaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5c4ee7640 4 set_mon_vals no callback set 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5aaffd640 10 monclient: dump: 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5c8975640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5c8975640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5c8975640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.779+0000 7fb5c8975640 1 -- 10.20.193.16:0/2904989 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c00caa80 con 0x7fb5b0003bf0 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.790+0000 7fb5aaffd640 1 -- 10.20.193.16:0/2904989 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fb5b8037c00 con 0x7fb5b0003bf0 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.790+0000 7fb5aaffd640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb590079170 0x7fb59007b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.790+0000 7fb5c5ee9640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb590079170 0x7fb59007b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.790+0000 7fb5c5ee9640 10 monclient: get_auth_request con 0x7fb590079170 auth_method 0 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.790+0000 7fb5aaffd640 10 client.4583.objecter ms_handle_connect 0x7fb590079170 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.790+0000 7fb5c5ee9640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb590079170 0x7fb59007b630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac002eb0 tx=0x7fb5ac002da0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.791+0000 7fb5aaffd640 1 -- 10.20.193.16:0/2904989 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fb5b80bd0b0 con 0x7fb5b0003bf0 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.791+0000 7fb5aaffd640 10 client.4583.objecter ms_dispatch2 0x7fb5c0000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.791+0000 7fb5aaffd640 3 client.4583.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.791+0000 7fb5aaffd640 3 client.4583.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:48.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.791+0000 7fb5aaffd640 20 client.4583.objecter dump_active .. 0 homeless 2026-02-17T05:56:48.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.823+0000 7f043338f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:56:48.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.823+0000 7f043338f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:56:48.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.823+0000 7f043338f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.823+0000 7f043338f640 1 -- 10.20.193.16:0/165260077 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f042c0cb860 con 0x7f0414003bf0 2026-02-17T05:56:48.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.826+0000 7f0419ffb640 1 -- 10.20.193.16:0/165260077 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f041c08ad30 con 0x7f0414003bf0 2026-02-17T05:56:48.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.826+0000 7f0419ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:56:48.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.826+0000 7f0419ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:48.827 INFO:teuthology.orchestra.run.trial016.stdout:120259084294 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 10 librados: watch_flush enter 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 10 librados: watch_flush exit 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 20 client.4522.objecter shutdown clearing up homeless session... 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 10 client.4522.objecter successfully canceled tick 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 -- 10.20.193.16:0/165260077 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f03fc079170 msgr2=0x7f03fc07b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f03fc079170 0x7f03fc07b630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0420002410 tx=0x7f0420007dc0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 10 monclient: shutdown 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 -- 10.20.193.16:0/165260077 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 msgr2=0x7f042c0c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 0x7f042c0c5dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f041c033000 tx=0x7f042c0ca7b0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 -- 10.20.193.16:0/165260077 shutdown_connections 2026-02-17T05:56:48.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f03fc079170 0x7f03fc07b630 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f042c1e7210 0x7f042c15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f042c1f8350 0x7f042c1e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 --2- 10.20.193.16:0/165260077 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0414003bf0 0x7f042c0c5dd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 -- 10.20.193.16:0/165260077 >> 10.20.193.16:0/165260077 conn(0x7f042c0cf600 msgr2=0x7f042c15dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 -- 10.20.193.16:0/165260077 shutdown_connections 2026-02-17T05:56:48.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 -- 10.20.193.16:0/165260077 wait complete. 2026-02-17T05:56:48.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.828+0000 7f043338f640 1 librados: shutdown 2026-02-17T05:56:48.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084295 got 120259084294 for osd.0 2026-02-17T05:56:48.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.851+0000 7f40f7d9e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:56:48.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.851+0000 7f40f7d9e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:56:48.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.851+0000 7f40f7d9e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.851+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3817797741 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f40f013c3f0 con 0x7f40f01536f0 2026-02-17T05:56:48.858 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.866+0000 7eff81488640 10 monclient: get_monmap_and_config 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.866+0000 7eff81488640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.866+0000 7eff81488640 10 monclient: monmap: 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:48.867156+0000 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:48.867156+0000 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:48.866 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.867 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.866+0000 7eff81488640 1 Processor -- start 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.866+0000 7eff81488640 1 -- start start 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.866+0000 7eff81488640 10 monclient: init 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 10 monclient: _add_conns ranks=[5,2,4,0,3,1] 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7eff7c0700f0 0x7eff7c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 10 monclient(hunting): picked mon.noname-c con 0x7eff7c0700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7eff7c0c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7eff7c1536f0 0x7eff7c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 10 monclient(hunting): picked mon.noname-b con 0x7eff7c1536f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff7c0c4820 con 0x7eff7c0c7fe0 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7eff7c0c4b00 con 0x7eff7c1536f0 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7eff7c0c4de0 con 0x7eff7c0700f0 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff81488640 10 monclient(hunting): authenticate will time out at 1061.436139s 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff7b7fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7eff7c1536f0 0x7eff7c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff7b7fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7eff7c1536f0 0x7eff7c153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50564/0 (socket says 10.20.193.16:50564) 2026-02-17T05:56:48.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff7b7fe640 1 -- 10.20.193.16:0/2820328141 learned_addr learned my addr 10.20.193.16:0/2820328141 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff7b7fe640 10 monclient(hunting): get_auth_request con 0x7eff7c1536f0 auth_method 0 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7f54e9d0a640 10 monclient: get_monmap_and_config 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7f54e9d0a640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7f54e9d0a640 10 monclient: monmap: 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:48.868746+0000 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:48.868746+0000 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:48.869 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.867+0000 7eff7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7eff7b7fe640 10 monclient(hunting): handle_auth_done global_id 4537 payload 274 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7eff7b7fe640 1 -- 10.20.193.16:0/2820328141 >> v2:10.20.193.148:3300/0 conn(0x7eff7c0700f0 msgr2=0x7eff7c0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7eff7b7fe640 1 --2- 10.20.193.16:0/2820328141 >> v2:10.20.193.148:3300/0 conn(0x7eff7c0700f0 0x7eff7c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7eff7b7fe640 1 -- 10.20.193.16:0/2820328141 >> v1:10.20.193.148:6789/0 conn(0x7eff7c0c7fe0 legacy=0x7eff7c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7eff7b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7eff7b7fe640 1 monclient: found mon.noname-b 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7eff7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7eff7b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7eff7b7fe640 1 -- 10.20.193.16:0/2820328141 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7c0c50c0 con 0x7eff7c1536f0 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 1 Processor -- start 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 1 -- start start 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 10 monclient: init 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 10 monclient: _add_conns ranks=[5,2,3,1,0,4] 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f54e40700f0 0x7f54e40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 10 monclient(hunting): picked mon.noname-c con 0x7f54e40700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f54e40c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f54e41536f0 0x7f54e4153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 10 monclient(hunting): picked mon.noname-a con 0x7f54e41536f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e40c4820 con 0x7f54e40c7fe0 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f54e40c4b00 con 0x7f54e41536f0 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f54e40c4de0 con 0x7f54e40700f0 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.868+0000 7f54e9d0a640 10 monclient(hunting): authenticate will time out at 1061.437829s 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.869+0000 7eff7b7fe640 1 --2- 10.20.193.16:0/2820328141 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c1536f0 0x7eff7c153b20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff7c070b90 tx=0x7eff6c02f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=958c39b17c10e6c0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.870+0000 7eff79ffb640 1 -- 10.20.193.16:0/2820328141 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c02fc50 con 0x7eff7c1536f0 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.870+0000 7eff79ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.870+0000 7eff79ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.870+0000 7eff79ffb640 10 monclient: dump: 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.870+0000 7eff79ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.870+0000 7eff79ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.870+0000 7eff79ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.870+0000 7eff81488640 5 monclient: authenticate success, global_id 4537 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.870+0000 7eff81488640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.871+0000 7eff79ffb640 1 -- 10.20.193.16:0/2820328141 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff6c02fdf0 con 0x7eff7c1536f0 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.871+0000 7eff79ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.871+0000 7eff79ffb640 1 -- 10.20.193.16:0/2820328141 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c037610 con 0x7eff7c1536f0 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.871+0000 7eff79ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.871+0000 7eff79ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.871+0000 7eff79ffb640 10 monclient: dump: 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.871+0000 7eff81488640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.871+0000 7eff81488640 4 set_mon_vals no callback set 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 10 monclient: shutdown 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 -- 10.20.193.16:0/2820328141 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c1536f0 msgr2=0x7eff7c153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 --2- 10.20.193.16:0/2820328141 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c1536f0 0x7eff7c153b20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff7c070b90 tx=0x7eff6c02f230 comp rx=0 tx=0).stop 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 -- 10.20.193.16:0/2820328141 shutdown_connections 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 --2- 10.20.193.16:0/2820328141 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c1536f0 0x7eff7c153b20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 --2- 10.20.193.16:0/2820328141 >> v2:10.20.193.148:3300/0 conn(0x7eff7c0700f0 0x7eff7c0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 -- 10.20.193.16:0/2820328141 >> 10.20.193.16:0/2820328141 conn(0x7eff7c0cd770 msgr2=0x7eff7c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 -- 10.20.193.16:0/2820328141 shutdown_connections 2026-02-17T05:56:48.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 -- 10.20.193.16:0/2820328141 wait complete. 2026-02-17T05:56:48.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 10 monclient: monmap: 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 librados: starting msgr at 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 librados: starting objecter 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 Processor -- start 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 -- start start 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 librados: setting wanted keys 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 1 librados: calling monclient init 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.872+0000 7eff81488640 10 monclient: init 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c0700f0 0x7eff7c1e7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 10 monclient(hunting): picked mon.noname-b con 0x7eff7c0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7eff7c0c7fe0 0x7eff7c1f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 10 monclient(hunting): picked mon.noname-a con 0x7eff7c0c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7eff7c1536f0 0x7eff7c1f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 10 monclient(hunting): picked mon.noname-c con 0x7eff7c1536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c154060 con 0x7eff7c0c7fe0 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c0c5680 con 0x7eff7c0700f0 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c074c00 con 0x7eff7c1536f0 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff81488640 10 monclient(hunting): authenticate will time out at 1061.442416s 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7f9c63fff640 1 -- 10.20.193.16:0/3648377350 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f9c780068d0 con 0x7f9c8813cfb0 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff7affd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c0700f0 0x7eff7c1e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff7affd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c0700f0 0x7eff7c1e7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50568/0 (socket says 10.20.193.16:50568) 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff7affd640 1 -- 10.20.193.16:0/1259153303 learned_addr learned my addr 10.20.193.16:0/1259153303 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff7affd640 10 monclient(hunting): get_auth_request con 0x7eff7c0700f0 auth_method 0 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff7affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.873+0000 7eff7affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.874+0000 7f9c63fff640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c58079170 0x7f9c5807b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.874+0000 7eff7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.874+0000 7eff7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.874+0000 7eff7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.874+0000 7f9c877fe640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c58079170 0x7f9c5807b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.874+0000 7f9c877fe640 10 monclient: get_auth_request con 0x7f9c58079170 auth_method 0 2026-02-17T05:56:48.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.874+0000 7f9c63fff640 10 client.4455.objecter ms_handle_connect 0x7f9c58079170 2026-02-17T05:56:48.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.874+0000 7f9c877fe640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c58079170 0x7f9c5807b630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c740036c0 tx=0x7f9c74002d00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.875+0000 7eff7affd640 10 monclient(hunting): handle_auth_done global_id 4540 payload 824 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.875+0000 7eff7affd640 1 -- 10.20.193.16:0/1259153303 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7eff7c1536f0 msgr2=0x7eff7c1f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.875+0000 7eff7affd640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7eff7c1536f0 0x7eff7c1f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.875+0000 7eff7affd640 1 -- 10.20.193.16:0/1259153303 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7eff7c0c7fe0 msgr2=0x7eff7c1f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.875+0000 7eff7affd640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7eff7c0c7fe0 0x7eff7c1f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.875+0000 7eff7affd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.875+0000 7eff7affd640 1 monclient: found mon.noname-b 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.875+0000 7eff7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.875+0000 7eff7affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.875+0000 7eff7affd640 1 -- 10.20.193.16:0/1259153303 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7c1e65a0 con 0x7eff7c0700f0 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.876+0000 7eff7affd640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c0700f0 0x7eff7c1e7920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff7000b500 tx=0x7eff7000b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.876+0000 7eff5b7fe640 10 client.?.objecter ms_handle_connect 0x7eff7c0700f0 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.876+0000 7eff5b7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff5b7fe640 1 -- 10.20.193.16:0/1259153303 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70004480 con 0x7eff7c0700f0 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff5b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff5b7fe640 10 monclient: dump: 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.878 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff5b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff5b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff5b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 5 monclient: authenticate success, global_id 4540 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 10 monclient: _renew_subs 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 1 -- 10.20.193.16:0/1259153303 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7c1e6880 con 0x7eff7c0700f0 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 10 client.4540.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 10 monclient: _renew_subs 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 1 -- 10.20.193.16:0/1259153303 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7c1e6e40 con 0x7eff7c0700f0 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 1 librados: init done 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.877+0000 7eff81488640 1 -- 10.20.193.16:0/1259153303 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff7c0704d0 con 0x7eff7c0700f0 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.878+0000 7eff5b7fe640 1 -- 10.20.193.16:0/1259153303 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff70014070 con 0x7eff7c0700f0 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.878+0000 7eff5b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.878+0000 7eff5b7fe640 1 -- 10.20.193.16:0/1259153303 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70002900 con 0x7eff7c0700f0 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.878+0000 7eff5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.878+0000 7eff5b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.878+0000 7eff797fa640 4 set_mon_vals no callback set 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.878+0000 7eff5b7fe640 10 monclient: dump: 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.879 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.880 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.880 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.880 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.883+0000 7f9c63fff640 1 -- 10.20.193.16:0/3648377350 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f9c78098060 con 0x7f9c8813cfb0 2026-02-17T05:56:48.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.883+0000 7f9c63fff640 10 client.4455.objecter ms_dispatch2 0x7f9c88000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:48.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.883+0000 7f9c63fff640 3 client.4455.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:48.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.883+0000 7f9c63fff640 3 client.4455.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:48.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.883+0000 7f9c63fff640 20 client.4455.objecter dump_active .. 0 homeless 2026-02-17T05:56:48.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.883+0000 7f9c63fff640 1 -- 10.20.193.16:0/3648377350 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9c78062ce0 con 0x7f9c8813cfb0 2026-02-17T05:56:48.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.883+0000 7f9c63fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.883+0000 7f9c63fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:48.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.887+0000 7f54e3fff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f54e41536f0 0x7f54e4153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.887+0000 7f54e3fff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f54e41536f0 0x7f54e4153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:60050/0 (socket says 10.20.193.16:60050) 2026-02-17T05:56:48.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.887+0000 7f54e3fff640 1 -- 10.20.193.16:0/2537194221 learned_addr learned my addr 10.20.193.16:0/2537194221 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.887+0000 7f54e3fff640 10 monclient(hunting): get_auth_request con 0x7f54e41536f0 auth_method 0 2026-02-17T05:56:48.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.887+0000 7f54e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.887+0000 7f54e3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.887+0000 7f54e3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.888+0000 7f54e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.888+0000 7f54e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.888+0000 7f54e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.889+0000 7f54e3fff640 10 monclient(hunting): handle_auth_done global_id 4458 payload 274 2026-02-17T05:56:48.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.889+0000 7f54e3fff640 1 -- 10.20.193.16:0/2537194221 >> v2:10.20.193.148:3300/0 conn(0x7f54e40700f0 msgr2=0x7f54e40704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.889+0000 7f54e3fff640 1 --2- 10.20.193.16:0/2537194221 >> v2:10.20.193.148:3300/0 conn(0x7f54e40700f0 0x7f54e40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.889+0000 7f54e3fff640 1 -- 10.20.193.16:0/2537194221 >> v1:10.20.193.148:6789/0 conn(0x7f54e40c7fe0 legacy=0x7f54e40c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.889+0000 7f54e3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.889+0000 7f54e3fff640 1 monclient: found mon.noname-a 2026-02-17T05:56:48.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.889+0000 7f54e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.889+0000 7f54e3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.889+0000 7f54e3fff640 1 -- 10.20.193.16:0/2537194221 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e40c50c0 con 0x7f54e41536f0 2026-02-17T05:56:48.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.890+0000 7f54e3fff640 1 --2- 10.20.193.16:0/2537194221 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e41536f0 0x7f54e4153b20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f54e4070b90 tx=0x7f54d802f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ebc74da92a602 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.891+0000 7fb5aaffd640 1 -- 10.20.193.16:0/2904989 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb5c014a220 con 0x7fb5b0003bf0 2026-02-17T05:56:48.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.891+0000 7fb5aaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.891+0000 7fb5aaffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 1 -- 10.20.193.16:0/2537194221 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d802fc50 con 0x7f54e41536f0 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 10 monclient: dump: 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 1 -- 10.20.193.16:0/2537194221 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d802fdf0 con 0x7f54e41536f0 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 1 -- 10.20.193.16:0/2537194221 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8037610 con 0x7f54e41536f0 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e9d0a640 5 monclient: authenticate success, global_id 4458 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e9d0a640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e9d0a640 4 set_mon_vals no callback set 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.892+0000 7f54e27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e27fc640 10 monclient: dump: 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.894 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e9d0a640 10 monclient: shutdown 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e9d0a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e9d0a640 1 -- 10.20.193.16:0/2537194221 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e41536f0 msgr2=0x7f54e4153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e9d0a640 1 --2- 10.20.193.16:0/2537194221 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e41536f0 0x7f54e4153b20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f54e4070b90 tx=0x7f54d802f230 comp rx=0 tx=0).stop 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e9d0a640 1 -- 10.20.193.16:0/2537194221 shutdown_connections 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e9d0a640 1 --2- 10.20.193.16:0/2537194221 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e41536f0 0x7f54e4153b20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e9d0a640 1 --2- 10.20.193.16:0/2537194221 >> v2:10.20.193.148:3300/0 conn(0x7f54e40700f0 0x7f54e40704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e9d0a640 1 -- 10.20.193.16:0/2537194221 >> 10.20.193.16:0/2537194221 conn(0x7f54e40cd770 msgr2=0x7f54e40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e9d0a640 1 -- 10.20.193.16:0/2537194221 shutdown_connections 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.893+0000 7f54e9d0a640 1 -- 10.20.193.16:0/2537194221 wait complete. 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f40de7fc640 1 -- 10.20.193.16:0/3817797741 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40ec061e30 con 0x7f40f01536f0 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f40de7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:56:48.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f40de7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient: monmap: 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.896 INFO:teuthology.orchestra.run.trial016.stdout:98784247811 2026-02-17T05:56:48.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 librados: starting msgr at 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 librados: starting objecter 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 Processor -- start 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 -- start start 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 librados: setting wanted keys 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 librados: calling monclient init 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient: init 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e40700f0 0x7f54e406d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient(hunting): picked mon.noname-a con 0x7f54e40700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f54e40c7fe0 0x7f54e4073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient(hunting): picked mon.noname-b con 0x7f54e40c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f54e41536f0 0x7f54e406e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient(hunting): picked mon.noname-c con 0x7f54e41536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54e4154060 con 0x7f54e40700f0 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40c5680 con 0x7f54e40c7fe0 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54e4074c00 con 0x7f54e41536f0 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.894+0000 7f54e9d0a640 10 monclient(hunting): authenticate will time out at 1061.463842s 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.895+0000 7f54e37fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e40700f0 0x7f54e406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.895+0000 7f54e37fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e40700f0 0x7f54e406d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:60072/0 (socket says 10.20.193.16:60072) 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.895+0000 7f54e37fe640 1 -- 10.20.193.16:0/1229688593 learned_addr learned my addr 10.20.193.16:0/1229688593 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.895+0000 7f54e37fe640 10 monclient(hunting): get_auth_request con 0x7f54e40700f0 auth_method 0 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.895+0000 7f54e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.895+0000 7f54e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.895+0000 7f54e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.895+0000 7f54e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.895+0000 7f54e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.895+0000 7f54e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 10 librados: watch_flush enter 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 10 librados: watch_flush exit 2026-02-17T05:56:48.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 20 client.4577.objecter shutdown clearing up homeless session... 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 10 client.4577.objecter successfully canceled tick 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3817797741 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f40c0079190 msgr2=0x7f40c007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f40c0079190 0x7f40c007b650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40f00c9540 tx=0x7f40e0031000 comp rx=0 tx=0).stop 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 10 monclient: shutdown 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3817797741 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f01536f0 msgr2=0x7f40f01e8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f01536f0 0x7f40f01e8260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00b990 tx=0x7f40ec00be60 comp rx=0 tx=0).stop 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3817797741 shutdown_connections 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f40c0079190 0x7f40c007b650 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f40f01536f0 0x7f40f01e8260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f40f00c7fe0 0x7f40f013e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 --2- 10.20.193.16:0/3817797741 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f40f00700f0 0x7f40f00c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3817797741 >> 10.20.193.16:0/3817797741 conn(0x7f40f00cd770 msgr2=0x7f40f0153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3817797741 shutdown_connections 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 -- 10.20.193.16:0/3817797741 wait complete. 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.896+0000 7f40f7d9e640 1 librados: shutdown 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.897+0000 7f54e37fe640 10 monclient(hunting): handle_auth_done global_id 4461 payload 824 2026-02-17T05:56:48.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.897+0000 7f54e37fe640 1 -- 10.20.193.16:0/1229688593 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f54e41536f0 msgr2=0x7f54e406e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.897+0000 7f54e37fe640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f54e41536f0 0x7f54e406e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.897+0000 7f54e37fe640 1 -- 10.20.193.16:0/1229688593 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f54e40c7fe0 msgr2=0x7f54e4073f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.897+0000 7f54e37fe640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f54e40c7fe0 0x7f54e4073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.897+0000 7f54e37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.897+0000 7f54e37fe640 1 monclient: found mon.noname-a 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.897+0000 7f54e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.897+0000 7f54e37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.897+0000 7f54e37fe640 1 -- 10.20.193.16:0/1229688593 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e41f35f0 con 0x7f54e40700f0 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.898+0000 7f54e37fe640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e40700f0 0x7f54e406d710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54d000b530 tx=0x7f54d000ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.898+0000 7f54bffff640 10 client.?.objecter ms_handle_connect 0x7f54e40700f0 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.898+0000 7f54bffff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 1 -- 10.20.193.16:0/1229688593 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0013020 con 0x7f54e40700f0 2026-02-17T05:56:48.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 10 monclient: dump: 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 1 -- 10.20.193.16:0/1229688593 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d0004480 con 0x7f54e40700f0 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54e9d0a640 5 monclient: authenticate success, global_id 4461 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54e9d0a640 10 monclient: _renew_subs 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54e9d0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54e9d0a640 1 -- 10.20.193.16:0/1229688593 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e41f38d0 con 0x7f54e40700f0 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 1 -- 10.20.193.16:0/1229688593 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0002920 con 0x7f54e40700f0 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54bffff640 10 monclient: dump: 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.902 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54e1ffb640 4 set_mon_vals no callback set 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54e9d0a640 10 client.4461.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54e9d0a640 10 monclient: _renew_subs 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54e9d0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54e9d0a640 1 -- 10.20.193.16:0/1229688593 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e41f3e90 con 0x7f54e40700f0 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.901+0000 7f54e9d0a640 1 librados: init done 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.902+0000 7f54e9d0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.902+0000 7f54e9d0a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.902+0000 7f54e9d0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.902+0000 7f54e9d0a640 1 -- 10.20.193.16:0/1229688593 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e40704d0 con 0x7f54e40700f0 2026-02-17T05:56:48.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.3 2026-02-17T05:56:48.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.928+0000 7fb50a985640 10 monclient: get_monmap_and_config 2026-02-17T05:56:48.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.928+0000 7fb50a985640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.928+0000 7fb50a985640 10 monclient: monmap: 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:48.929669+0000 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:48.929669+0000 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 1 Processor -- start 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 1 -- start start 2026-02-17T05:56:48.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 10 monclient: init 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 10 monclient: _add_conns ranks=[5,1,4,0,3,2] 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fb504155e00 0x7fb5040ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 10 monclient(hunting): picked mon.noname-c con 0x7fb504155e00 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb5041561e0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fb504074dc0 0x7fb5040cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 10 monclient(hunting): picked mon.noname-b con 0x7fb504074dc0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5040c96d0 con 0x7fb5041561e0 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fb504074860 con 0x7fb504074dc0 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb5040c9940 con 0x7fb504155e00 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.929+0000 7fb50a985640 10 monclient(hunting): authenticate will time out at 1061.498935s 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.930+0000 7fb503fff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fb504155e00 0x7fb5040ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.930+0000 7fb503fff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fb504155e00 0x7fb5040ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57304/0 (socket says 10.20.193.16:57304) 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.930+0000 7fb503fff640 1 -- 10.20.193.16:0/1639702547 learned_addr learned my addr 10.20.193.16:0/1639702547 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.930+0000 7fb503fff640 10 monclient(hunting): get_auth_request con 0x7fb504155e00 auth_method 0 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.930+0000 7fb503fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.930+0000 7fb503fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.930+0000 7fb503fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.930+0000 7fb503fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.930+0000 7fb503fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.930+0000 7fb503fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.931+0000 7fb503fff640 10 monclient(hunting): handle_auth_done global_id 4586 payload 274 2026-02-17T05:56:48.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.931+0000 7fb503fff640 1 -- 10.20.193.16:0/1639702547 >> v2:10.20.193.128:3300/0 conn(0x7fb504074dc0 msgr2=0x7fb5040cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.931+0000 7fb503fff640 1 --2- 10.20.193.16:0/1639702547 >> v2:10.20.193.128:3300/0 conn(0x7fb504074dc0 0x7fb5040cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.931+0000 7fb503fff640 1 -- 10.20.193.16:0/1639702547 >> v1:10.20.193.128:6789/0 conn(0x7fb5041561e0 legacy=0x7fb504074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.931+0000 7fb503fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.931+0000 7fb503fff640 1 monclient: found mon.noname-c 2026-02-17T05:56:48.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.931+0000 7fb503fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.931+0000 7fb503fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.931+0000 7fb503fff640 1 -- 10.20.193.16:0/1639702547 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5040c9c20 con 0x7fb504155e00 2026-02-17T05:56:48.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.932+0000 7fb503fff640 1 --2- 10.20.193.16:0/1639702547 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 0x7fb5040ca700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb50406ce20 tx=0x7fb4f002eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c0487366ab2178 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 1 -- 10.20.193.16:0/1639702547 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f003c070 con 0x7fb504155e00 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 10 monclient: dump: 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.934 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 5 monclient: authenticate success, global_id 4586 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 1 -- 10.20.193.16:0/1639702547 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f002fb40 con 0x7fb504155e00 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 1 -- 10.20.193.16:0/1639702547 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0037440 con 0x7fb504155e00 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 4 set_mon_vals no callback set 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb502ffd640 10 monclient: dump: 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.935 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 10 monclient: shutdown 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 1 -- 10.20.193.16:0/1639702547 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 msgr2=0x7fb5040ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 1 --2- 10.20.193.16:0/1639702547 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 0x7fb5040ca700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb50406ce20 tx=0x7fb4f002eed0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 1 -- 10.20.193.16:0/1639702547 shutdown_connections 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 1 --2- 10.20.193.16:0/1639702547 >> v2:10.20.193.128:3300/0 conn(0x7fb504074dc0 0x7fb5040cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 1 --2- 10.20.193.16:0/1639702547 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 0x7fb5040ca700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 1 -- 10.20.193.16:0/1639702547 >> 10.20.193.16:0/1639702547 conn(0x7fb5040ce6c0 msgr2=0x7fb5040cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 1 -- 10.20.193.16:0/1639702547 shutdown_connections 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.934+0000 7fb50a985640 1 -- 10.20.193.16:0/1639702547 wait complete. 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.935+0000 7fb50a985640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.935+0000 7fb50a985640 10 monclient: monmap: 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.935+0000 7fb50a985640 1 librados: starting msgr at 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.935+0000 7fb50a985640 1 librados: starting objecter 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.935+0000 7fb50a985640 1 Processor -- start 2026-02-17T05:56:48.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.935+0000 7fb50a985640 1 -- start start 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.935+0000 7fb50a985640 1 librados: setting wanted keys 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.935+0000 7fb50a985640 1 librados: calling monclient init 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.935+0000 7fb50a985640 10 monclient: init 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb504074dc0 0x7fb5041f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 10 monclient(hunting): picked mon.noname-b con 0x7fb504074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 0x7fb50414ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 10 monclient(hunting): picked mon.noname-c con 0x7fb504155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5041561e0 0x7fb50414c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 10 monclient(hunting): picked mon.noname-a con 0x7fb5041561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5040c9360 con 0x7fb5041561e0 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fb5040ca1e0 con 0x7fb504074dc0 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb5040c8c20 con 0x7fb504155e00 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb50a985640 10 monclient(hunting): authenticate will time out at 1061.505055s 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb5037fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 0x7fb50414ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb5037fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 0x7fb50414ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57320/0 (socket says 10.20.193.16:57320) 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb5037fe640 1 -- 10.20.193.16:0/2234917072 learned_addr learned my addr 10.20.193.16:0/2234917072 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb5037fe640 10 monclient(hunting): get_auth_request con 0x7fb504155e00 auth_method 0 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb5037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb5037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb5037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb5037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb5037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.936+0000 7fb5037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7f988e4e8640 10 monclient: get_monmap_and_config 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7f988e4e8640 10 monclient: build_initial_monmap 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7f988e4e8640 10 monclient: monmap: 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:48.939100+0000 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:48.939100+0000 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:48.939 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7fb5037fe640 10 monclient(hunting): handle_auth_done global_id 4589 payload 824 2026-02-17T05:56:48.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7fb5037fe640 1 -- 10.20.193.16:0/2234917072 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb504074dc0 msgr2=0x7fb5041f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7fb5037fe640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb504074dc0 0x7fb5041f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7fb5037fe640 1 -- 10.20.193.16:0/2234917072 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5041561e0 msgr2=0x7fb50414c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:48.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7fb5037fe640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5041561e0 0x7fb50414c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:48.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7fb5037fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7fb5037fe640 1 monclient: found mon.noname-c 2026-02-17T05:56:48.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7fb5037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7fb5037fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7fb5037fe640 1 -- 10.20.193.16:0/2234917072 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5041f4870 con 0x7fb504155e00 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7f988e4e8640 1 Processor -- start 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7f988e4e8640 1 -- start start 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.938+0000 7f988e4e8640 10 monclient: init 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 10 monclient: _add_conns ranks=[0,5,2,3,1,4] 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9888155e00 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f98881561e0 0x7f98880ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 10 monclient(hunting): picked mon.noname-c con 0x7f98881561e0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9888074dc0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98880c96d0 con 0x7f9888155e00 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9888074860 con 0x7f9888074dc0 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f98880c9940 con 0x7f98881561e0 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:48.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7f988e4e8640 10 monclient(hunting): authenticate will time out at 1061.508136s 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7fb5037fe640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 0x7fb50414ba00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400b500 tx=0x7fb4f400b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7fb500ff9640 10 client.?.objecter ms_handle_connect 0x7fb504155e00 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.939+0000 7fb500ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb500ff9640 1 -- 10.20.193.16:0/2234917072 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4004480 con 0x7fb504155e00 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb500ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb500ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb500ff9640 10 monclient: dump: 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.942 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb500ff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb500ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb500ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 5 monclient: authenticate success, global_id 4589 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 10 monclient: _renew_subs 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 1 -- 10.20.193.16:0/2234917072 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50414ce90 con 0x7fb504155e00 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7f98877fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f98881561e0 0x7f98880ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 10 client.4589.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 10 monclient: _renew_subs 2026-02-17T05:56:48.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 1 -- 10.20.193.16:0/2234917072 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041432d0 con 0x7fb504155e00 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 1 librados: init done 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.940+0000 7fb50a985640 1 -- 10.20.193.16:0/2234917072 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5040c8fe0 con 0x7fb504155e00 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7fb500ff9640 1 -- 10.20.193.16:0/2234917072 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f4014070 con 0x7fb504155e00 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7fb500ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7fb500ff9640 1 -- 10.20.193.16:0/2234917072 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4002900 con 0x7fb504155e00 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7fb500ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7fb500ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7fb5027fc640 4 set_mon_vals no callback set 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7fb500ff9640 10 monclient: dump: 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:48.944 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7f98877fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f98881561e0 0x7f98880ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57326/0 (socket says 10.20.193.16:57326) 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7f98877fe640 1 -- 10.20.193.16:0/1942190051 learned_addr learned my addr 10.20.193.16:0/1942190051 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7f98877fe640 10 monclient(hunting): get_auth_request con 0x7f98881561e0 auth_method 0 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7f98877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7f98877fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7f98877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7f98877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7f98877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.941+0000 7f98877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.942+0000 7f98877fe640 10 monclient(hunting): handle_auth_done global_id 4592 payload 274 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.942+0000 7f98877fe640 1 -- 10.20.193.16:0/1942190051 >> v1:10.20.193.148:6789/0 conn(0x7f9888074dc0 legacy=0x7f988815f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.942+0000 7f98877fe640 1 -- 10.20.193.16:0/1942190051 >> v1:10.20.193.16:6789/0 conn(0x7f9888155e00 legacy=0x7f9888074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.942+0000 7f98877fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.942+0000 7f98877fe640 1 monclient: found mon.noname-c 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.942+0000 7f98877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.942+0000 7f98877fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.942+0000 7f98877fe640 1 -- 10.20.193.16:0/1942190051 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98880c9c20 con 0x7f98881561e0 2026-02-17T05:56:48.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.972+0000 7f9c8df85640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:56:48.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.972+0000 7f9c8df85640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:56:48.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.972+0000 7f9c8df85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:48.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.972+0000 7f9c8df85640 1 -- 10.20.193.16:0/3648377350 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9c88146100 con 0x7f9c8813cfb0 2026-02-17T05:56:48.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.980+0000 7fb5c8975640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-17T05:56:48.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.980+0000 7fb5c8975640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-17T05:56:48.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.980+0000 7fb5c8975640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:48.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.980+0000 7fb5c8975640 1 -- 10.20.193.16:0/2904989 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7fb5c00cc520 con 0x7fb5b0003bf0 2026-02-17T05:56:48.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.984+0000 7eff5b7fe640 1 -- 10.20.193.16:0/1259153303 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7eff70019460 con 0x7eff7c0700f0 2026-02-17T05:56:48.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.984+0000 7eff5b7fe640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7eff48079170 0x7eff4807b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:48.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.984+0000 7eff7a7fc640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7eff48079170 0x7eff4807b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:48.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.984+0000 7eff7a7fc640 10 monclient: get_auth_request con 0x7eff48079170 auth_method 0 2026-02-17T05:56:48.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.985+0000 7eff5b7fe640 10 client.4540.objecter ms_handle_connect 0x7eff48079170 2026-02-17T05:56:48.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.985+0000 7eff7a7fc640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7eff48079170 0x7eff4807b630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff7c142790 tx=0x7eff64006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:48.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.985+0000 7eff5b7fe640 1 -- 10.20.193.16:0/1259153303 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7eff70097b30 con 0x7eff7c0700f0 2026-02-17T05:56:48.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.985+0000 7eff5b7fe640 10 client.4540.objecter ms_dispatch2 0x7eff7c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:48.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.985+0000 7eff5b7fe640 3 client.4540.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:48.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.985+0000 7eff5b7fe640 3 client.4540.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:48.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.985+0000 7eff5b7fe640 20 client.4540.objecter dump_active .. 0 homeless 2026-02-17T05:56:48.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.992+0000 7eff5b7fe640 1 -- 10.20.193.16:0/1259153303 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7eff700626a0 con 0x7eff7c0700f0 2026-02-17T05:56:48.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.992+0000 7eff5b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:48.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:48.992+0000 7eff5b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.000 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-17T05:56:49.000 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-17T05:56:49.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.013+0000 7f54bffff640 1 -- 10.20.193.16:0/1229688593 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f54d0002ac0 con 0x7f54e40700f0 2026-02-17T05:56:49.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.013+0000 7f54bffff640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f54b40791c0 0x7f54b407b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.013+0000 7f54e2ffd640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f54b40791c0 0x7f54b407b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.013+0000 7f54e2ffd640 10 monclient: get_auth_request con 0x7f54b40791c0 auth_method 0 2026-02-17T05:56:49.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.013+0000 7f54bffff640 10 client.4461.objecter ms_handle_connect 0x7f54b40791c0 2026-02-17T05:56:49.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.013+0000 7f54e2ffd640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f54b40791c0 0x7f54b407b680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54e415aa70 tx=0x7f54d4006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.023+0000 7f54bffff640 1 -- 10.20.193.16:0/1229688593 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f54d005e1d0 con 0x7f54e40700f0 2026-02-17T05:56:49.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.023+0000 7f54bffff640 10 client.4461.objecter ms_dispatch2 0x7f54e4000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.023+0000 7f54bffff640 3 client.4461.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.023+0000 7f54bffff640 3 client.4461.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.023+0000 7f54bffff640 20 client.4461.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.023+0000 7f54bffff640 1 -- 10.20.193.16:0/1229688593 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f54d001d020 con 0x7f54e40700f0 2026-02-17T05:56:49.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.023+0000 7f54bffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.023+0000 7f54bffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.026+0000 7f9c63fff640 1 -- 10.20.193.16:0/3648377350 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9c780625e0 con 0x7f9c8813cfb0 2026-02-17T05:56:49.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.026+0000 7f9c63fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:56:49.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.026+0000 7f9c63fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.026 INFO:teuthology.orchestra.run.trial016.stdout:115964116999 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 10 librados: watch_flush enter 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 10 librados: watch_flush exit 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 20 client.4455.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 10 client.4455.objecter successfully canceled tick 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 -- 10.20.193.16:0/3648377350 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c58079170 msgr2=0x7f9c5807b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c58079170 0x7f9c5807b630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c740036c0 tx=0x7f9c74002d00 comp rx=0 tx=0).stop 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 10 monclient: shutdown 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 -- 10.20.193.16:0/3648377350 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c8813cfb0 msgr2=0x7f9c881f00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c8813cfb0 0x7f9c881f00a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800eb00 tx=0x7f9c780056b0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 -- 10.20.193.16:0/3648377350 shutdown_connections 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9c58079170 0x7f9c5807b630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9c881542c0 0x7f9c8806d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9c8813cfb0 0x7f9c881f00a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 --2- 10.20.193.16:0/3648377350 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9c6c009ab0 0x7f9c8813c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 -- 10.20.193.16:0/3648377350 >> 10.20.193.16:0/3648377350 conn(0x7f9c880cd770 msgr2=0x7f9c8814e910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 -- 10.20.193.16:0/3648377350 shutdown_connections 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 -- 10.20.193.16:0/3648377350 wait complete. 2026-02-17T05:56:49.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.028+0000 7f9c8df85640 1 librados: shutdown 2026-02-17T05:56:49.031 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2026-02-17T05:56:49.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964117001 got 115964116999 for osd.5 2026-02-17T05:56:49.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.040+0000 7f98877fe640 1 --2- 10.20.193.16:0/1942190051 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 0x7f98880ca700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9878009970 tx=0x7f987802eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac6d5a5d4cdc73f0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.051+0000 7fb500ff9640 1 -- 10.20.193.16:0/2234917072 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fb4f4019460 con 0x7fb504155e00 2026-02-17T05:56:49.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.051+0000 7fb500ff9640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4d4079140 0x7fb4d407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.051+0000 7fb503fff640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4d4079140 0x7fb4d407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.051+0000 7fb503fff640 10 monclient: get_auth_request con 0x7fb4d4079140 auth_method 0 2026-02-17T05:56:49.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.051+0000 7fb503fff640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4d4079140 0x7fb4d407b600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb5040693f0 tx=0x7fb4f0031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.051+0000 7fb500ff9640 10 client.4589.objecter ms_handle_connect 0x7fb4d4079140 2026-02-17T05:56:49.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.053+0000 7fb500ff9640 1 -- 10.20.193.16:0/2234917072 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fb4f405e020 con 0x7fb504155e00 2026-02-17T05:56:49.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.053+0000 7fb500ff9640 10 client.4589.objecter ms_dispatch2 0x7fb504000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.053+0000 7fb500ff9640 3 client.4589.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.053+0000 7fb500ff9640 3 client.4589.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.053+0000 7fb500ff9640 20 client.4589.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.059+0000 7fb5aaffd640 1 -- 10.20.193.16:0/2904989 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fb5b80877f0 con 0x7fb5b0003bf0 2026-02-17T05:56:49.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.059+0000 7fb5aaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-17T05:56:49.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.059+0000 7fb5aaffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.060 INFO:teuthology.orchestra.run.trial016.stdout:94489280517 2026-02-17T05:56:49.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.060+0000 7fb500ff9640 1 -- 10.20.193.16:0/2234917072 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb4f40626e0 con 0x7fb504155e00 2026-02-17T05:56:49.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.060+0000 7fb500ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.060+0000 7fb500ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.061+0000 7f9886ffd640 1 -- 10.20.193.16:0/1942190051 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987803c070 con 0x7f98881561e0 2026-02-17T05:56:49.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.061+0000 7f9886ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.061+0000 7f9886ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:49.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.061+0000 7f9886ffd640 10 monclient: dump: 2026-02-17T05:56:49.063 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.063 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.063 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.063 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.063 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.063 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.064 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.064 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.064 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.064 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.061+0000 7f9886ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.061+0000 7f9886ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.061+0000 7f9886ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f9886ffd640 1 -- 10.20.193.16:0/1942190051 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987802fb60 con 0x7f98881561e0 2026-02-17T05:56:49.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 5 monclient: authenticate success, global_id 4592 2026-02-17T05:56:49.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f9886ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f9886ffd640 1 -- 10.20.193.16:0/1942190051 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878037440 con 0x7f98881561e0 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f9886ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 4 set_mon_vals no callback set 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f9886ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f9886ffd640 10 monclient: dump: 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.065 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.066 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.066 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.066 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.066 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.066 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 10 librados: watch_flush enter 2026-02-17T05:56:49.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 10 librados: watch_flush exit 2026-02-17T05:56:49.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 20 client.4583.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 10 client.4583.objecter successfully canceled tick 2026-02-17T05:56:49.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 -- 10.20.193.16:0/2904989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb590079170 msgr2=0x7fb59007b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb590079170 0x7fb59007b630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac002eb0 tx=0x7fb5ac002da0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 10 monclient: shutdown 2026-02-17T05:56:49.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 -- 10.20.193.16:0/2904989 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 msgr2=0x7fb5c0144350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5c0144350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb5b803a040 tx=0x7fb5b80048d0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 -- 10.20.193.16:0/2904989 shutdown_connections 2026-02-17T05:56:49.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb590079170 0x7fb59007b630 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5c01f3c30 0x7fb5c0149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5c0149da0 0x7fb5c00ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 --2- 10.20.193.16:0/2904989 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5b0003bf0 0x7fb5c0144350 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 -- 10.20.193.16:0/2904989 >> 10.20.193.16:0/2904989 conn(0x7fb5c00ce6c0 msgr2=0x7fb5c0073650 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 -- 10.20.193.16:0/2904989 shutdown_connections 2026-02-17T05:56:49.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 -- 10.20.193.16:0/2904989 wait complete. 2026-02-17T05:56:49.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7fb5c8975640 1 librados: shutdown 2026-02-17T05:56:49.071 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-17T05:56:49.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 10 monclient: shutdown 2026-02-17T05:56:49.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 1 -- 10.20.193.16:0/1942190051 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 msgr2=0x7f98880ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 1 --2- 10.20.193.16:0/1942190051 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 0x7f98880ca700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9878009970 tx=0x7f987802eef0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 1 -- 10.20.193.16:0/1942190051 shutdown_connections 2026-02-17T05:56:49.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 1 --2- 10.20.193.16:0/1942190051 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 0x7f98880ca700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 1 -- 10.20.193.16:0/1942190051 >> 10.20.193.16:0/1942190051 conn(0x7f98880ce6c0 msgr2=0x7f98880cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 1 -- 10.20.193.16:0/1942190051 shutdown_connections 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.062+0000 7f988e4e8640 1 -- 10.20.193.16:0/1942190051 wait complete. 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.063+0000 7f988e4e8640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.063+0000 7f988e4e8640 10 monclient: monmap: 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.077 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.078 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.078 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.063+0000 7f988e4e8640 1 librados: starting msgr at 2026-02-17T05:56:49.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.063+0000 7f988e4e8640 1 librados: starting objecter 2026-02-17T05:56:49.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.063+0000 7f988e4e8640 1 Processor -- start 2026-02-17T05:56:49.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.063+0000 7f988e4e8640 1 -- start start 2026-02-17T05:56:49.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.063+0000 7f988e4e8640 1 librados: setting wanted keys 2026-02-17T05:56:49.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.063+0000 7f988e4e8640 1 librados: calling monclient init 2026-02-17T05:56:49.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.063+0000 7f988e4e8640 10 monclient: init 2026-02-17T05:56:49.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:49.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9888074dc0 0x7f988814e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 10 monclient(hunting): picked mon.noname-b con 0x7f9888074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9888155e00 0x7f9888153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 10 monclient(hunting): picked mon.noname-a con 0x7f9888155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 0x7f98880c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 10 monclient(hunting): picked mon.noname-c con 0x7f98881561e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98880c9360 con 0x7f9888155e00 2026-02-17T05:56:49.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f98880ca1e0 con 0x7f9888074dc0 2026-02-17T05:56:49.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f988815fa10 con 0x7f98881561e0 2026-02-17T05:56:49.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988e4e8640 10 monclient(hunting): authenticate will time out at 1061.633370s 2026-02-17T05:56:49.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988ca5e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 0x7f98880c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988ca5e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 0x7f98880c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57330/0 (socket says 10.20.193.16:57330) 2026-02-17T05:56:49.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988ca5e640 1 -- 10.20.193.16:0/4258518166 learned_addr learned my addr 10.20.193.16:0/4258518166 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988ca5e640 10 monclient(hunting): get_auth_request con 0x7f98881561e0 auth_method 0 2026-02-17T05:56:49.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988ca5e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988ca5e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988ca5e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f9887fff640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9888074dc0 0x7f988814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988ca5e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.064+0000 7f988ca5e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.065+0000 7f988ca5e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.065+0000 7f9887fff640 10 monclient(hunting): get_auth_request con 0x7f9888074dc0 auth_method 0 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.065+0000 7f9887fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.065+0000 7f9887fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.065+0000 7f9887fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.065+0000 7f9887fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.065+0000 7f9887fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.065+0000 7f9887fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f98877fe640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9888155e00 0x7f9888153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f988ca5e640 10 monclient(hunting): handle_auth_done global_id 4595 payload 824 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f988ca5e640 1 -- 10.20.193.16:0/4258518166 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9888074dc0 msgr2=0x7f988814e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f988ca5e640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9888074dc0 0x7f988814e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f988ca5e640 1 -- 10.20.193.16:0/4258518166 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9888155e00 msgr2=0x7f9888153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f988ca5e640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9888155e00 0x7f9888153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f988ca5e640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f988ca5e640 1 monclient: found mon.noname-c 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f988ca5e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f988ca5e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.066+0000 7f988ca5e640 1 -- 10.20.193.16:0/4258518166 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988814e650 con 0x7f98881561e0 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.067+0000 7f988ca5e640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 0x7f98880c7f10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f987c00b990 tx=0x7f987c00be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.067+0000 7f9884ff9640 10 client.?.objecter ms_handle_connect 0x7f98881561e0 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.067+0000 7f9884ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 1 -- 10.20.193.16:0/4258518166 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c006e60 con 0x7f98881561e0 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 10 monclient: dump: 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.083 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f988e4e8640 5 monclient: authenticate success, global_id 4595 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 1 -- 10.20.193.16:0/4258518166 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987c010070 con 0x7f98881561e0 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 1 -- 10.20.193.16:0/4258518166 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c00c9d0 con 0x7f98881561e0 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f9884ff9640 10 monclient: dump: 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f988e4e8640 10 monclient: _renew_subs 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f98867fc640 4 set_mon_vals no callback set 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f988e4e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f988e4e8640 1 -- 10.20.193.16:0/4258518166 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98881f3bf0 con 0x7f98881561e0 2026-02-17T05:56:49.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f988e4e8640 10 client.4595.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:49.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f988e4e8640 10 monclient: _renew_subs 2026-02-17T05:56:49.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f988e4e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f988e4e8640 1 -- 10.20.193.16:0/4258518166 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98881f41b0 con 0x7f98881561e0 2026-02-17T05:56:49.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7f988e4e8640 1 librados: init done 2026-02-17T05:56:49.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7fc0b3553640 10 monclient: get_monmap_and_config 2026-02-17T05:56:49.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7fc0b3553640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.069+0000 7fc0b3553640 10 monclient: monmap: 2026-02-17T05:56:49.086 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.086 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:49.070682+0000 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:49.070682+0000 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:49.087 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7f671c8a9640 10 monclient: get_monmap_and_config 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7f671c8a9640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7f671c8a9640 10 monclient: monmap: 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:49.071037+0000 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:49.071037+0000 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:49.088 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 1 Processor -- start 2026-02-17T05:56:49.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7f988e4e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7f988e4e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7f988e4e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 1 -- start start 2026-02-17T05:56:49.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 10 monclient: init 2026-02-17T05:56:49.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7f988e4e8640 1 -- 10.20.193.16:0/4258518166 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98880ca700 con 0x7f98881561e0 2026-02-17T05:56:49.090 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-17T05:56:49.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7f671c8a9640 1 Processor -- start 2026-02-17T05:56:49.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7f671c8a9640 1 -- start start 2026-02-17T05:56:49.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7f671c8a9640 10 monclient: init 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 10 monclient: _add_conns ranks=[4,1,3,2,0,5] 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fc0ac155e00 0x7fc0ac0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 10 monclient(hunting): picked mon.noname-b con 0x7fc0ac155e00 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc0ac1561e0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fc0ac074dc0 0x7fc0ac0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 10 monclient(hunting): picked mon.noname-a con 0x7fc0ac074dc0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ac0c96d0 con 0x7fc0ac1561e0 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ac074860 con 0x7fc0ac074dc0 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ac0c9940 con 0x7fc0ac155e00 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.070+0000 7fc0b3553640 10 monclient(hunting): authenticate will time out at 1061.639848s 2026-02-17T05:56:49.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7fc0b12c8640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fc0ac155e00 0x7fc0ac0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 10 monclient: _add_conns ranks=[5,4,0,3,2,1] 2026-02-17T05:56:49.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6714155e00 0x7f67140ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280519 got 94489280517 for osd.15 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 10 monclient(hunting): picked mon.noname-c con 0x7f6714155e00 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f67141561e0 0x7f67140cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 10 monclient(hunting): picked mon.noname-b con 0x7f67141561e0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6714074dc0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67140c96d0 con 0x7f6714074dc0 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f6714074860 con 0x7f67141561e0 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f67140c9940 con 0x7f6714155e00 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7f671c8a9640 10 monclient(hunting): authenticate will time out at 1061.640354s 2026-02-17T05:56:49.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7fc0b12c8640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fc0ac155e00 0x7fc0ac0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50612/0 (socket says 10.20.193.16:50612) 2026-02-17T05:56:49.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7fc0b12c8640 1 -- 10.20.193.16:0/1573146651 learned_addr learned my addr 10.20.193.16:0/1573146651 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7fc0b12c8640 10 monclient(hunting): get_auth_request con 0x7fc0ac155e00 auth_method 0 2026-02-17T05:56:49.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7fc0b12c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7fc0b12c8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.071+0000 7fc0b12c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 10 monclient(hunting): handle_auth_done global_id 4546 payload 274 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 1 -- 10.20.193.16:0/1573146651 >> v2:10.20.193.16:3300/0 conn(0x7fc0ac074dc0 msgr2=0x7fc0ac0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 1 --2- 10.20.193.16:0/1573146651 >> v2:10.20.193.16:3300/0 conn(0x7fc0ac074dc0 0x7fc0ac0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 1 -- 10.20.193.16:0/1573146651 >> v1:10.20.193.128:6789/0 conn(0x7fc0ac1561e0 legacy=0x7fc0ac074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 1 monclient: found mon.noname-b 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.072+0000 7fc0b12c8640 1 -- 10.20.193.16:0/1573146651 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac0c9c20 con 0x7fc0ac155e00 2026-02-17T05:56:49.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.073+0000 7f6719e1d640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f67141561e0 0x7f67140cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.073+0000 7f6719e1d640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f67141561e0 0x7f67140cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50622/0 (socket says 10.20.193.16:50622) 2026-02-17T05:56:49.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.073+0000 7f6719e1d640 1 -- 10.20.193.16:0/569204852 learned_addr learned my addr 10.20.193.16:0/569204852 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.073+0000 7f6719e1d640 10 monclient(hunting): get_auth_request con 0x7f67141561e0 auth_method 0 2026-02-17T05:56:49.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.073+0000 7f6719e1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.073+0000 7f6719e1d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.073+0000 7f6719e1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.074+0000 7f6719e1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.074+0000 7f6719e1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.074+0000 7f6719e1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.074+0000 7fc0b12c8640 1 --2- 10.20.193.16:0/1573146651 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac155e00 0x7fc0ac0ca700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac06ce20 tx=0x7fc09c02eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9198ea7f565ef35 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.076+0000 7f6719e1d640 10 monclient(hunting): handle_auth_done global_id 4549 payload 274 2026-02-17T05:56:49.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.076+0000 7f6719e1d640 1 -- 10.20.193.16:0/569204852 >> v2:10.20.193.148:3300/0 conn(0x7f6714155e00 msgr2=0x7f67140ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.076+0000 7f6719e1d640 1 --2- 10.20.193.16:0/569204852 >> v2:10.20.193.148:3300/0 conn(0x7f6714155e00 0x7f67140ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.076+0000 7f6719e1d640 1 -- 10.20.193.16:0/569204852 >> v1:10.20.193.16:6789/0 conn(0x7f6714074dc0 legacy=0x7f6714074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.076+0000 7f6719e1d640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.076+0000 7f6719e1d640 1 monclient: found mon.noname-b 2026-02-17T05:56:49.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.076+0000 7f6719e1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.076+0000 7f6719e1d640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.076+0000 7f6719e1d640 1 -- 10.20.193.16:0/569204852 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67140c9c20 con 0x7f67141561e0 2026-02-17T05:56:49.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.077+0000 7f6719e1d640 1 --2- 10.20.193.16:0/569204852 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67141561e0 0x7f67140cc1a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f671406ce20 tx=0x7f670002eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7766209bc637c8d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.099 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 1 -- 10.20.193.16:0/1573146651 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c03c070 con 0x7fc0ac155e00 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 10 monclient: dump: 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.100 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 1 -- 10.20.193.16:0/1573146651 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09c02fbe0 con 0x7fc0ac155e00 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 5 monclient: authenticate success, global_id 4546 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 1 -- 10.20.193.16:0/1573146651 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c037440 con 0x7fc0ac155e00 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 4 set_mon_vals no callback set 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc09bfff640 10 monclient: dump: 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.101 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 10 monclient: shutdown 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 1 -- 10.20.193.16:0/1573146651 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac155e00 msgr2=0x7fc0ac0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 1 --2- 10.20.193.16:0/1573146651 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac155e00 0x7fc0ac0ca700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac06ce20 tx=0x7fc09c02eed0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 1 -- 10.20.193.16:0/1573146651 shutdown_connections 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 1 --2- 10.20.193.16:0/1573146651 >> v2:10.20.193.16:3300/0 conn(0x7fc0ac074dc0 0x7fc0ac0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 1 --2- 10.20.193.16:0/1573146651 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac155e00 0x7fc0ac0ca700 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 1 -- 10.20.193.16:0/1573146651 >> 10.20.193.16:0/1573146651 conn(0x7fc0ac0ce6c0 msgr2=0x7fc0ac0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 1 -- 10.20.193.16:0/1573146651 shutdown_connections 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.078+0000 7fc0b3553640 1 -- 10.20.193.16:0/1573146651 wait complete. 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient: monmap: 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.102 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 librados: starting msgr at 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 librados: starting objecter 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 Processor -- start 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 -- start start 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 librados: setting wanted keys 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 librados: calling monclient init 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient: init 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac074dc0 0x7fc0ac1f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient(hunting): picked mon.noname-b con 0x7fc0ac074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0ac155e00 0x7fc0ac0cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient(hunting): picked mon.noname-c con 0x7fc0ac155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc0ac1561e0 0x7fc0ac151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient(hunting): picked mon.noname-a con 0x7fc0ac1561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0c9360 con 0x7fc0ac1561e0 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0ca1e0 con 0x7fc0ac074dc0 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0c8c20 con 0x7fc0ac155e00 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.079+0000 7fc0b3553640 10 monclient(hunting): authenticate will time out at 1061.648940s 2026-02-17T05:56:49.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 1 -- 10.20.193.16:0/569204852 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670003c070 con 0x7f67141561e0 2026-02-17T05:56:49.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:49.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 10 monclient: dump: 2026-02-17T05:56:49.104 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.104 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.104 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 1 -- 10.20.193.16:0/569204852 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670002fb40 con 0x7f67141561e0 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 1 -- 10.20.193.16:0/569204852 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700037440 con 0x7f67141561e0 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 5 monclient: authenticate success, global_id 4549 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 4 set_mon_vals no callback set 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671961c640 10 monclient: dump: 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.105 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 10 monclient: shutdown 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 1 -- 10.20.193.16:0/569204852 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67141561e0 msgr2=0x7f67140cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 1 --2- 10.20.193.16:0/569204852 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67141561e0 0x7f67140cc1a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f671406ce20 tx=0x7f670002eed0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 1 -- 10.20.193.16:0/569204852 shutdown_connections 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 1 --2- 10.20.193.16:0/569204852 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67141561e0 0x7f67140cc1a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 1 --2- 10.20.193.16:0/569204852 >> v2:10.20.193.148:3300/0 conn(0x7f6714155e00 0x7f67140ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 1 -- 10.20.193.16:0/569204852 >> 10.20.193.16:0/569204852 conn(0x7f67140ce6c0 msgr2=0x7f67140cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f9884ff9640 1 -- 10.20.193.16:0/4258518166 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f987c00cb70 con 0x7f98881561e0 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 1 -- 10.20.193.16:0/569204852 shutdown_connections 2026-02-17T05:56:49.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 1 -- 10.20.193.16:0/569204852 wait complete. 2026-02-17T05:56:49.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7eff81488640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:56:49.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7eff81488640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:56:49.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7eff81488640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7eff81488640 1 -- 10.20.193.16:0/1259153303 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7eff7c1573f0 con 0x7eff7c0700f0 2026-02-17T05:56:49.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7fc0b12c8640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac074dc0 0x7fc0ac1f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f9884ff9640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9858079140 0x7f985807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f671c8a9640 10 monclient: monmap: 2026-02-17T05:56:49.107 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.107 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f9887fff640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9858079140 0x7f985807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7f9887fff640 10 monclient: get_auth_request con 0x7f9858079140 auth_method 0 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7fc0b12c8640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac074dc0 0x7fc0ac1f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50624/0 (socket says 10.20.193.16:50624) 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7fc0b12c8640 1 -- 10.20.193.16:0/3816580839 learned_addr learned my addr 10.20.193.16:0/3816580839 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7fc0b12c8640 10 monclient(hunting): get_auth_request con 0x7fc0ac074dc0 auth_method 0 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7fc0b12c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7fc0b12c8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.080+0000 7fc0b12c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 librados: starting msgr at 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 librados: starting objecter 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 Processor -- start 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f9884ff9640 10 client.4595.objecter ms_handle_connect 0x7f9858079140 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f9887fff640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9858079140 0x7f985807b600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f988814e0a0 tx=0x7f98740091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 -- start start 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 librados: setting wanted keys 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 librados: calling monclient init 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient: init 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7fc0b12c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7fc0b12c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7fc0b12c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6714074dc0 0x7f67141420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient(hunting): picked mon.noname-c con 0x7f6714074dc0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6714155e00 0x7f67141f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient(hunting): picked mon.noname-b con 0x7f6714155e00 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67141561e0 0x7f67140c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient(hunting): picked mon.noname-a con 0x7f67141561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67140c9360 con 0x7f67141561e0 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f67140ca1e0 con 0x7f6714155e00 2026-02-17T05:56:49.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67140c8c20 con 0x7f6714074dc0 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.081+0000 7f671c8a9640 10 monclient(hunting): authenticate will time out at 1061.650535s 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7f9884ff9640 1 -- 10.20.193.16:0/4258518166 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f987c05daf0 con 0x7f98881561e0 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7f9884ff9640 10 client.4595.objecter ms_dispatch2 0x7f9888000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7f9884ff9640 3 client.4595.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7f9884ff9640 3 client.4595.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7f9884ff9640 20 client.4595.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7fc0b12c8640 10 monclient(hunting): handle_auth_done global_id 4552 payload 824 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7fc0b12c8640 1 -- 10.20.193.16:0/3816580839 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0ac155e00 msgr2=0x7fc0ac0cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7fc0b12c8640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0ac155e00 0x7fc0ac0cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7fc0b12c8640 1 -- 10.20.193.16:0/3816580839 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc0ac1561e0 msgr2=0x7fc0ac151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7fc0b12c8640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc0ac1561e0 0x7fc0ac151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7fc0b12c8640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7fc0b12c8640 1 monclient: found mon.noname-b 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7fc0b12c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7fc0b12c8640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.082+0000 7fc0b12c8640 1 -- 10.20.193.16:0/3816580839 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac1f4870 con 0x7fc0ac074dc0 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.085+0000 7f6719e1d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6714155e00 0x7f67141f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.085+0000 7eff5b7fe640 1 -- 10.20.193.16:0/1259153303 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7eff70061fa0 con 0x7eff7c0700f0 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.085+0000 7eff5b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.085+0000 7eff5b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stdout:90194313222 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.085+0000 7fc0b12c8640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac074dc0 0x7fc0ac1f4330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009a70 tx=0x7fc09c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.085+0000 7fc099ffb640 10 client.?.objecter ms_handle_connect 0x7fc0ac074dc0 2026-02-17T05:56:49.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.085+0000 7fc099ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:49.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.086+0000 7f6719e1d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6714155e00 0x7f67141f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50630/0 (socket says 10.20.193.16:50630) 2026-02-17T05:56:49.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.086+0000 7f6719e1d640 1 -- 10.20.193.16:0/2317619249 learned_addr learned my addr 10.20.193.16:0/2317619249 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.086+0000 7f6719e1d640 10 monclient(hunting): get_auth_request con 0x7f6714155e00 auth_method 0 2026-02-17T05:56:49.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.086+0000 7f6719e1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.086+0000 7f6719e1d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.086+0000 7f6719e1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.086+0000 7f6719e1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.086+0000 7f6719e1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.086+0000 7f6719e1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 10 librados: watch_flush enter 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 10 librados: watch_flush exit 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 20 client.4540.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 10 client.4540.objecter successfully canceled tick 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 -- 10.20.193.16:0/1259153303 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7eff48079170 msgr2=0x7eff4807b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7eff48079170 0x7eff4807b630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff7c142790 tx=0x7eff64006ca0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 10 monclient: shutdown 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 -- 10.20.193.16:0/1259153303 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c0700f0 msgr2=0x7eff7c1e7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c0700f0 0x7eff7c1e7920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff7000b500 tx=0x7eff7000b9d0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 -- 10.20.193.16:0/1259153303 shutdown_connections 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7eff48079170 0x7eff4807b630 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7eff7c1536f0 0x7eff7c1f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7eff7c0c7fe0 0x7eff7c1f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 --2- 10.20.193.16:0/1259153303 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7eff7c0700f0 0x7eff7c1e7920 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 -- 10.20.193.16:0/1259153303 >> 10.20.193.16:0/1259153303 conn(0x7eff7c0cd770 msgr2=0x7eff7c151180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.087+0000 7eff81488640 1 -- 10.20.193.16:0/1259153303 shutdown_connections 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7eff81488640 1 -- 10.20.193.16:0/1259153303 wait complete. 2026-02-17T05:56:49.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7eff81488640 1 librados: shutdown 2026-02-17T05:56:49.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7f6719e1d640 10 monclient(hunting): handle_auth_done global_id 4555 payload 824 2026-02-17T05:56:49.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7f6719e1d640 1 -- 10.20.193.16:0/2317619249 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6714074dc0 msgr2=0x7f67141420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7f6719e1d640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6714074dc0 0x7f67141420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7f6719e1d640 1 -- 10.20.193.16:0/2317619249 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67141561e0 msgr2=0x7f67140c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7f6719e1d640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67141561e0 0x7f67140c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7f6719e1d640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7f6719e1d640 1 monclient: found mon.noname-b 2026-02-17T05:56:49.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7f6719e1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7f6719e1d640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.088+0000 7f6719e1d640 1 -- 10.20.193.16:0/2317619249 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67141f4b60 con 0x7f6714155e00 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 1 -- 10.20.193.16:0/3816580839 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c047070 con 0x7fc0ac074dc0 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 10 monclient: dump: 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 1 -- 10.20.193.16:0/3816580839 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09c042630 con 0x7fc0ac074dc0 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc09b7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc0b3553640 5 monclient: authenticate success, global_id 4552 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc0b3553640 10 monclient: _renew_subs 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc0b3553640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc0b3553640 1 -- 10.20.193.16:0/3816580839 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac0cc5d0 con 0x7fc0ac074dc0 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc0b3553640 10 client.4552.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc0b3553640 10 monclient: _renew_subs 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc0b3553640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc0b3553640 1 -- 10.20.193.16:0/3816580839 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac0ccb90 con 0x7fc0ac074dc0 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc0b3553640 1 librados: init done 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 1 -- 10.20.193.16:0/3816580839 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c03c040 con 0x7fc0ac074dc0 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.089+0000 7fc099ffb640 10 monclient: dump: 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.090+0000 7fc0b3553640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.090+0000 7fc0b3553640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.090+0000 7fc0b3553640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.090+0000 7fc0b3553640 1 -- 10.20.193.16:0/3816580839 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ac0c8fa0 con 0x7fc0ac074dc0 2026-02-17T05:56:49.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.090+0000 7f6719e1d640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6714155e00 0x7f67141f4070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f670002f3e0 tx=0x7f67000047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.090+0000 7f66feffd640 10 client.?.objecter ms_handle_connect 0x7f6714155e00 2026-02-17T05:56:49.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.090+0000 7f66feffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:49.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313223 got 90194313222 for osd.2 2026-02-17T05:56:49.119 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.103+0000 7f01947ca640 10 monclient: get_monmap_and_config 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.103+0000 7f01947ca640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.103+0000 7f01947ca640 10 monclient: monmap: 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:49.104093+0000 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:49.104093+0000 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.103+0000 7f01947ca640 1 Processor -- start 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.103+0000 7f01947ca640 1 -- start start 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.103+0000 7f01947ca640 10 monclient: init 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 10 monclient: _add_conns ranks=[1,3,5,4,0,2] 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f018c1538d0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:49.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f018c15b530 0x7f018c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 10 monclient(hunting): picked mon.noname-a con 0x7f018c15b530 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f018c0c7d40 0x7f018c0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 10 monclient(hunting): picked mon.noname-c con 0x7f018c0c7d40 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f018c0c58c0 con 0x7f018c1538d0 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f018c0c5ba0 con 0x7f018c15b530 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f018c0c5e80 con 0x7f018c0c7d40 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.104+0000 7f01947ca640 10 monclient(hunting): authenticate will time out at 1061.673216s 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.107+0000 7f0191d3e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f018c15b530 0x7f018c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.107+0000 7f0191d3e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f018c15b530 0x7f018c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:60118/0 (socket says 10.20.193.16:60118) 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.107+0000 7f0191d3e640 1 -- 10.20.193.16:0/892170562 learned_addr learned my addr 10.20.193.16:0/892170562 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.107+0000 7f0191d3e640 10 monclient(hunting): get_auth_request con 0x7f018c15b530 auth_method 0 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.107+0000 7f0191d3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.107+0000 7f0191d3e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.107+0000 7f0191d3e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.107+0000 7f0191d3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.107+0000 7f0191d3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.107+0000 7f0191d3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.108+0000 7f0191d3e640 10 monclient(hunting): handle_auth_done global_id 4464 payload 274 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.108+0000 7f0191d3e640 1 -- 10.20.193.16:0/892170562 >> v2:10.20.193.148:3300/0 conn(0x7f018c0c7d40 msgr2=0x7f018c0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.108+0000 7f0191d3e640 1 --2- 10.20.193.16:0/892170562 >> v2:10.20.193.148:3300/0 conn(0x7f018c0c7d40 0x7f018c0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.108+0000 7f0191d3e640 1 -- 10.20.193.16:0/892170562 >> v1:10.20.193.128:6789/0 conn(0x7f018c1538d0 legacy=0x7f018c15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.108+0000 7f0191d3e640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.108+0000 7f0191d3e640 1 monclient: found mon.noname-a 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.108+0000 7f0191d3e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.108+0000 7f0191d3e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.108+0000 7f0191d3e640 1 -- 10.20.193.16:0/892170562 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018c0c6160 con 0x7f018c15b530 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.109+0000 7f0191d3e640 1 --2- 10.20.193.16:0/892170562 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c15b530 0x7f018c074b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f018c06de30 tx=0x7f018002eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5420800272e7a51d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 1 -- 10.20.193.16:0/892170562 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018003c070 con 0x7f018c15b530 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 10 monclient: dump: 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.122 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 5 monclient: authenticate success, global_id 4464 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 1 -- 10.20.193.16:0/892170562 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018002fb20 con 0x7f018c15b530 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 1 -- 10.20.193.16:0/892170562 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180037440 con 0x7f018c15b530 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 4 set_mon_vals no callback set 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f019153d640 10 monclient: dump: 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 10 monclient: shutdown 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 1 -- 10.20.193.16:0/892170562 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c15b530 msgr2=0x7f018c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 1 --2- 10.20.193.16:0/892170562 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c15b530 0x7f018c074b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f018c06de30 tx=0x7f018002eeb0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 1 -- 10.20.193.16:0/892170562 shutdown_connections 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 1 --2- 10.20.193.16:0/892170562 >> v2:10.20.193.148:3300/0 conn(0x7f018c0c7d40 0x7f018c0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 1 --2- 10.20.193.16:0/892170562 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c15b530 0x7f018c074b20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 1 -- 10.20.193.16:0/892170562 >> 10.20.193.16:0/892170562 conn(0x7f018c073f00 msgr2=0x7f018c074310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 1 -- 10.20.193.16:0/892170562 shutdown_connections 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.111+0000 7f01947ca640 1 -- 10.20.193.16:0/892170562 wait complete. 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.112+0000 7f01947ca640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.112+0000 7f01947ca640 10 monclient: monmap: 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.112+0000 7f01947ca640 1 librados: starting msgr at 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.112+0000 7f01947ca640 1 librados: starting objecter 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.112+0000 7f01947ca640 1 Processor -- start 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.112+0000 7f01947ca640 1 -- start start 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.112+0000 7f01947ca640 1 librados: setting wanted keys 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.112+0000 7f01947ca640 1 librados: calling monclient init 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.112+0000 7f01947ca640 10 monclient: init 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c0c7d40 0x7f018c13eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 10 monclient(hunting): picked mon.noname-a con 0x7f018c0c7d40 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f018c1538d0 0x7f018c1547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 10 monclient(hunting): picked mon.noname-b con 0x7f018c1538d0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f018c15b530 0x7f018c154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 10 monclient(hunting): picked mon.noname-c con 0x7f018c15b530 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f018c0c8690 con 0x7f018c0c7d40 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f018c15dde0 con 0x7f018c1538d0 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f018c153cb0 con 0x7f018c15b530 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f01947ca640 10 monclient(hunting): authenticate will time out at 1061.682098s 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f019253f640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c0c7d40 0x7f018c13eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f019253f640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c0c7d40 0x7f018c13eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:60134/0 (socket says 10.20.193.16:60134) 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f019253f640 1 -- 10.20.193.16:0/327461132 learned_addr learned my addr 10.20.193.16:0/327461132 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f019253f640 10 monclient(hunting): get_auth_request con 0x7f018c0c7d40 auth_method 0 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f019253f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f019253f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f019253f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f019253f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f019253f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.113+0000 7f019253f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f019253f640 10 monclient(hunting): handle_auth_done global_id 4467 payload 824 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f019253f640 1 -- 10.20.193.16:0/327461132 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f018c15b530 msgr2=0x7f018c154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f019253f640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f018c15b530 0x7f018c154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f019253f640 1 -- 10.20.193.16:0/327461132 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f018c1538d0 msgr2=0x7f018c1547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f019253f640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f018c1538d0 0x7f018c1547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f019253f640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f019253f640 1 monclient: found mon.noname-a 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f019253f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f019253f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f019253f640 1 -- 10.20.193.16:0/327461132 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018c1f4980 con 0x7f018c0c7d40 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f54e9d0a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f54e9d0a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f54e9d0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.115+0000 7f54e9d0a640 1 -- 10.20.193.16:0/1229688593 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 12} v 0) -- 0x7f54e41573f0 con 0x7f54e40700f0 2026-02-17T05:56:49.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.116+0000 7f019253f640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c0c7d40 0x7f018c13eec0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f017c00b570 tx=0x7f017c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.116+0000 7f0172ffd640 10 client.?.objecter ms_handle_connect 0x7f018c0c7d40 2026-02-17T05:56:49.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.116+0000 7f0172ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:49.126 INFO:teuthology.orchestra.run.trial016.stdout:120259084297 2026-02-17T05:56:49.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.118+0000 7f54bffff640 1 -- 10.20.193.16:0/1229688593 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 12}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f54d0062890 con 0x7f54e40700f0 2026-02-17T05:56:49.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.118+0000 7f54bffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-17T05:56:49.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.118+0000 7f54bffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 1 -- 10.20.193.16:0/327461132 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c013020 con 0x7f018c0c7d40 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 10 monclient: dump: 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 1 -- 10.20.193.16:0/327461132 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f017c0044a0 con 0x7f018c0c7d40 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 1 -- 10.20.193.16:0/327461132 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c0029c0 con 0x7f018c0c7d40 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0172ffd640 10 monclient: dump: 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.127 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f0190d3c640 4 set_mon_vals no callback set 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f01947ca640 5 monclient: authenticate success, global_id 4467 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f01947ca640 10 monclient: _renew_subs 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f01947ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f01947ca640 1 -- 10.20.193.16:0/327461132 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018c1f3e00 con 0x7f018c0c7d40 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f01947ca640 10 client.4467.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f01947ca640 10 monclient: _renew_subs 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f01947ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f01947ca640 1 -- 10.20.193.16:0/327461132 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018c1f43c0 con 0x7f018c0c7d40 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f01947ca640 1 librados: init done 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f54e9d0a640 10 librados: watch_flush enter 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.120+0000 7f54e9d0a640 10 librados: watch_flush exit 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 20 client.4461.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 10 client.4461.objecter successfully canceled tick 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 -- 10.20.193.16:0/1229688593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f54b40791c0 msgr2=0x7f54b407b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f54b40791c0 0x7f54b407b680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54e415aa70 tx=0x7f54d4006ca0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 10 monclient: shutdown 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 -- 10.20.193.16:0/1229688593 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e40700f0 msgr2=0x7f54e406d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e40700f0 0x7f54e406d710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54d000b530 tx=0x7f54d000ba00 comp rx=0 tx=0).stop 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 -- 10.20.193.16:0/1229688593 shutdown_connections 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f54b40791c0 0x7f54b407b680 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f54e40c7fe0 0x7f54e4073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f54e41536f0 0x7f54e406e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 --2- 10.20.193.16:0/1229688593 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f54e40700f0 0x7f54e406d710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 -- 10.20.193.16:0/1229688593 >> 10.20.193.16:0/1229688593 conn(0x7f54e40cd770 msgr2=0x7f54e4151180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 -- 10.20.193.16:0/1229688593 shutdown_connections 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 -- 10.20.193.16:0/1229688593 wait complete. 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f54e9d0a640 1 librados: shutdown 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f01947ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f01947ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f01947ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.121+0000 7f01947ca640 1 -- 10.20.193.16:0/327461132 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f018c0c8150 con 0x7f018c0c7d40 2026-02-17T05:56:49.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084296 got 120259084297 for osd.12 2026-02-17T05:56:49.130 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:49.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.148+0000 7fb50a985640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-17T05:56:49.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.148+0000 7fb50a985640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-17T05:56:49.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.148+0000 7fb50a985640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.148+0000 7fb50a985640 1 -- 10.20.193.16:0/2234917072 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 13} v 0) -- 0x7fb504150a40 con 0x7fb504155e00 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.163+0000 7f58b6b9b640 10 monclient: get_monmap_and_config 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.163+0000 7f58b6b9b640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.163+0000 7f58b6b9b640 10 monclient: monmap: 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:49.164027+0000 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:49.164027+0000 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.163+0000 7f58b6b9b640 1 Processor -- start 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.163+0000 7f58b6b9b640 1 -- start start 2026-02-17T05:56:49.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.163+0000 7f58b6b9b640 10 monclient: init 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 10 monclient: _add_conns ranks=[4,2,5,0,3,1] 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f58b015f5f0 0x7f58b015f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 10 monclient(hunting): picked mon.noname-b con 0x7f58b015f5f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f58b00cc0c0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f58b00cc4f0 0x7f58b015d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 10 monclient(hunting): picked mon.noname-c con 0x7f58b00cc4f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b00ccb00 con 0x7f58b00cc0c0 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f58b00ccde0 con 0x7f58b015f5f0 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f58b00cd0c0 con 0x7f58b00cc4f0 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.164+0000 7f58b6b9b640 10 monclient(hunting): authenticate will time out at 1061.733119s 2026-02-17T05:56:49.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.183+0000 7f9884ff9640 1 -- 10.20.193.16:0/4258518166 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f987c0621b0 con 0x7f98881561e0 2026-02-17T05:56:49.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.183+0000 7f9884ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.183+0000 7f9884ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.185+0000 7f57a2734640 10 monclient: get_monmap_and_config 2026-02-17T05:56:49.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.185+0000 7f57a2734640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.185+0000 7f57a2734640 10 monclient: monmap: 2026-02-17T05:56:49.185 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.185 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.185 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:49.186553+0000 2026-02-17T05:56:49.185 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:49.186553+0000 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 1 Processor -- start 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 1 -- start start 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 10 monclient: init 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 10 monclient: _add_conns ranks=[4,2,3,1,5,0] 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f579c155da0 0x7f579c0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 10 monclient(hunting): picked mon.noname-b con 0x7f579c155da0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f579c156180 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f579c15f2c0 0x7f579c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 10 monclient(hunting): picked mon.noname-a con 0x7f579c15f2c0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f579c15fc50 con 0x7f579c156180 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f579c06f530 con 0x7f579c15f2c0 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f579c06f810 con 0x7f579c155da0 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.186+0000 7f57a2734640 10 monclient(hunting): authenticate will time out at 1061.755591s 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.187+0000 7fd31fe4a640 10 monclient: get_monmap_and_config 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.187+0000 7fd31fe4a640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.187+0000 7fd31fe4a640 10 monclient: monmap: 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.187 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:49.188181+0000 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:49.188181+0000 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.187+0000 7fd31fe4a640 1 Processor -- start 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.187+0000 7fd31fe4a640 1 -- start start 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.187+0000 7fd31fe4a640 10 monclient: init 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 10 monclient: _add_conns ranks=[5,2,3,1,0,4] 2026-02-17T05:56:49.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd318155e00 0x7fd3180ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 10 monclient(hunting): picked mon.noname-c con 0x7fd318155e00 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd3181561e0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fd318074dc0 0x7fd3180cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 10 monclient(hunting): picked mon.noname-a con 0x7fd318074dc0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3180c96d0 con 0x7fd3181561e0 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd318074860 con 0x7fd318074dc0 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd3180c9940 con 0x7fd318155e00 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.188+0000 7fd31fe4a640 10 monclient(hunting): authenticate will time out at 1061.757211s 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.193+0000 7f695deb7640 10 monclient: get_monmap_and_config 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.193+0000 7f695deb7640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.193+0000 7f695deb7640 10 monclient: monmap: 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:49.194345+0000 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:49.194345+0000 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:49.193 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.193+0000 7f695deb7640 1 Processor -- start 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.193+0000 7f695deb7640 1 -- start start 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.193+0000 7f695deb7640 10 monclient: init 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 10 monclient: _add_conns ranks=[5,1,4,0,2,3] 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f69580700f0 0x7f69580704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 10 monclient(hunting): picked mon.noname-c con 0x7f69580700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f69580c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f69581536f0 0x7f6958153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 10 monclient(hunting): picked mon.noname-b con 0x7f69581536f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69580c4820 con 0x7f69580c7fe0 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f69580c4b00 con 0x7f69581536f0 2026-02-17T05:56:49.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f69580c4de0 con 0x7f69580700f0 2026-02-17T05:56:49.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.194+0000 7f695deb7640 10 monclient(hunting): authenticate will time out at 1061.763393s 2026-02-17T05:56:49.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.197+0000 7fc099ffb640 1 -- 10.20.193.16:0/3816580839 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fc09c03f3c0 con 0x7fc0ac074dc0 2026-02-17T05:56:49.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.197+0000 7fc099ffb640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc07c0791c0 0x7fc07c07b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.197+0000 7fc0b0ac7640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc07c0791c0 0x7fc07c07b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.197+0000 7fc0b0ac7640 10 monclient: get_auth_request con 0x7fc07c0791c0 auth_method 0 2026-02-17T05:56:49.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.197+0000 7fc099ffb640 10 client.4552.objecter ms_handle_connect 0x7fc07c0791c0 2026-02-17T05:56:49.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.197+0000 7fc0b0ac7640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc07c0791c0 0x7fc07c07b680 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0002770 tx=0x7fc0a0007ad0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.199+0000 7fc099ffb640 1 -- 10.20.193.16:0/3816580839 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fc09c082f70 con 0x7fc0ac074dc0 2026-02-17T05:56:49.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.199+0000 7fc099ffb640 10 client.4552.objecter ms_dispatch2 0x7fc0ac000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.199+0000 7fc099ffb640 3 client.4552.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.199+0000 7fc099ffb640 3 client.4552.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.199+0000 7fc099ffb640 20 client.4552.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.206+0000 7fc099ffb640 1 -- 10.20.193.16:0/3816580839 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc09c051020 con 0x7fc0ac074dc0 2026-02-17T05:56:49.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.206+0000 7fc099ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.206+0000 7fc099ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 1 -- 10.20.193.16:0/2317619249 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700045020 con 0x7f6714155e00 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 10 monclient: dump: 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 1 -- 10.20.193.16:0/2317619249 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670003d040 con 0x7f6714155e00 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 1 -- 10.20.193.16:0/2317619249 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670003c070 con 0x7f6714155e00 2026-02-17T05:56:49.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f671c8a9640 5 monclient: authenticate success, global_id 4555 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f671c8a9640 10 monclient: _renew_subs 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f671c8a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f671c8a9640 1 -- 10.20.193.16:0/2317619249 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67141f4e40 con 0x7f6714155e00 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f6718e1b640 4 set_mon_vals no callback set 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f66feffd640 10 monclient: dump: 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f671c8a9640 10 client.4555.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f671c8a9640 10 monclient: _renew_subs 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f671c8a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f671c8a9640 1 -- 10.20.193.16:0/2317619249 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67140c5b80 con 0x7f6714155e00 2026-02-17T05:56:49.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.208+0000 7f671c8a9640 1 librados: init done 2026-02-17T05:56:49.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.209+0000 7f671c8a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.209+0000 7f671c8a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.209+0000 7f671c8a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.209+0000 7f671c8a9640 1 -- 10.20.193.16:0/2317619249 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67140c8fe0 con 0x7f6714155e00 2026-02-17T05:56:49.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.218+0000 7f58b639a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f58b00cc4f0 0x7f58b015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f58b639a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f58b00cc4f0 0x7f58b015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57388/0 (socket says 10.20.193.16:57388) 2026-02-17T05:56:49.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f58b639a640 1 -- 10.20.193.16:0/1772812038 learned_addr learned my addr 10.20.193.16:0/1772812038 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f58b639a640 10 monclient(hunting): get_auth_request con 0x7f58b00cc4f0 auth_method 0 2026-02-17T05:56:49.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f58b639a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f58b639a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f58b639a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f69577fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f69580700f0 0x7f69580704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f69577fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f69580700f0 0x7f69580704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57406/0 (socket says 10.20.193.16:57406) 2026-02-17T05:56:49.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f69577fe640 1 -- 10.20.193.16:0/286620519 learned_addr learned my addr 10.20.193.16:0/286620519 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f69577fe640 10 monclient(hunting): get_auth_request con 0x7f69580700f0 auth_method 0 2026-02-17T05:56:49.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f69577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f69577fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f69577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f58b639a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f58b639a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.219+0000 7f58b639a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f69577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f69577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f69577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f58b639a640 10 monclient(hunting): handle_auth_done global_id 4598 payload 274 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f58b639a640 1 -- 10.20.193.16:0/1772812038 >> v2:10.20.193.128:3300/0 conn(0x7f58b015f5f0 msgr2=0x7f58b015f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f58b639a640 1 --2- 10.20.193.16:0/1772812038 >> v2:10.20.193.128:3300/0 conn(0x7f58b015f5f0 0x7f58b015f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f58b639a640 1 -- 10.20.193.16:0/1772812038 >> v1:10.20.193.148:6789/0 conn(0x7f58b00cc0c0 legacy=0x7f58b0073eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f58b639a640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f58b639a640 1 monclient: found mon.noname-c 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f58b639a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f58b639a640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.220+0000 7f58b639a640 1 -- 10.20.193.16:0/1772812038 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b00cd3a0 con 0x7f58b00cc4f0 2026-02-17T05:56:49.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.222+0000 7f69577fe640 10 monclient(hunting): handle_auth_done global_id 4601 payload 274 2026-02-17T05:56:49.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.222+0000 7f69577fe640 1 -- 10.20.193.16:0/286620519 >> v2:10.20.193.128:3300/0 conn(0x7f69581536f0 msgr2=0x7f6958153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.222+0000 7f69577fe640 1 --2- 10.20.193.16:0/286620519 >> v2:10.20.193.128:3300/0 conn(0x7f69581536f0 0x7f6958153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.222+0000 7f69577fe640 1 -- 10.20.193.16:0/286620519 >> v1:10.20.193.128:6789/0 conn(0x7f69580c7fe0 legacy=0x7f69580c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.222+0000 7f69577fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.222+0000 7f69577fe640 1 monclient: found mon.noname-c 2026-02-17T05:56:49.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.222+0000 7f69577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.222+0000 7f69577fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.222+0000 7f69577fe640 1 -- 10.20.193.16:0/286620519 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69580c50c0 con 0x7f69580700f0 2026-02-17T05:56:49.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.223+0000 7f58b639a640 1 --2- 10.20.193.16:0/1772812038 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 0x7f58b015d380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009e50 tx=0x7f58ac02f260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ed03766b48a7c81 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.224+0000 7f69577fe640 1 --2- 10.20.193.16:0/286620519 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580700f0 0x7f69580704d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6958070b90 tx=0x7f694402ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e34534339bbb2ca5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.225+0000 7fd31dbbf640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd318155e00 0x7fd3180ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.226+0000 7fd31dbbf640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd318155e00 0x7fd3180ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57404/0 (socket says 10.20.193.16:57404) 2026-02-17T05:56:49.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.226+0000 7fd31dbbf640 1 -- 10.20.193.16:0/2139014575 learned_addr learned my addr 10.20.193.16:0/2139014575 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.226+0000 7fd31dbbf640 10 monclient(hunting): get_auth_request con 0x7fd318155e00 auth_method 0 2026-02-17T05:56:49.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.226+0000 7fd31dbbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.226+0000 7fd31dbbf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.226+0000 7fd31dbbf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.227+0000 7fd31dbbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.227+0000 7fd31dbbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.227+0000 7fd31dbbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.227+0000 7f579b7fe640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f579c156180 0x7f579c0c9f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46626/0 (socket says 10.20.193.16:46626) 2026-02-17T05:56:49.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.227+0000 7f579b7fe640 1 -- 10.20.193.16:0/2321540905 learned_addr learned my addr 10.20.193.16:0/2321540905 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:49.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.228+0000 7fd31dbbf640 10 monclient(hunting): handle_auth_done global_id 4604 payload 274 2026-02-17T05:56:49.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.228+0000 7fd31dbbf640 1 -- 10.20.193.16:0/2139014575 >> v2:10.20.193.16:3300/0 conn(0x7fd318074dc0 msgr2=0x7fd3180cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.228+0000 7fd31dbbf640 1 --2- 10.20.193.16:0/2139014575 >> v2:10.20.193.16:3300/0 conn(0x7fd318074dc0 0x7fd3180cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.228+0000 7fd31dbbf640 1 -- 10.20.193.16:0/2139014575 >> v1:10.20.193.148:6789/0 conn(0x7fd3181561e0 legacy=0x7fd318074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.228+0000 7fd31dbbf640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.228+0000 7fd31dbbf640 1 monclient: found mon.noname-c 2026-02-17T05:56:49.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.228+0000 7fd31dbbf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.228+0000 7fd31dbbf640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.228+0000 7fd31dbbf640 1 -- 10.20.193.16:0/2139014575 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3180c9c20 con 0x7fd318155e00 2026-02-17T05:56:49.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.229+0000 7fb500ff9640 1 -- 10.20.193.16:0/2234917072 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 13}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb4f4061fe0 con 0x7fb504155e00 2026-02-17T05:56:49.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.229+0000 7fb500ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-17T05:56:49.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.229+0000 7fb500ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.229 INFO:teuthology.orchestra.run.trial016.stdout:115964116998 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 1 -- 10.20.193.16:0/286620519 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694403c070 con 0x7f69580700f0 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 10 monclient: dump: 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.230 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 1 -- 10.20.193.16:0/286620519 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f694402fa90 con 0x7f69580700f0 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 1 -- 10.20.193.16:0/286620519 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694402fe60 con 0x7f69580700f0 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 5 monclient: authenticate success, global_id 4601 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 4 set_mon_vals no callback set 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f69567fc640 10 monclient: dump: 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.231 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.232 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.232 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.232 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 10 monclient: shutdown 2026-02-17T05:56:49.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 1 -- 10.20.193.16:0/286620519 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580700f0 msgr2=0x7f69580704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 1 --2- 10.20.193.16:0/286620519 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580700f0 0x7f69580704d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6958070b90 tx=0x7f694402ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:49.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7fd31dbbf640 1 --2- 10.20.193.16:0/2139014575 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd318155e00 0x7fd3180ca700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd31806ce20 tx=0x7fd30802eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=125689289533b3ab server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 1 -- 10.20.193.16:0/286620519 shutdown_connections 2026-02-17T05:56:49.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 1 --2- 10.20.193.16:0/286620519 >> v2:10.20.193.128:3300/0 conn(0x7f69581536f0 0x7f6958153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 1 --2- 10.20.193.16:0/286620519 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580700f0 0x7f69580704d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 1 -- 10.20.193.16:0/286620519 >> 10.20.193.16:0/286620519 conn(0x7f69580cd770 msgr2=0x7f69580cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 1 -- 10.20.193.16:0/286620519 shutdown_connections 2026-02-17T05:56:49.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.230+0000 7f695deb7640 1 -- 10.20.193.16:0/286620519 wait complete. 2026-02-17T05:56:49.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient: monmap: 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 librados: starting msgr at 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 librados: starting objecter 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 Processor -- start 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 -- start start 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 librados: setting wanted keys 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 librados: calling monclient init 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient: init 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f69580700f0 0x7f6958140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient(hunting): picked mon.noname-b con 0x7f69580700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580c7fe0 0x7f695806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient(hunting): picked mon.noname-c con 0x7f69580c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69581536f0 0x7f69581f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient(hunting): picked mon.noname-a con 0x7f69581536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6958154060 con 0x7f69581536f0 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f69580c5680 con 0x7f69580700f0 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6958074c00 con 0x7f69580c7fe0 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7f695deb7640 10 monclient(hunting): authenticate will time out at 1061.800772s 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 10 librados: watch_flush enter 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 10 librados: watch_flush exit 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 20 client.4589.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 10 client.4589.objecter successfully canceled tick 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 1 -- 10.20.193.16:0/2234917072 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4d4079140 msgr2=0x7fb4d407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4d4079140 0x7fb4d407b600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb5040693f0 tx=0x7fb4f0031000 comp rx=0 tx=0).stop 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 10 monclient: shutdown 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 1 -- 10.20.193.16:0/2234917072 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 msgr2=0x7fb50414ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 0x7fb50414ba00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400b500 tx=0x7fb4f400b9d0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 1 -- 10.20.193.16:0/2234917072 shutdown_connections 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4d4079140 0x7fb4d407b600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5041561e0 0x7fb50414c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb504155e00 0x7fb50414ba00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 1 --2- 10.20.193.16:0/2234917072 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb504074dc0 0x7fb5041f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.231+0000 7fb50a985640 1 -- 10.20.193.16:0/2234917072 >> 10.20.193.16:0/2234917072 conn(0x7fb5040ce6c0 msgr2=0x7fb504153620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.232+0000 7fb50a985640 1 -- 10.20.193.16:0/2234917072 shutdown_connections 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.232+0000 7fb50a985640 1 -- 10.20.193.16:0/2234917072 wait complete. 2026-02-17T05:56:49.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.232+0000 7fb50a985640 1 librados: shutdown 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.232+0000 7f6956ffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580c7fe0 0x7f695806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.232+0000 7f58b4b97640 1 -- 10.20.193.16:0/1772812038 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac02fc80 con 0x7f58b00cc4f0 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 10 monclient: dump: 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.237 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 1 -- 10.20.193.16:0/1772812038 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac02fe20 con 0x7f58b00cc4f0 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 1 -- 10.20.193.16:0/1772812038 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac037610 con 0x7f58b00cc4f0 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 5 monclient: authenticate success, global_id 4598 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 4 set_mon_vals no callback set 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b4b97640 10 monclient: dump: 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.238 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f6956ffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580c7fe0 0x7f695806de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57408/0 (socket says 10.20.193.16:57408) 2026-02-17T05:56:49.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f6956ffd640 1 -- 10.20.193.16:0/1644113027 learned_addr learned my addr 10.20.193.16:0/1644113027 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f6956ffd640 10 monclient(hunting): get_auth_request con 0x7f69580c7fe0 auth_method 0 2026-02-17T05:56:49.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f6956ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f6956ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f6956ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 10 monclient: shutdown 2026-02-17T05:56:49.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1772812038 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 msgr2=0x7f58b015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 1 --2- 10.20.193.16:0/1772812038 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 0x7f58b015d380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009e50 tx=0x7f58ac02f260 comp rx=0 tx=0).stop 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1772812038 shutdown_connections 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 1 --2- 10.20.193.16:0/1772812038 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 0x7f58b015d380 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 1 --2- 10.20.193.16:0/1772812038 >> v2:10.20.193.128:3300/0 conn(0x7f58b015f5f0 0x7f58b015f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1772812038 >> 10.20.193.16:0/1772812038 conn(0x7f58b00d1b60 msgr2=0x7f58b00d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1772812038 shutdown_connections 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1772812038 wait complete. 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 1 -- 10.20.193.16:0/2139014575 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30803c070 con 0x7fd318155e00 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 10 monclient: dump: 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.240 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 1 -- 10.20.193.16:0/2139014575 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30802fb40 con 0x7fd318155e00 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 1 -- 10.20.193.16:0/2139014575 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308037440 con 0x7fd318155e00 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 5 monclient: authenticate success, global_id 4604 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 4 set_mon_vals no callback set 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31cbbd640 10 monclient: dump: 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.241 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.242 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f6956ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f6956ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f6956ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 10 monclient: shutdown 2026-02-17T05:56:49.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 1 -- 10.20.193.16:0/2139014575 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd318155e00 msgr2=0x7fd3180ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 1 --2- 10.20.193.16:0/2139014575 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd318155e00 0x7fd3180ca700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd31806ce20 tx=0x7fd30802eed0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f0172ffd640 1 -- 10.20.193.16:0/327461132 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f017c002b60 con 0x7f018c0c7d40 2026-02-17T05:56:49.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f579bfff640 1 --2- 10.20.193.16:0/2321540905 >> v2:10.20.193.128:3300/0 conn(0x7f579c155da0 0x7f579c0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 1 -- 10.20.193.16:0/2139014575 shutdown_connections 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 1 --2- 10.20.193.16:0/2139014575 >> v2:10.20.193.16:3300/0 conn(0x7fd318074dc0 0x7fd3180cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 1 --2- 10.20.193.16:0/2139014575 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd318155e00 0x7fd3180ca700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 1 -- 10.20.193.16:0/2139014575 >> 10.20.193.16:0/2139014575 conn(0x7fd3180ce6c0 msgr2=0x7fd3180cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 1 -- 10.20.193.16:0/2139014575 shutdown_connections 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7fd31fe4a640 1 -- 10.20.193.16:0/2139014575 wait complete. 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.233+0000 7f58b6b9b640 10 monclient: monmap: 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.243 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f0172ffd640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f015c079140 0x7f015c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 librados: starting msgr at 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 librados: starting objecter 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 Processor -- start 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f0191d3e640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f015c079140 0x7f015c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 -- start start 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 librados: setting wanted keys 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 librados: calling monclient init 2026-02-17T05:56:49.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient: init 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient: monmap: 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f0191d3e640 10 monclient: get_auth_request con 0x7f015c079140 auth_method 0 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 librados: starting msgr at 2026-02-17T05:56:49.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 librados: starting objecter 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 Processor -- start 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 -- start start 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 librados: setting wanted keys 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 librados: calling monclient init 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient: init 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f58b00cc0c0 0x7f58b014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient(hunting): picked mon.noname-b con 0x7f58b00cc0c0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 0x7f58b014e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient(hunting): picked mon.noname-c con 0x7f58b00cc4f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f58b015f5f0 0x7f58b0148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f6956ffd640 10 monclient(hunting): handle_auth_done global_id 4607 payload 824 2026-02-17T05:56:49.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient(hunting): picked mon.noname-a con 0x7f58b015f5f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58b00745c0 con 0x7f58b015f5f0 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f58b00cd960 con 0x7f58b00cc0c0 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f6956ffd640 1 -- 10.20.193.16:0/1644113027 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f69580700f0 msgr2=0x7f6958140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f6956ffd640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f69580700f0 0x7f6958140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f6956ffd640 1 -- 10.20.193.16:0/1644113027 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69581536f0 msgr2=0x7f69581f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f6956ffd640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69581536f0 0x7f69581f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f6956ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f6956ffd640 1 monclient: found mon.noname-c 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f6956ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f6956ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58b0074740 con 0x7f58b00cc4f0 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f58b6b9b640 10 monclient(hunting): authenticate will time out at 1061.803928s 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd318074dc0 0x7fd3181f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient(hunting): picked mon.noname-c con 0x7fd318074dc0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7f6956ffd640 1 -- 10.20.193.16:0/1644113027 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69581f48f0 con 0x7f69580c7fe0 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd318155e00 0x7fd31814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient(hunting): picked mon.noname-a con 0x7fd318155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd3181561e0 0x7fd31814c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient(hunting): picked mon.noname-b con 0x7fd3181561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3180c9360 con 0x7fd318155e00 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd3180ca1e0 con 0x7fd3181561e0 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd3180c8c20 con 0x7fd318074dc0 2026-02-17T05:56:49.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.234+0000 7fd31fe4a640 10 monclient(hunting): authenticate will time out at 1061.803979s 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.235+0000 7f0172ffd640 10 client.4467.objecter ms_handle_connect 0x7f015c079140 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.235+0000 7f0191d3e640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f015c079140 0x7f015c07b600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f018c073130 tx=0x7f0180031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.236+0000 7f6956ffd640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580c7fe0 0x7f695806de80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f694800b500 tx=0x7f694800b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.236+0000 7f6933fff640 10 client.?.objecter ms_handle_connect 0x7f69580c7fe0 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.236+0000 7f6933fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.236+0000 7f579bfff640 10 monclient(hunting): get_auth_request con 0x7f579c155da0 auth_method 0 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.236+0000 7f579bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.236+0000 7f579bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.236+0000 7f579bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.237+0000 7f579bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.237+0000 7f579bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.237+0000 7f579bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f58b5398640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 0x7f58b014e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f579bfff640 10 monclient(hunting): handle_auth_done global_id 4558 payload 274 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f579bfff640 1 -- 10.20.193.16:0/2321540905 >> v2:10.20.193.16:3300/0 conn(0x7f579c15f2c0 msgr2=0x7f579c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f579bfff640 1 --2- 10.20.193.16:0/2321540905 >> v2:10.20.193.16:3300/0 conn(0x7f579c15f2c0 0x7f579c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f579bfff640 1 -- 10.20.193.16:0/2321540905 >> v1:10.20.193.148:6789/0 conn(0x7f579c156180 legacy=0x7f579c0c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f579bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f579bfff640 1 monclient: found mon.noname-b 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f579bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f579bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f579bfff640 1 -- 10.20.193.16:0/2321540905 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c06faf0 con 0x7f579c155da0 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f58b5398640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 0x7f58b014e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57424/0 (socket says 10.20.193.16:57424) 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f58b5398640 1 -- 10.20.193.16:0/1306035717 learned_addr learned my addr 10.20.193.16:0/1306035717 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f58b5398640 10 monclient(hunting): get_auth_request con 0x7f58b00cc4f0 auth_method 0 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f58b5398640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f58b5398640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.238+0000 7f58b5398640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 1 -- 10.20.193.16:0/1644113027 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948004480 con 0x7f69580c7fe0 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 10 monclient: dump: 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 1 -- 10.20.193.16:0/1644113027 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6948014070 con 0x7f69580c7fe0 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f695deb7640 5 monclient: authenticate success, global_id 4607 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f695deb7640 10 monclient: _renew_subs 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f695deb7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f695deb7640 1 -- 10.20.193.16:0/1644113027 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69581f4610 con 0x7f69580c7fe0 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 1 -- 10.20.193.16:0/1644113027 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948002920 con 0x7f69580c7fe0 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6933fff640 10 monclient: dump: 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.251 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f6955ffb640 4 set_mon_vals no callback set 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f695deb7640 10 client.4607.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f695deb7640 10 monclient: _renew_subs 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f695deb7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f695deb7640 1 -- 10.20.193.16:0/1644113027 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695813f650 con 0x7f69580c7fe0 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f695deb7640 1 librados: init done 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f58b5398640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f58b5398640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.239+0000 7f58b5398640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.240+0000 7f695deb7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.240+0000 7f695deb7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.240+0000 7f695deb7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.240+0000 7f695deb7640 1 -- 10.20.193.16:0/1644113027 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69580704d0 con 0x7f69580c7fe0 2026-02-17T05:56:49.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116998 got 115964116998 for osd.13 2026-02-17T05:56:49.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f579bfff640 1 --2- 10.20.193.16:0/2321540905 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c155da0 0x7f579c0d01b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f579c06a340 tx=0x7f578802eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4656cd76c03c5680 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.253 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:49.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f58b5398640 10 monclient(hunting): handle_auth_done global_id 4610 payload 824 2026-02-17T05:56:49.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f58b5398640 1 -- 10.20.193.16:0/1306035717 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f58b00cc0c0 msgr2=0x7f58b014dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f58b5398640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f58b00cc0c0 0x7f58b014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f58b5398640 1 -- 10.20.193.16:0/1306035717 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f58b015f5f0 msgr2=0x7f58b0148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f58b5398640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f58b015f5f0 0x7f58b0148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f58b5398640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f58b5398640 1 monclient: found mon.noname-c 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f58b5398640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f58b5398640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.241+0000 7f58b5398640 1 -- 10.20.193.16:0/1306035717 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b00cad50 con 0x7f58b00cc4f0 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.243+0000 7f0172ffd640 1 -- 10.20.193.16:0/327461132 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f017c05e2e0 con 0x7f018c0c7d40 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.244+0000 7f0172ffd640 10 client.4467.objecter ms_dispatch2 0x7f018c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.244+0000 7f0172ffd640 3 client.4467.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.244+0000 7f0172ffd640 3 client.4467.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.244+0000 7f0172ffd640 20 client.4467.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.244+0000 7f0172ffd640 1 -- 10.20.193.16:0/327461132 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f017c01d020 con 0x7f018c0c7d40 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.244+0000 7f0172ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.244+0000 7f0172ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.252+0000 7fd31d3be640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd318155e00 0x7fd31814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.253+0000 7fd31d3be640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd318155e00 0x7fd31814ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:60190/0 (socket says 10.20.193.16:60190) 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.253+0000 7fd31d3be640 1 -- 10.20.193.16:0/3412598768 learned_addr learned my addr 10.20.193.16:0/3412598768 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.253+0000 7fd31d3be640 10 monclient(hunting): get_auth_request con 0x7fd318155e00 auth_method 0 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.253+0000 7fd31d3be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.253+0000 7fd31d3be640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.253+0000 7fd31d3be640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.254+0000 7fd31d3be640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.254+0000 7fd31d3be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.254+0000 7fd31d3be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.260+0000 7fd31d3be640 10 monclient(hunting): handle_auth_done global_id 4470 payload 824 2026-02-17T05:56:49.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.260+0000 7fd31d3be640 1 -- 10.20.193.16:0/3412598768 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd318074dc0 msgr2=0x7fd3181f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.260+0000 7fd31d3be640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd318074dc0 0x7fd3181f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.260+0000 7fd31d3be640 1 -- 10.20.193.16:0/3412598768 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd3181561e0 msgr2=0x7fd31814c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.260+0000 7fd31d3be640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd3181561e0 0x7fd31814c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.260+0000 7fd31d3be640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.260+0000 7fd31d3be640 1 monclient: found mon.noname-a 2026-02-17T05:56:49.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.260+0000 7fd31d3be640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.260+0000 7fd31d3be640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.260+0000 7fd31d3be640 1 -- 10.20.193.16:0/3412598768 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3181f4870 con 0x7fd318155e00 2026-02-17T05:56:49.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.267+0000 7fd31d3be640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd318155e00 0x7fd31814ba00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00b4d0 tx=0x7fd30c00b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.267+0000 7fd3067fc640 10 client.?.objecter ms_handle_connect 0x7fd318155e00 2026-02-17T05:56:49.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.267+0000 7fd3067fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:49.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.268+0000 7f988e4e8640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-17T05:56:49.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.268+0000 7f988e4e8640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-17T05:56:49.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.268+0000 7f988e4e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.268+0000 7f988e4e8640 1 -- 10.20.193.16:0/4258518166 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9888073e60 con 0x7f98881561e0 2026-02-17T05:56:49.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.299+0000 7fc0b3553640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:56:49.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.299+0000 7fc0b3553640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:56:49.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.299+0000 7fc0b3553640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.299+0000 7fc0b3553640 1 -- 10.20.193.16:0/3816580839 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc0ac0c5800 con 0x7fc0ac074dc0 2026-02-17T05:56:49.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 1 -- 10.20.193.16:0/3412598768 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c004480 con 0x7fd318155e00 2026-02-17T05:56:49.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 10 monclient: dump: 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 1 -- 10.20.193.16:0/3412598768 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c015070 con 0x7fd318155e00 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd31fe4a640 5 monclient: authenticate success, global_id 4470 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd31fe4a640 10 monclient: _renew_subs 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd31fe4a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd31fe4a640 1 -- 10.20.193.16:0/3412598768 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31814ce90 con 0x7fd318155e00 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 1 -- 10.20.193.16:0/3412598768 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c002920 con 0x7fd318155e00 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd3067fc640 10 monclient: dump: 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.307 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd307fff640 4 set_mon_vals no callback set 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd31fe4a640 10 client.4470.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd31fe4a640 10 monclient: _renew_subs 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd31fe4a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd31fe4a640 1 -- 10.20.193.16:0/3412598768 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3181432d0 con 0x7fd318155e00 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.305+0000 7fd31fe4a640 1 librados: init done 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.306+0000 7fd31fe4a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.306+0000 7fd31fe4a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.306+0000 7fd31fe4a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.306+0000 7fd31fe4a640 1 -- 10.20.193.16:0/3412598768 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3180c8fe0 con 0x7fd318155e00 2026-02-17T05:56:49.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.316+0000 7fd3067fc640 1 -- 10.20.193.16:0/3412598768 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fd30c010480 con 0x7fd318155e00 2026-02-17T05:56:49.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.316+0000 7fd3067fc640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd2e8079190 0x7fd2e807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.317+0000 7fd31dbbf640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd2e8079190 0x7fd2e807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.317+0000 7fd31dbbf640 10 monclient: get_auth_request con 0x7fd2e8079190 auth_method 0 2026-02-17T05:56:49.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.317+0000 7fd3067fc640 10 client.4470.objecter ms_handle_connect 0x7fd2e8079190 2026-02-17T05:56:49.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.317+0000 7fd31dbbf640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd2e8079190 0x7fd2e807b650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3180693f0 tx=0x7fd308031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.318+0000 7fd3067fc640 1 -- 10.20.193.16:0/3412598768 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fd30c05e440 con 0x7fd318155e00 2026-02-17T05:56:49.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.318+0000 7fd3067fc640 10 client.4470.objecter ms_dispatch2 0x7fd318000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.318+0000 7fd3067fc640 3 client.4470.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.318+0000 7fd3067fc640 3 client.4470.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.318+0000 7fd3067fc640 20 client.4470.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.334+0000 7f01947ca640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-17T05:56:49.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.334+0000 7f01947ca640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-17T05:56:49.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.334+0000 7f01947ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.334+0000 7f01947ca640 1 -- 10.20.193.16:0/327461132 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 14} v 0) -- 0x7f018c06bd60 con 0x7f018c0c7d40 2026-02-17T05:56:49.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.346+0000 7f66feffd640 1 -- 10.20.193.16:0/2317619249 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f6700041a50 con 0x7f6714155e00 2026-02-17T05:56:49.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.346+0000 7f66feffd640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f66e4079190 0x7f66e407b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.347+0000 7f671a61e640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f66e4079190 0x7f66e407b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.347+0000 7f671a61e640 10 monclient: get_auth_request con 0x7f66e4079190 auth_method 0 2026-02-17T05:56:49.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.347+0000 7f66feffd640 10 client.4555.objecter ms_handle_connect 0x7f66e4079190 2026-02-17T05:56:49.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.347+0000 7f671a61e640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f66e4079190 0x7f66e407b650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f670c0098a0 tx=0x7f670c006d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.348+0000 7f66feffd640 1 -- 10.20.193.16:0/2317619249 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f6700083a00 con 0x7f6714155e00 2026-02-17T05:56:49.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.348+0000 7f66feffd640 10 client.4555.objecter ms_dispatch2 0x7f6714000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.348+0000 7f66feffd640 3 client.4555.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.348+0000 7f66feffd640 3 client.4555.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.348+0000 7f66feffd640 20 client.4555.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.349+0000 7f6933fff640 1 -- 10.20.193.16:0/1644113027 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f6948019460 con 0x7f69580c7fe0 2026-02-17T05:56:49.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.349+0000 7f6933fff640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6928079190 0x7f692807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.349+0000 7f69577fe640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6928079190 0x7f692807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.349+0000 7f69577fe640 10 monclient: get_auth_request con 0x7f6928079190 auth_method 0 2026-02-17T05:56:49.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.350+0000 7f6933fff640 10 client.4607.objecter ms_handle_connect 0x7f6928079190 2026-02-17T05:56:49.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.350+0000 7f69577fe640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6928079190 0x7f692807b650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f69580c5b80 tx=0x7f6944031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.350+0000 7f6933fff640 1 -- 10.20.193.16:0/1644113027 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f694805e010 con 0x7f69580c7fe0 2026-02-17T05:56:49.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.350+0000 7f6933fff640 10 client.4607.objecter ms_dispatch2 0x7f6958000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.350+0000 7f6933fff640 3 client.4607.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.350+0000 7f6933fff640 3 client.4607.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.351+0000 7f6933fff640 20 client.4607.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.354+0000 7f66feffd640 1 -- 10.20.193.16:0/2317619249 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f670004f020 con 0x7f6714155e00 2026-02-17T05:56:49.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.354+0000 7f66feffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.354+0000 7f66feffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.357+0000 7f58b5398640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 0x7f58b014e4f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f58a400b570 tx=0x7f58a400ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.357+0000 7f589e7fc640 10 client.?.objecter ms_handle_connect 0x7f58b00cc4f0 2026-02-17T05:56:49.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.357+0000 7f589e7fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:49.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.357+0000 7f6933fff640 1 -- 10.20.193.16:0/1644113027 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f694800e360 con 0x7f69580c7fe0 2026-02-17T05:56:49.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.357+0000 7f6933fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.357+0000 7f6933fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.361+0000 7f9884ff9640 1 -- 10.20.193.16:0/4258518166 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f987c061ab0 con 0x7f98881561e0 2026-02-17T05:56:49.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.361+0000 7f9884ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-17T05:56:49.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.361+0000 7f9884ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.362 INFO:teuthology.orchestra.run.trial016.stdout:90194313221 2026-02-17T05:56:49.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 1 -- 10.20.193.16:0/2321540905 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578803c070 con 0x7f579c155da0 2026-02-17T05:56:49.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:49.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 10 monclient: dump: 2026-02-17T05:56:49.364 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.365 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.365 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.365 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.365 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.365 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.365 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.365 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.365 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.366 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 1 -- 10.20.193.16:0/2321540905 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f578802fb80 con 0x7f579c155da0 2026-02-17T05:56:49.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 1 -- 10.20.193.16:0/2321540905 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788037440 con 0x7f579c155da0 2026-02-17T05:56:49.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 5 monclient: authenticate success, global_id 4558 2026-02-17T05:56:49.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 4 set_mon_vals no callback set 2026-02-17T05:56:49.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:49.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f579affd640 10 monclient: dump: 2026-02-17T05:56:49.367 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.367 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.367 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.367 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.368 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.368 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.368 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.368 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.368 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.368 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 10 librados: watch_flush enter 2026-02-17T05:56:49.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 10 librados: watch_flush exit 2026-02-17T05:56:49.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 20 client.4595.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 10 client.4595.objecter successfully canceled tick 2026-02-17T05:56:49.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 -- 10.20.193.16:0/4258518166 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9858079140 msgr2=0x7f985807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9858079140 0x7f985807b600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f988814e0a0 tx=0x7f98740091c0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 10 monclient: shutdown 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 -- 10.20.193.16:0/4258518166 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 msgr2=0x7f98880c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 0x7f98880c7f10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f987c00b990 tx=0x7f987c00be60 comp rx=0 tx=0).stop 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 -- 10.20.193.16:0/4258518166 shutdown_connections 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9858079140 0x7f985807b600 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f98881561e0 0x7f98880c7f10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9888155e00 0x7f9888153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 --2- 10.20.193.16:0/4258518166 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9888074dc0 0x7f988814e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 -- 10.20.193.16:0/4258518166 >> 10.20.193.16:0/4258518166 conn(0x7f98880ce6c0 msgr2=0x7f98880c8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 -- 10.20.193.16:0/4258518166 shutdown_connections 2026-02-17T05:56:49.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 -- 10.20.193.16:0/4258518166 wait complete. 2026-02-17T05:56:49.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f988e4e8640 1 librados: shutdown 2026-02-17T05:56:49.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 10 monclient: shutdown 2026-02-17T05:56:49.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 1 -- 10.20.193.16:0/2321540905 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c155da0 msgr2=0x7f579c0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 1 --2- 10.20.193.16:0/2321540905 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c155da0 0x7f579c0d01b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f579c06a340 tx=0x7f578802eeb0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 1 -- 10.20.193.16:0/2321540905 shutdown_connections 2026-02-17T05:56:49.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 1 --2- 10.20.193.16:0/2321540905 >> v2:10.20.193.16:3300/0 conn(0x7f579c15f2c0 0x7f579c15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 1 --2- 10.20.193.16:0/2321540905 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c155da0 0x7f579c0d01b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 1 -- 10.20.193.16:0/2321540905 >> 10.20.193.16:0/2321540905 conn(0x7f579c0cf600 msgr2=0x7f579c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 1 -- 10.20.193.16:0/2321540905 shutdown_connections 2026-02-17T05:56:49.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.363+0000 7f57a2734640 1 -- 10.20.193.16:0/2321540905 wait complete. 2026-02-17T05:56:49.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.364+0000 7f57a2734640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.364+0000 7f57a2734640 10 monclient: monmap: 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.364+0000 7f57a2734640 1 librados: starting msgr at 2026-02-17T05:56:49.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.364+0000 7f57a2734640 1 librados: starting objecter 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.364+0000 7f57a2734640 1 Processor -- start 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.364+0000 7f57a2734640 1 -- start start 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.364+0000 7f57a2734640 1 librados: setting wanted keys 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.364+0000 7f57a2734640 1 librados: calling monclient init 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.364+0000 7f57a2734640 10 monclient: init 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f579c155da0 0x7f579c1f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 10 monclient(hunting): picked mon.noname-a con 0x7f579c155da0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f579c156180 0x7f579c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 10 monclient(hunting): picked mon.noname-c con 0x7f579c156180 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c15f2c0 0x7f579c14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 10 monclient(hunting): picked mon.noname-b con 0x7f579c15f2c0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f579c06d870 con 0x7f579c155da0 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f579c0700b0 con 0x7f579c15f2c0 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f579c0705d0 con 0x7f579c156180 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.365+0000 7f57a2734640 10 monclient(hunting): authenticate will time out at 1061.934163s 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.366+0000 7f57a0caa640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c15f2c0 0x7f579c14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.366+0000 7f57a0caa640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c15f2c0 0x7f579c14dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50708/0 (socket says 10.20.193.16:50708) 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.366+0000 7f57a0caa640 1 -- 10.20.193.16:0/525399189 learned_addr learned my addr 10.20.193.16:0/525399189 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.378 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.366+0000 7f57a0caa640 10 monclient(hunting): get_auth_request con 0x7f579c15f2c0 auth_method 0 2026-02-17T05:56:49.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.366+0000 7f57a0caa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.366+0000 7f57a0caa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.366+0000 7f57a0caa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.366+0000 7f57a0caa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.366+0000 7f57a0caa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.366+0000 7f57a0caa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 1 -- 10.20.193.16:0/1306035717 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4013020 con 0x7f58b00cc4f0 2026-02-17T05:56:49.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:49.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 10 monclient: dump: 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 1 -- 10.20.193.16:0/1306035717 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a40044a0 con 0x7f58b00cc4f0 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 1 -- 10.20.193.16:0/1306035717 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a40029c0 con 0x7f58b00cc4f0 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f58b6b9b640 5 monclient: authenticate success, global_id 4610 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589ffff640 4 set_mon_vals no callback set 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f589e7fc640 10 monclient: dump: 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.382 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f58b6b9b640 10 monclient: _renew_subs 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f58b6b9b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1306035717 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b0153240 con 0x7f58b00cc4f0 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f58b6b9b640 10 client.4610.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f58b6b9b640 10 monclient: _renew_subs 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f58b6b9b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1306035717 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b0153800 con 0x7f58b00cc4f0 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.367+0000 7f58b6b9b640 1 librados: init done 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f58b6b9b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f58b6b9b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f58b6b9b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1306035717 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b00c81e0 con 0x7f58b00cc4f0 2026-02-17T05:56:49.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f579b7fe640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f579c156180 0x7f579c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f57a0caa640 10 monclient(hunting): handle_auth_done global_id 4561 payload 824 2026-02-17T05:56:49.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f57a0caa640 1 -- 10.20.193.16:0/525399189 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f579c156180 msgr2=0x7f579c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f57a0caa640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f579c156180 0x7f579c0cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f57a0caa640 1 -- 10.20.193.16:0/525399189 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f579c155da0 msgr2=0x7f579c1f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f57a0caa640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f579c155da0 0x7f579c1f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f57a0caa640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f57a0caa640 1 monclient: found mon.noname-b 2026-02-17T05:56:49.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f57a0caa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f57a0caa640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.368+0000 7f57a0caa640 1 -- 10.20.193.16:0/525399189 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c1f4dc0 con 0x7f579c15f2c0 2026-02-17T05:56:49.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.369+0000 7f57a0caa640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c15f2c0 0x7f579c14dd40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f579000b970 tx=0x7f579000be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.369+0000 7f5798ff9640 10 client.?.objecter ms_handle_connect 0x7f579c15f2c0 2026-02-17T05:56:49.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.369+0000 7f5798ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:49.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.6 2026-02-17T05:56:49.386 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:49.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.403+0000 7fc099ffb640 1 -- 10.20.193.16:0/3816580839 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc09c087630 con 0x7fc0ac074dc0 2026-02-17T05:56:49.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.403+0000 7fc099ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:56:49.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.403+0000 7fc099ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.403 INFO:teuthology.orchestra.run.trial016.stdout:98784247814 2026-02-17T05:56:49.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 1 -- 10.20.193.16:0/525399189 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579000c790 con 0x7f579c15f2c0 2026-02-17T05:56:49.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:49.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 10 monclient: dump: 2026-02-17T05:56:49.405 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.405 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.405 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.405 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.405 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.405 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 1 -- 10.20.193.16:0/525399189 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5790010070 con 0x7f579c15f2c0 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 1 -- 10.20.193.16:0/525399189 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579000cb90 con 0x7f579c15f2c0 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f5798ff9640 10 monclient: dump: 2026-02-17T05:56:49.406 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f579a7fc640 4 set_mon_vals no callback set 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 5 monclient: authenticate success, global_id 4561 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 10 monclient: _renew_subs 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 1 -- 10.20.193.16:0/525399189 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579c14d2a0 con 0x7f579c15f2c0 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 10 client.4561.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:49.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 10 monclient: _renew_subs 2026-02-17T05:56:49.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 1 -- 10.20.193.16:0/525399189 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579c1f4800 con 0x7f579c15f2c0 2026-02-17T05:56:49.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 1 librados: init done 2026-02-17T05:56:49.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.404+0000 7f57a2734640 1 -- 10.20.193.16:0/525399189 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f579c0d01b0 con 0x7f579c15f2c0 2026-02-17T05:56:49.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 10 librados: watch_flush enter 2026-02-17T05:56:49.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 10 librados: watch_flush exit 2026-02-17T05:56:49.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 20 client.4552.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 10 client.4552.objecter successfully canceled tick 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 -- 10.20.193.16:0/3816580839 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc07c0791c0 msgr2=0x7fc07c07b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc07c0791c0 0x7fc07c07b680 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0002770 tx=0x7fc0a0007ad0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 10 monclient: shutdown 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 -- 10.20.193.16:0/3816580839 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac074dc0 msgr2=0x7fc0ac1f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac074dc0 0x7fc0ac1f4330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009a70 tx=0x7fc09c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 -- 10.20.193.16:0/3816580839 shutdown_connections 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc07c0791c0 0x7fc07c07b680 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc0ac1561e0 0x7fc0ac151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0ac155e00 0x7fc0ac0cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 --2- 10.20.193.16:0/3816580839 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0ac074dc0 0x7fc0ac1f4330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 -- 10.20.193.16:0/3816580839 >> 10.20.193.16:0/3816580839 conn(0x7fc0ac0ce6c0 msgr2=0x7fc0ac154960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 -- 10.20.193.16:0/3816580839 shutdown_connections 2026-02-17T05:56:49.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 -- 10.20.193.16:0/3816580839 wait complete. 2026-02-17T05:56:49.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.405+0000 7fc0b3553640 1 librados: shutdown 2026-02-17T05:56:49.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247814 for osd.1 2026-02-17T05:56:49.414 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:49.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.428+0000 7fd3067fc640 1 -- 10.20.193.16:0/3412598768 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd30c00d360 con 0x7fd318155e00 2026-02-17T05:56:49.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.428+0000 7fd3067fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.428+0000 7fd3067fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.430+0000 7f0172ffd640 1 -- 10.20.193.16:0/327461132 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 14}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f017c0629a0 con 0x7f018c0c7d40 2026-02-17T05:56:49.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.430+0000 7f0172ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-17T05:56:49.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.430+0000 7f0172ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.431 INFO:teuthology.orchestra.run.trial016.stdout:107374182409 2026-02-17T05:56:49.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 10 librados: watch_flush enter 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 10 librados: watch_flush exit 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 20 client.4467.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 10 client.4467.objecter successfully canceled tick 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 1 -- 10.20.193.16:0/327461132 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f015c079140 msgr2=0x7f015c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f015c079140 0x7f015c07b600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f018c073130 tx=0x7f0180031000 comp rx=0 tx=0).stop 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 10 monclient: shutdown 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 1 -- 10.20.193.16:0/327461132 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c0c7d40 msgr2=0x7f018c13eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c0c7d40 0x7f018c13eec0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f017c00b570 tx=0x7f017c00ba40 comp rx=0 tx=0).stop 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 1 -- 10.20.193.16:0/327461132 shutdown_connections 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f015c079140 0x7f015c07b600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f018c1538d0 0x7f018c1547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f018c15b530 0x7f018c154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 1 --2- 10.20.193.16:0/327461132 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f018c0c7d40 0x7f018c13eec0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.432+0000 7f01947ca640 1 -- 10.20.193.16:0/327461132 >> 10.20.193.16:0/327461132 conn(0x7f018c073f00 msgr2=0x7f018c151e00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.433+0000 7f01947ca640 1 -- 10.20.193.16:0/327461132 shutdown_connections 2026-02-17T05:56:49.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.433+0000 7f01947ca640 1 -- 10.20.193.16:0/327461132 wait complete. 2026-02-17T05:56:49.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.433+0000 7f01947ca640 1 librados: shutdown 2026-02-17T05:56:49.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182409 got 107374182409 for osd.14 2026-02-17T05:56:49.443 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:49.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.442+0000 7f671c8a9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-17T05:56:49.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.442+0000 7f671c8a9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-17T05:56:49.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.442+0000 7f671c8a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.442+0000 7f671c8a9640 1 -- 10.20.193.16:0/2317619249 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f6714159730 con 0x7f6714155e00 2026-02-17T05:56:49.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.442+0000 7f695deb7640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-17T05:56:49.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.442+0000 7f695deb7640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-17T05:56:49.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.442+0000 7f695deb7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.442+0000 7f695deb7640 1 -- 10.20.193.16:0/1644113027 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f69580c8c50 con 0x7f69580c7fe0 2026-02-17T05:56:49.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.501+0000 7f6933fff640 1 -- 10.20.193.16:0/1644113027 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f69480626d0 con 0x7f69580c7fe0 2026-02-17T05:56:49.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.501+0000 7f6933fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-17T05:56:49.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.501+0000 7f6933fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.502 INFO:teuthology.orchestra.run.trial016.stdout:115964117001 2026-02-17T05:56:49.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 10 librados: watch_flush enter 2026-02-17T05:56:49.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 10 librados: watch_flush exit 2026-02-17T05:56:49.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 20 client.4607.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 10 client.4607.objecter successfully canceled tick 2026-02-17T05:56:49.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 -- 10.20.193.16:0/1644113027 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6928079190 msgr2=0x7f692807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6928079190 0x7f692807b650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f69580c5b80 tx=0x7f6944031000 comp rx=0 tx=0).stop 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 10 monclient: shutdown 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 -- 10.20.193.16:0/1644113027 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580c7fe0 msgr2=0x7f695806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580c7fe0 0x7f695806de80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f694800b500 tx=0x7f694800b9d0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 -- 10.20.193.16:0/1644113027 shutdown_connections 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6928079190 0x7f692807b650 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69581536f0 0x7f69581f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69580c7fe0 0x7f695806de80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 --2- 10.20.193.16:0/1644113027 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f69580700f0 0x7f6958140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 -- 10.20.193.16:0/1644113027 >> 10.20.193.16:0/1644113027 conn(0x7f69580cd770 msgr2=0x7f6958146b60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 -- 10.20.193.16:0/1644113027 shutdown_connections 2026-02-17T05:56:49.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 -- 10.20.193.16:0/1644113027 wait complete. 2026-02-17T05:56:49.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.503+0000 7f695deb7640 1 librados: shutdown 2026-02-17T05:56:49.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.508+0000 7f589e7fc640 1 -- 10.20.193.16:0/1306035717 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f58a4002b60 con 0x7f58b00cc4f0 2026-02-17T05:56:49.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.508+0000 7f589e7fc640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5880079140 0x7f588007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.508+0000 7f58b5b99640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5880079140 0x7f588007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.508+0000 7f58b5b99640 10 monclient: get_auth_request con 0x7f5880079140 auth_method 0 2026-02-17T05:56:49.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.509+0000 7f589e7fc640 10 client.4610.objecter ms_handle_connect 0x7f5880079140 2026-02-17T05:56:49.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.509+0000 7f58b5b99640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5880079140 0x7f588007b600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58b0150770 tx=0x7f58a0006cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.510+0000 7f589e7fc640 1 -- 10.20.193.16:0/1306035717 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f58a405e2c0 con 0x7f58b00cc4f0 2026-02-17T05:56:49.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.510+0000 7f589e7fc640 10 client.4610.objecter ms_dispatch2 0x7f58b0000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.510+0000 7f589e7fc640 3 client.4610.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.510+0000 7f589e7fc640 3 client.4610.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.510+0000 7f589e7fc640 20 client.4610.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.510+0000 7f5798ff9640 1 -- 10.20.193.16:0/525399189 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f579001d030 con 0x7f579c15f2c0 2026-02-17T05:56:49.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.510+0000 7f5798ff9640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f576c079190 0x7f576c07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.510+0000 7f579bfff640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f576c079190 0x7f576c07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.510+0000 7f579bfff640 10 monclient: get_auth_request con 0x7f576c079190 auth_method 0 2026-02-17T05:56:49.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.510+0000 7f579bfff640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f576c079190 0x7f576c07b650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5788039000 tx=0x7f5788031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.511+0000 7f5798ff9640 10 client.4561.objecter ms_handle_connect 0x7f576c079190 2026-02-17T05:56:49.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964117001 got 115964117001 for osd.7 2026-02-17T05:56:49.514 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:49.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.512+0000 7f5798ff9640 1 -- 10.20.193.16:0/525399189 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f5790012e00 con 0x7f579c15f2c0 2026-02-17T05:56:49.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.512+0000 7f5798ff9640 10 client.4561.objecter ms_dispatch2 0x7f579c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.512+0000 7f5798ff9640 3 client.4561.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.512+0000 7f5798ff9640 3 client.4561.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.512+0000 7f5798ff9640 20 client.4561.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.515+0000 7fd31fe4a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-17T05:56:49.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.515+0000 7fd31fe4a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-17T05:56:49.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.515+0000 7fd31fe4a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.515+0000 7fd31fe4a640 1 -- 10.20.193.16:0/3412598768 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fd318150a40 con 0x7fd318155e00 2026-02-17T05:56:49.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.518+0000 7f589e7fc640 1 -- 10.20.193.16:0/1306035717 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f58b00c81e0 con 0x7f58b00cc4f0 2026-02-17T05:56:49.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.518+0000 7f589e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.518+0000 7f589e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.518+0000 7fd3067fc640 1 -- 10.20.193.16:0/3412598768 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd30c062b00 con 0x7fd318155e00 2026-02-17T05:56:49.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.518+0000 7fd3067fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-17T05:56:49.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.518+0000 7fd3067fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.519 INFO:teuthology.orchestra.run.trial016.stdout:107374182405 2026-02-17T05:56:49.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.519+0000 7f5798ff9640 1 -- 10.20.193.16:0/525399189 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5790024080 con 0x7f579c15f2c0 2026-02-17T05:56:49.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.519+0000 7f5798ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.519+0000 7f5798ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 10 librados: watch_flush enter 2026-02-17T05:56:49.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 10 librados: watch_flush exit 2026-02-17T05:56:49.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 20 client.4470.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 10 client.4470.objecter successfully canceled tick 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 -- 10.20.193.16:0/3412598768 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd2e8079190 msgr2=0x7fd2e807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd2e8079190 0x7fd2e807b650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3180693f0 tx=0x7fd308031000 comp rx=0 tx=0).stop 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 10 monclient: shutdown 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 -- 10.20.193.16:0/3412598768 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd318155e00 msgr2=0x7fd31814ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd318155e00 0x7fd31814ba00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00b4d0 tx=0x7fd30c00b9a0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 -- 10.20.193.16:0/3412598768 shutdown_connections 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd2e8079190 0x7fd2e807b650 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd3181561e0 0x7fd31814c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd318155e00 0x7fd31814ba00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 --2- 10.20.193.16:0/3412598768 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd318074dc0 0x7fd3181f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 -- 10.20.193.16:0/3412598768 >> 10.20.193.16:0/3412598768 conn(0x7fd3180ce6c0 msgr2=0x7fd318153620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 -- 10.20.193.16:0/3412598768 shutdown_connections 2026-02-17T05:56:49.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 -- 10.20.193.16:0/3412598768 wait complete. 2026-02-17T05:56:49.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.520+0000 7fd31fe4a640 1 librados: shutdown 2026-02-17T05:56:49.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.521+0000 7f66feffd640 1 -- 10.20.193.16:0/2317619249 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f67000880c0 con 0x7f6714155e00 2026-02-17T05:56:49.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.521+0000 7f66feffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-17T05:56:49.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.521+0000 7f66feffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.524 INFO:teuthology.orchestra.run.trial016.stdout:90194313223 2026-02-17T05:56:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 10 librados: watch_flush enter 2026-02-17T05:56:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 10 librados: watch_flush exit 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 20 client.4555.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 10 client.4555.objecter successfully canceled tick 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 -- 10.20.193.16:0/2317619249 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f66e4079190 msgr2=0x7f66e407b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f66e4079190 0x7f66e407b650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f670c0098a0 tx=0x7f670c006d90 comp rx=0 tx=0).stop 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 10 monclient: shutdown 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 -- 10.20.193.16:0/2317619249 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6714155e00 msgr2=0x7f67141f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6714155e00 0x7f67141f4070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f670002f3e0 tx=0x7f67000047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 -- 10.20.193.16:0/2317619249 shutdown_connections 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f66e4079190 0x7f66e407b650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6714155e00 0x7f67141f4070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67141561e0 0x7f67140c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 --2- 10.20.193.16:0/2317619249 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6714074dc0 0x7f67141420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 -- 10.20.193.16:0/2317619249 >> 10.20.193.16:0/2317619249 conn(0x7f67140ce6c0 msgr2=0x7f671406de10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 -- 10.20.193.16:0/2317619249 shutdown_connections 2026-02-17T05:56:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 -- 10.20.193.16:0/2317619249 wait complete. 2026-02-17T05:56:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.524+0000 7f671c8a9640 1 librados: shutdown 2026-02-17T05:56:49.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182405 got 107374182405 for osd.8 2026-02-17T05:56:49.530 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:49.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313223 got 90194313223 for osd.10 2026-02-17T05:56:49.533 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:49.605 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.605+0000 7f57a2734640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:56:49.606 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.605+0000 7f57a2734640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:56:49.606 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.605+0000 7f57a2734640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.606 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.605+0000 7f57a2734640 1 -- 10.20.193.16:0/525399189 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f579c1f4ae0 con 0x7f579c15f2c0 2026-02-17T05:56:49.606 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.605+0000 7f58b6b9b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-17T05:56:49.606 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.605+0000 7f58b6b9b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-17T05:56:49.607 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.605+0000 7f58b6b9b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.607 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.605+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1306035717 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f58b0050a10 con 0x7f58b00cc4f0 2026-02-17T05:56:49.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.607+0000 7f589e7fc640 1 -- 10.20.193.16:0/1306035717 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58b0050a10 con 0x7f58b00cc4f0 2026-02-17T05:56:49.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.607+0000 7f589e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-17T05:56:49.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.607+0000 7f589e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.608 INFO:teuthology.orchestra.run.trial016.stdout:98784247813 2026-02-17T05:56:49.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.608+0000 7f5798ff9640 1 -- 10.20.193.16:0/525399189 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5790062330 con 0x7f579c15f2c0 2026-02-17T05:56:49.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.608+0000 7f5798ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:56:49.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.608+0000 7f5798ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:49.609 INFO:teuthology.orchestra.run.trial016.stdout:107374182405 2026-02-17T05:56:49.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 10 librados: watch_flush enter 2026-02-17T05:56:49.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 10 librados: watch_flush exit 2026-02-17T05:56:49.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 20 client.4610.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 10 client.4610.objecter successfully canceled tick 2026-02-17T05:56:49.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1306035717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5880079140 msgr2=0x7f588007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5880079140 0x7f588007b600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58b0150770 tx=0x7f58a0006cf0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 10 monclient: shutdown 2026-02-17T05:56:49.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1306035717 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 msgr2=0x7f58b014e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 0x7f58b014e4f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f58a400b570 tx=0x7f58a400ba40 comp rx=0 tx=0).stop 2026-02-17T05:56:49.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 10 librados: watch_flush enter 2026-02-17T05:56:49.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 10 librados: watch_flush exit 2026-02-17T05:56:49.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1306035717 shutdown_connections 2026-02-17T05:56:49.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5880079140 0x7f588007b600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f58b015f5f0 0x7f58b0148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f58b00cc4f0 0x7f58b014e4f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 --2- 10.20.193.16:0/1306035717 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f58b00cc0c0 0x7f58b014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 20 client.4561.objecter shutdown clearing up homeless session... 2026-02-17T05:56:49.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1306035717 >> 10.20.193.16:0/1306035717 conn(0x7f58b00d1b60 msgr2=0x7f58b00d1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1306035717 shutdown_connections 2026-02-17T05:56:49.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 10 client.4561.objecter successfully canceled tick 2026-02-17T05:56:49.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 -- 10.20.193.16:0/1306035717 wait complete. 2026-02-17T05:56:49.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f58b6b9b640 1 librados: shutdown 2026-02-17T05:56:49.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 -- 10.20.193.16:0/525399189 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f576c079190 msgr2=0x7f576c07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f576c079190 0x7f576c07b650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5788039000 tx=0x7f5788031000 comp rx=0 tx=0).stop 2026-02-17T05:56:49.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 10 monclient: shutdown 2026-02-17T05:56:49.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 -- 10.20.193.16:0/525399189 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c15f2c0 msgr2=0x7f579c14dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c15f2c0 0x7f579c14dd40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f579000b970 tx=0x7f579000be40 comp rx=0 tx=0).stop 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 -- 10.20.193.16:0/525399189 shutdown_connections 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f576c079190 0x7f576c07b650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f579c15f2c0 0x7f579c14dd40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f579c156180 0x7f579c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 --2- 10.20.193.16:0/525399189 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f579c155da0 0x7f579c1f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 -- 10.20.193.16:0/525399189 >> 10.20.193.16:0/525399189 conn(0x7f579c0cf600 msgr2=0x7f579c0ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 -- 10.20.193.16:0/525399189 shutdown_connections 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 -- 10.20.193.16:0/525399189 wait complete. 2026-02-17T05:56:49.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.609+0000 7f57a2734640 1 librados: shutdown 2026-02-17T05:56:49.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247813 for osd.9 2026-02-17T05:56:49.620 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:49.620 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182405 got 107374182405 for osd.4 2026-02-17T05:56:49.620 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:49.712 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-17T05:56:49.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.781+0000 7f4ea44bf640 10 monclient: get_monmap_and_config 2026-02-17T05:56:49.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.781+0000 7f4ea44bf640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.781+0000 7f4ea44bf640 10 monclient: monmap: 2026-02-17T05:56:49.782 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.782 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.783 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:49.782230+0000 2026-02-17T05:56:49.783 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:49.782230+0000 2026-02-17T05:56:49.783 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.783 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.783 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:49.783 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:49.783 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:49.783 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:49.783 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:49.783 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:49.784 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.781+0000 7f4ea44bf640 1 Processor -- start 2026-02-17T05:56:49.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.781+0000 7f4ea44bf640 1 -- start start 2026-02-17T05:56:49.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.781+0000 7f4ea44bf640 10 monclient: init 2026-02-17T05:56:49.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 10 monclient: _add_conns ranks=[4,5,3,2,1,0] 2026-02-17T05:56:49.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f4e9c0700f0 0x7f4e9c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 10 monclient(hunting): picked mon.noname-b con 0x7f4e9c0700f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:49.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f4e9c0c7fe0 0x7f4e9c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 10 monclient(hunting): picked mon.noname-c con 0x7f4e9c0c7fe0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:49.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f4e9c1536f0 0x7f4e9c153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 10 monclient(hunting): picked mon.noname-a con 0x7f4e9c1536f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:49.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4e9c0c8970 con 0x7f4e9c1536f0 2026-02-17T05:56:49.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f4e9c0c4820 con 0x7f4e9c0700f0 2026-02-17T05:56:49.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f4e9c0c4b00 con 0x7f4e9c0c7fe0 2026-02-17T05:56:49.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea44bf640 10 monclient(hunting): authenticate will time out at 1062.351223s 2026-02-17T05:56:49.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea2a35640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f4e9c1536f0 0x7f4e9c153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea2a35640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f4e9c1536f0 0x7f4e9c153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:60206/0 (socket says 10.20.193.16:60206) 2026-02-17T05:56:49.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea2a35640 1 -- 10.20.193.16:0/1525637223 learned_addr learned my addr 10.20.193.16:0/1525637223 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea2a35640 10 monclient(hunting): get_auth_request con 0x7f4e9c1536f0 auth_method 0 2026-02-17T05:56:49.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea2a35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea2a35640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.782+0000 7f4ea2a35640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.783+0000 7f4ea2a35640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.783+0000 7f4ea2a35640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.783+0000 7f4ea2a35640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.784+0000 7f4ea2a35640 10 monclient(hunting): handle_auth_done global_id 4473 payload 274 2026-02-17T05:56:49.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.784+0000 7f4ea2a35640 1 -- 10.20.193.16:0/1525637223 >> v2:10.20.193.148:3300/0 conn(0x7f4e9c0c7fe0 msgr2=0x7f4e9c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.784+0000 7f4ea2a35640 1 --2- 10.20.193.16:0/1525637223 >> v2:10.20.193.148:3300/0 conn(0x7f4e9c0c7fe0 0x7f4e9c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.784+0000 7f4ea2a35640 1 -- 10.20.193.16:0/1525637223 >> v2:10.20.193.128:3300/0 conn(0x7f4e9c0700f0 msgr2=0x7f4e9c0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.784+0000 7f4ea2a35640 1 --2- 10.20.193.16:0/1525637223 >> v2:10.20.193.128:3300/0 conn(0x7f4e9c0700f0 0x7f4e9c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.784+0000 7f4ea2a35640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.784+0000 7f4ea2a35640 1 monclient: found mon.noname-a 2026-02-17T05:56:49.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.784+0000 7f4ea2a35640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.784+0000 7f4ea2a35640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:49.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.784+0000 7f4ea2a35640 1 -- 10.20.193.16:0/1525637223 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c0c4de0 con 0x7f4e9c1536f0 2026-02-17T05:56:49.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.785+0000 7f4ea2a35640 1 --2- 10.20.193.16:0/1525637223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4e9c1536f0 0x7f4e9c153b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c070b90 tx=0x7f4e9802f200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678cb24aede99d24 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 1 -- 10.20.193.16:0/1525637223 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9802fc70 con 0x7f4e9c1536f0 2026-02-17T05:56:49.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:49.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 10 monclient: dump: 2026-02-17T05:56:49.800 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.800 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.800 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.800 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.801 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.801 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.801 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.801 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.801 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.801 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 1 -- 10.20.193.16:0/1525637223 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9802fe10 con 0x7f4e9c1536f0 2026-02-17T05:56:49.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 5 monclient: authenticate success, global_id 4473 2026-02-17T05:56:49.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:49.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 1 -- 10.20.193.16:0/1525637223 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98037610 con 0x7f4e9c1536f0 2026-02-17T05:56:49.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea1232640 10 monclient: dump: 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.803 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 4 set_mon_vals no callback set 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 10 monclient: shutdown 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 1 -- 10.20.193.16:0/1525637223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4e9c1536f0 msgr2=0x7f4e9c153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 1 --2- 10.20.193.16:0/1525637223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4e9c1536f0 0x7f4e9c153b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c070b90 tx=0x7f4e9802f200 comp rx=0 tx=0).stop 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 1 -- 10.20.193.16:0/1525637223 shutdown_connections 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 1 --2- 10.20.193.16:0/1525637223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4e9c1536f0 0x7f4e9c153b00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 1 --2- 10.20.193.16:0/1525637223 >> v2:10.20.193.148:3300/0 conn(0x7f4e9c0c7fe0 0x7f4e9c0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 1 --2- 10.20.193.16:0/1525637223 >> v2:10.20.193.128:3300/0 conn(0x7f4e9c0700f0 0x7f4e9c0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 1 -- 10.20.193.16:0/1525637223 >> 10.20.193.16:0/1525637223 conn(0x7f4e9c0cd770 msgr2=0x7f4e9c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 1 -- 10.20.193.16:0/1525637223 shutdown_connections 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.793+0000 7f4ea44bf640 1 -- 10.20.193.16:0/1525637223 wait complete. 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.794+0000 7f4ea44bf640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.794+0000 7f4ea44bf640 10 monclient: monmap: 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.805 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.794+0000 7f4ea44bf640 1 librados: starting msgr at 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.794+0000 7f4ea44bf640 1 librados: starting objecter 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.794+0000 7f4ea44bf640 1 Processor -- start 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.794+0000 7f4ea44bf640 1 -- start start 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.794+0000 7f4ea44bf640 1 librados: setting wanted keys 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.794+0000 7f4ea44bf640 1 librados: calling monclient init 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.794+0000 7f4ea44bf640 10 monclient: init 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4e9c0700f0 0x7f4e9c150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 10 monclient(hunting): picked mon.noname-b con 0x7f4e9c0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4e9c0c7fe0 0x7f4e9c1f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 10 monclient(hunting): picked mon.noname-c con 0x7f4e9c0c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4e9c1536f0 0x7f4e9c1f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 10 monclient(hunting): picked mon.noname-a con 0x7f4e9c1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c0c5680 con 0x7f4e9c1536f0 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c0c53a0 con 0x7f4e9c0700f0 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c154040 con 0x7f4e9c0c7fe0 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea44bf640 10 monclient(hunting): authenticate will time out at 1062.364484s 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.795+0000 7f4ea2234640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4e9c0700f0 0x7f4e9c150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea2234640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4e9c0700f0 0x7f4e9c150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50726/0 (socket says 10.20.193.16:50726) 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea2234640 1 -- 10.20.193.16:0/366864926 learned_addr learned my addr 10.20.193.16:0/366864926 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea2234640 10 monclient(hunting): get_auth_request con 0x7f4e9c0700f0 auth_method 0 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea2234640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea2234640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea2234640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea1a33640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4e9c0c7fe0 0x7f4e9c1f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea1a33640 10 monclient(hunting): get_auth_request con 0x7f4e9c0c7fe0 auth_method 0 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea1a33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea1a33640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea1a33640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea2234640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea2234640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea2234640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea1a33640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea1a33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.796+0000 7f4ea1a33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.797+0000 7f4ea2234640 10 monclient(hunting): handle_auth_done global_id 4564 payload 824 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.797+0000 7f4ea2234640 1 -- 10.20.193.16:0/366864926 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4e9c0c7fe0 msgr2=0x7f4e9c1f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.797+0000 7f4ea2234640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4e9c0c7fe0 0x7f4e9c1f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.797+0000 7f4ea2234640 1 -- 10.20.193.16:0/366864926 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4e9c1536f0 msgr2=0x7f4e9c1f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.797+0000 7f4ea2234640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4e9c1536f0 0x7f4e9c1f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.797+0000 7f4ea2234640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.797+0000 7f4ea2234640 1 monclient: found mon.noname-b 2026-02-17T05:56:49.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.797+0000 7f4ea2234640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.797+0000 7f4ea2234640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.797+0000 7f4ea2234640 1 -- 10.20.193.16:0/366864926 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c1f4820 con 0x7f4e9c0700f0 2026-02-17T05:56:49.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.798+0000 7f4ea2234640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4e9c0700f0 0x7f4e9c150020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00b500 tx=0x7f4e8c00b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.798+0000 7f4e82ffd640 10 client.?.objecter ms_handle_connect 0x7f4e9c0700f0 2026-02-17T05:56:49.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.798+0000 7f4e82ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:49.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 1 -- 10.20.193.16:0/366864926 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c004480 con 0x7f4e9c0700f0 2026-02-17T05:56:49.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 10 monclient: dump: 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.813 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 5 monclient: authenticate success, global_id 4564 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 10 monclient: _renew_subs 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 1 -- 10.20.193.16:0/366864926 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9c06ecc0 con 0x7f4e9c0700f0 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 10 client.4564.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 10 monclient: _renew_subs 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 1 -- 10.20.193.16:0/366864926 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9c1f4ac0 con 0x7f4e9c0700f0 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 1 librados: init done 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 1 -- 10.20.193.16:0/366864926 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e8c014070 con 0x7f4e9c0700f0 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 1 -- 10.20.193.16:0/366864926 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0028e0 con 0x7f4e9c0700f0 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea0a31640 4 set_mon_vals no callback set 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4e82ffd640 10 monclient: dump: 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.815 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.816 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.816 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.816 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.816 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.816 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.800+0000 7f4ea44bf640 1 -- 10.20.193.16:0/366864926 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9c074c00 con 0x7f4e9c0700f0 2026-02-17T05:56:49.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.812+0000 7f4e82ffd640 1 -- 10.20.193.16:0/366864926 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f4e8c002a80 con 0x7f4e9c0700f0 2026-02-17T05:56:49.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.812+0000 7f4e82ffd640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4e6c079170 0x7f4e6c07b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.812+0000 7f4ea1a33640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4e6c079170 0x7f4e6c07b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.812+0000 7f4ea1a33640 10 monclient: get_auth_request con 0x7f4e6c079170 auth_method 0 2026-02-17T05:56:49.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.813+0000 7f4ea1a33640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4e6c079170 0x7f4e6c07b630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0c67c0 tx=0x7f4e90009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.813+0000 7f4e82ffd640 10 client.4564.objecter ms_handle_connect 0x7f4e6c079170 2026-02-17T05:56:49.838 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-17T05:56:49.906 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-17T05:56:49.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.906+0000 7fc2d29e7640 10 monclient: get_monmap_and_config 2026-02-17T05:56:49.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.906+0000 7fc2d29e7640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.906+0000 7fc2d29e7640 10 monclient: monmap: 2026-02-17T05:56:49.909 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.909 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.909 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:49.907099+0000 2026-02-17T05:56:49.909 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:49.907099+0000 2026-02-17T05:56:49.909 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.909 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.906+0000 7fc2d29e7640 1 Processor -- start 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.906+0000 7fc2d29e7640 1 -- start start 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.906+0000 7fc2d29e7640 10 monclient: init 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 10 monclient: _add_conns ranks=[1,5,4,0,2,3] 2026-02-17T05:56:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc2cc0700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fc2cc0c86f0 0x7fc2cc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 10 monclient(hunting): picked mon.noname-c con 0x7fc2cc0c86f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fc2cc1536f0 0x7fc2cc153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 10 monclient(hunting): picked mon.noname-b con 0x7fc2cc1536f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2cc0c4820 con 0x7fc2cc0700f0 2026-02-17T05:56:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fc2cc0c4b00 con 0x7fc2cc1536f0 2026-02-17T05:56:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fc2cc0c4de0 con 0x7fc2cc0c86f0 2026-02-17T05:56:49.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2d29e7640 10 monclient(hunting): authenticate will time out at 1062.476127s 2026-02-17T05:56:49.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7f4e82ffd640 1 -- 10.20.193.16:0/366864926 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f4e8c0979d0 con 0x7f4e9c0700f0 2026-02-17T05:56:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7f4e82ffd640 10 client.4564.objecter ms_dispatch2 0x7f4e9c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7f4e82ffd640 3 client.4564.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7f4e82ffd640 3 client.4564.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7f4e82ffd640 20 client.4564.objecter dump_active .. 0 homeless 2026-02-17T05:56:49.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2cb7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fc2cc0c86f0 0x7fc2cc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2cb7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fc2cc0c86f0 0x7fc2cc0704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57468/0 (socket says 10.20.193.16:57468) 2026-02-17T05:56:49.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2cb7fe640 1 -- 10.20.193.16:0/3120313755 learned_addr learned my addr 10.20.193.16:0/3120313755 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2cb7fe640 10 monclient(hunting): get_auth_request con 0x7fc2cc0c86f0 auth_method 0 2026-02-17T05:56:49.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2cb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.907+0000 7fc2cb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 10 monclient(hunting): handle_auth_done global_id 4616 payload 274 2026-02-17T05:56:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 1 -- 10.20.193.16:0/3120313755 >> v2:10.20.193.128:3300/0 conn(0x7fc2cc1536f0 msgr2=0x7fc2cc153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 1 --2- 10.20.193.16:0/3120313755 >> v2:10.20.193.128:3300/0 conn(0x7fc2cc1536f0 0x7fc2cc153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 1 -- 10.20.193.16:0/3120313755 >> v1:10.20.193.128:6789/0 conn(0x7fc2cc0700f0 legacy=0x7fc2cc0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 1 monclient: found mon.noname-c 2026-02-17T05:56:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.908+0000 7fc2cb7fe640 1 -- 10.20.193.16:0/3120313755 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc0c50c0 con 0x7fc2cc0c86f0 2026-02-17T05:56:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.909+0000 7fc2cb7fe640 1 --2- 10.20.193.16:0/3120313755 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc0c86f0 0x7fc2cc0704d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0c5b80 tx=0x7fc2bc02ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae57c6294fc94485 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 1 -- 10.20.193.16:0/3120313755 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc03c070 con 0x7fc2cc0c86f0 2026-02-17T05:56:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 10 monclient: dump: 2026-02-17T05:56:49.921 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.921 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.921 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.921 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.921 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.921 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.922 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.922 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.922 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.922 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 1 -- 10.20.193.16:0/3120313755 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2bc02fa90 con 0x7fc2cc0c86f0 2026-02-17T05:56:49.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 5 monclient: authenticate success, global_id 4616 2026-02-17T05:56:49.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 1 -- 10.20.193.16:0/3120313755 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc02fe60 con 0x7fc2cc0c86f0 2026-02-17T05:56:49.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 4 set_mon_vals no callback set 2026-02-17T05:56:49.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:49.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2caffd640 10 monclient: dump: 2026-02-17T05:56:49.923 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.923 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.923 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.923 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.924 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.924 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.924 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.924 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.924 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.924 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 10 monclient: shutdown 2026-02-17T05:56:49.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 1 -- 10.20.193.16:0/3120313755 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc0c86f0 msgr2=0x7fc2cc0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 1 --2- 10.20.193.16:0/3120313755 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc0c86f0 0x7fc2cc0704d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0c5b80 tx=0x7fc2bc02ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:49.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 1 -- 10.20.193.16:0/3120313755 shutdown_connections 2026-02-17T05:56:49.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 1 --2- 10.20.193.16:0/3120313755 >> v2:10.20.193.128:3300/0 conn(0x7fc2cc1536f0 0x7fc2cc153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 1 --2- 10.20.193.16:0/3120313755 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc0c86f0 0x7fc2cc0704d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 1 -- 10.20.193.16:0/3120313755 >> 10.20.193.16:0/3120313755 conn(0x7fc2cc0cd770 msgr2=0x7fc2cc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 1 -- 10.20.193.16:0/3120313755 shutdown_connections 2026-02-17T05:56:49.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.911+0000 7fc2d29e7640 1 -- 10.20.193.16:0/3120313755 wait complete. 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.912+0000 7fc2d29e7640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.912+0000 7fc2d29e7640 10 monclient: monmap: 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.926 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.912+0000 7fc2d29e7640 1 librados: starting msgr at 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.912+0000 7fc2d29e7640 1 librados: starting objecter 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.912+0000 7fc2d29e7640 1 Processor -- start 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.912+0000 7fc2d29e7640 1 -- start start 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.912+0000 7fc2d29e7640 1 librados: setting wanted keys 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.912+0000 7fc2d29e7640 1 librados: calling monclient init 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.912+0000 7fc2d29e7640 10 monclient: init 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc2cc0700f0 0x7fc2cc1425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 10 monclient(hunting): picked mon.noname-a con 0x7fc2cc0700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc2cc0c86f0 0x7fc2cc1f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 10 monclient(hunting): picked mon.noname-b con 0x7fc2cc0c86f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc1536f0 0x7fc2cc072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 10 monclient(hunting): picked mon.noname-c con 0x7fc2cc1536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc154060 con 0x7fc2cc0700f0 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc0c5680 con 0x7fc2cc0c86f0 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc074c00 con 0x7fc2cc1536f0 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d29e7640 10 monclient(hunting): authenticate will time out at 1062.482369s 2026-02-17T05:56:49.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d0f5d640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc1536f0 0x7fc2cc072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7f4e82ffd640 1 -- 10.20.193.16:0/366864926 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4e8c062650 con 0x7f4e9c0700f0 2026-02-17T05:56:49.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7f4e82ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7f4e82ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:49.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2cbfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc2cc0700f0 0x7fc2cc1425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d0f5d640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc1536f0 0x7fc2cc072e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:57472/0 (socket says 10.20.193.16:57472) 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d0f5d640 1 -- 10.20.193.16:0/4095534177 learned_addr learned my addr 10.20.193.16:0/4095534177 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d0f5d640 10 monclient(hunting): get_auth_request con 0x7fc2cc1536f0 auth_method 0 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d0f5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d0f5d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.913+0000 7fc2d0f5d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.914+0000 7fc2cbfff640 10 monclient(hunting): get_auth_request con 0x7fc2cc0700f0 auth_method 0 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.914+0000 7fc2cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.914+0000 7fc2cbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.914+0000 7fc2cbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.914+0000 7fc2d0f5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.914+0000 7fc2d0f5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.914+0000 7fc2d0f5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.914+0000 7fc2cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.914+0000 7fc2cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.914+0000 7fc2cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.915+0000 7fc2d0f5d640 10 monclient(hunting): handle_auth_done global_id 4619 payload 824 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.915+0000 7fc2d0f5d640 1 -- 10.20.193.16:0/4095534177 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc2cc0c86f0 msgr2=0x7fc2cc1f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.915+0000 7fc2d0f5d640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc2cc0c86f0 0x7fc2cc1f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.915+0000 7fc2d0f5d640 1 -- 10.20.193.16:0/4095534177 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc2cc0700f0 msgr2=0x7fc2cc1425c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.915+0000 7fc2d0f5d640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc2cc0700f0 0x7fc2cc1425c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.915+0000 7fc2d0f5d640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.915+0000 7fc2d0f5d640 1 monclient: found mon.noname-c 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.915+0000 7fc2d0f5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.915+0000 7fc2d0f5d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.915+0000 7fc2d0f5d640 1 -- 10.20.193.16:0/4095534177 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc0c84d0 con 0x7fc2cc1536f0 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.916+0000 7fc2d0f5d640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc1536f0 0x7fc2cc072e50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000b9a0 tx=0x7fc2c000be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.916+0000 7fc2c8ff9640 10 client.?.objecter ms_handle_connect 0x7fc2cc1536f0 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.916+0000 7fc2c8ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2c8ff9640 1 -- 10.20.193.16:0/4095534177 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000c790 con 0x7fc2cc1536f0 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2c8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2c8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2c8ff9640 10 monclient: dump: 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.935 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2c8ff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2c8ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2c8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 5 monclient: authenticate success, global_id 4619 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 10 monclient: _renew_subs 2026-02-17T05:56:49.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 1 -- 10.20.193.16:0/4095534177 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc1f1d90 con 0x7fc2cc1536f0 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 10 client.4619.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 10 monclient: _renew_subs 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 1 -- 10.20.193.16:0/4095534177 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc1f2350 con 0x7fc2cc1536f0 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 1 librados: init done 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.917+0000 7fc2d29e7640 1 -- 10.20.193.16:0/4095534177 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2cc0704d0 con 0x7fc2cc1536f0 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.918+0000 7fc2c8ff9640 1 -- 10.20.193.16:0/4095534177 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c0010070 con 0x7fc2cc1536f0 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.918+0000 7fc2c8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.918+0000 7fc2c8ff9640 1 -- 10.20.193.16:0/4095534177 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000cb30 con 0x7fc2cc1536f0 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.918+0000 7fc2c8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.918+0000 7fc2c8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.918+0000 7fc2c8ff9640 10 monclient: dump: 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.938 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.939 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.918+0000 7fc2ca7fc640 4 set_mon_vals no callback set 2026-02-17T05:56:49.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.975+0000 7febc9e66640 10 monclient: get_monmap_and_config 2026-02-17T05:56:49.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.975+0000 7febc9e66640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.975+0000 7febc9e66640 10 monclient: monmap: 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:49.976600+0000 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:49.976600+0000 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:49.977 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:49.978 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:49.978 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 1 Processor -- start 2026-02-17T05:56:49.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 1 -- start start 2026-02-17T05:56:49.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 10 monclient: init 2026-02-17T05:56:49.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 10 monclient: _add_conns ranks=[4,3,0,1,5,2] 2026-02-17T05:56:49.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7febc40700f0 0x7febc40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 10 monclient(hunting): picked mon.noname-b con 0x7febc40700f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:49.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7febc40c7fe0 0x7febc40c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 10 monclient(hunting): picked mon.noname-a con 0x7febc40c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:49.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7febc41536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:49.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc40c8970 con 0x7febc41536f0 2026-02-17T05:56:49.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7febc40c4820 con 0x7febc40c7fe0 2026-02-17T05:56:49.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7febc40c4b00 con 0x7febc40700f0 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.976+0000 7febc9e66640 10 monclient(hunting): authenticate will time out at 1062.545607s 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc37fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7febc40700f0 0x7febc40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc2ffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7febc40c7fe0 0x7febc40c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc37fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7febc40700f0 0x7febc40704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50758/0 (socket says 10.20.193.16:50758) 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc37fe640 1 -- 10.20.193.16:0/344023358 learned_addr learned my addr 10.20.193.16:0/344023358 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc37fe640 10 monclient(hunting): get_auth_request con 0x7febc40700f0 auth_method 0 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc2ffd640 10 monclient(hunting): get_auth_request con 0x7febc40c7fe0 auth_method 0 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.977+0000 7febc2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.978+0000 7febc37fe640 10 monclient(hunting): handle_auth_done global_id 4567 payload 274 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.978+0000 7febc37fe640 1 -- 10.20.193.16:0/344023358 >> v2:10.20.193.16:3300/0 conn(0x7febc40c7fe0 msgr2=0x7febc40c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:49.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.978+0000 7febc37fe640 1 --2- 10.20.193.16:0/344023358 >> v2:10.20.193.16:3300/0 conn(0x7febc40c7fe0 0x7febc40c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.978+0000 7febc37fe640 1 -- 10.20.193.16:0/344023358 >> v1:10.20.193.16:6789/0 conn(0x7febc41536f0 legacy=0x7febc4153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.978+0000 7febc37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.978+0000 7febc37fe640 1 monclient: found mon.noname-b 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.978+0000 7febc37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.978+0000 7febc37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.978+0000 7febc37fe640 1 -- 10.20.193.16:0/344023358 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc40c4de0 con 0x7febc40700f0 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.979+0000 7febc37fe640 1 --2- 10.20.193.16:0/344023358 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7febc40700f0 0x7febc40704d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7febc4070b90 tx=0x7febb002ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=746dd4825b7ff4e5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.980+0000 7febc27fc640 1 -- 10.20.193.16:0/344023358 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb003c070 con 0x7febc40700f0 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.980+0000 7febc27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.980+0000 7febc27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.980+0000 7febc27fc640 10 monclient: dump: 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.988 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.980+0000 7febc27fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.980+0000 7febc27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.980+0000 7febc27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:49.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.980+0000 7febc9e66640 5 monclient: authenticate success, global_id 4567 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.980+0000 7febc9e66640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc27fc640 1 -- 10.20.193.16:0/344023358 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb002fb30 con 0x7febc40700f0 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc27fc640 1 -- 10.20.193.16:0/344023358 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0037440 con 0x7febc40700f0 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 4 set_mon_vals no callback set 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc27fc640 10 monclient: dump: 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.990 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 10 monclient: shutdown 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 1 -- 10.20.193.16:0/344023358 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7febc40700f0 msgr2=0x7febc40704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 1 --2- 10.20.193.16:0/344023358 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7febc40700f0 0x7febc40704d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7febc4070b90 tx=0x7febb002ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:49.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 1 -- 10.20.193.16:0/344023358 shutdown_connections 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 1 --2- 10.20.193.16:0/344023358 >> v2:10.20.193.16:3300/0 conn(0x7febc40c7fe0 0x7febc40c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 1 --2- 10.20.193.16:0/344023358 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7febc40700f0 0x7febc40704d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 1 -- 10.20.193.16:0/344023358 >> 10.20.193.16:0/344023358 conn(0x7febc40cd770 msgr2=0x7febc40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 1 -- 10.20.193.16:0/344023358 shutdown_connections 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 1 -- 10.20.193.16:0/344023358 wait complete. 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 10 monclient: build_initial_monmap 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.981+0000 7febc9e66640 10 monclient: monmap: 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:49.992 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 librados: starting msgr at 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 librados: starting objecter 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 Processor -- start 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 -- start start 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 librados: setting wanted keys 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 librados: calling monclient init 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient: init 2026-02-17T05:56:49.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7febc40700f0 0x7febc4155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient(hunting): picked mon.noname-c con 0x7febc40700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7febc40c7fe0 0x7febc4146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient(hunting): picked mon.noname-a con 0x7febc40c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7febc41536f0 0x7febc4072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient(hunting): picked mon.noname-b con 0x7febc41536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7febc40c5680 con 0x7febc40c7fe0 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7febc40c53a0 con 0x7febc41536f0 2026-02-17T05:56:49.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7febc4074c00 con 0x7febc40700f0 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.982+0000 7febc9e66640 10 monclient(hunting): authenticate will time out at 1062.551936s 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc2ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7febc40c7fe0 0x7febc4146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc3fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7febc41536f0 0x7febc4072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc2ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7febc40c7fe0 0x7febc4146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:60246/0 (socket says 10.20.193.16:60246) 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc2ffd640 1 -- 10.20.193.16:0/3620083531 learned_addr learned my addr 10.20.193.16:0/3620083531 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc2ffd640 10 monclient(hunting): get_auth_request con 0x7febc40c7fe0 auth_method 0 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc3fff640 10 monclient(hunting): get_auth_request con 0x7febc41536f0 auth_method 0 2026-02-17T05:56:49.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:49.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:49.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:49.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:49.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:49.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.983+0000 7febc2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.984+0000 7febc3fff640 10 monclient(hunting): handle_auth_done global_id 4570 payload 824 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.984+0000 7febc3fff640 1 -- 10.20.193.16:0/3620083531 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7febc40700f0 msgr2=0x7febc4155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.985+0000 7febc3fff640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7febc40700f0 0x7febc4155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.985+0000 7febc3fff640 1 -- 10.20.193.16:0/3620083531 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7febc40c7fe0 msgr2=0x7febc4146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.985+0000 7febc3fff640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7febc40c7fe0 0x7febc4146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.985+0000 7febc3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.985+0000 7febc3fff640 1 monclient: found mon.noname-b 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.985+0000 7febc3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.985+0000 7febc3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.985+0000 7febc3fff640 1 -- 10.20.193.16:0/3620083531 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc4154460 con 0x7febc41536f0 2026-02-17T05:56:49.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7febc3fff640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7febc41536f0 0x7febc4072400 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7febb800cae0 tx=0x7febb800a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7feb9ffff640 10 client.?.objecter ms_handle_connect 0x7febc41536f0 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7feb9ffff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7feb9ffff640 1 -- 10.20.193.16:0/3620083531 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb800acb0 con 0x7febc41536f0 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7feb9ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7feb9ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7feb9ffff640 10 monclient: dump: 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.000 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7feb9ffff640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7feb9ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7feb9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.986+0000 7febc9e66640 5 monclient: authenticate success, global_id 4570 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 10 monclient: _renew_subs 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 1 -- 10.20.193.16:0/3620083531 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc4154740 con 0x7febc41536f0 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 10 client.4570.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 10 monclient: _renew_subs 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 1 -- 10.20.193.16:0/3620083531 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc4154d00 con 0x7febc41536f0 2026-02-17T05:56:50.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 1 librados: init done 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc9e66640 1 -- 10.20.193.16:0/3620083531 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc40704d0 con 0x7febc41536f0 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7feb9ffff640 1 -- 10.20.193.16:0/3620083531 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb800ae50 con 0x7febc41536f0 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7feb9ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7feb9ffff640 1 -- 10.20.193.16:0/3620083531 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8003c70 con 0x7febc41536f0 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7feb9ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7feb9ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7feb9ffff640 10 monclient: dump: 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.002 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.003 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.003 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.003 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.003 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.003 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.003 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:49.987+0000 7febc1ffb640 4 set_mon_vals no callback set 2026-02-17T05:56:50.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.000+0000 7f4ea44bf640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-17T05:56:50.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.000+0000 7f4ea44bf640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-17T05:56:50.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.000+0000 7f4ea44bf640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.000+0000 7f4ea44bf640 1 -- 10.20.193.16:0/366864926 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f4e9c1573f0 con 0x7f4e9c0700f0 2026-02-17T05:56:50.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.027+0000 7fc2c8ff9640 1 -- 10.20.193.16:0/4095534177 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fc2c001d030 con 0x7fc2cc1536f0 2026-02-17T05:56:50.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.027+0000 7fc2c8ff9640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc29c079140 0x7fc29c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.027+0000 7fc2cbfff640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc29c079140 0x7fc29c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.027+0000 7fc2cbfff640 10 monclient: get_auth_request con 0x7fc29c079140 auth_method 0 2026-02-17T05:56:50.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.028+0000 7fc2cbfff640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc29c079140 0x7fc29c07b600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8004660 tx=0x7fc2b8009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.028+0000 7fc2c8ff9640 10 client.4619.objecter ms_handle_connect 0x7fc29c079140 2026-02-17T05:56:50.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.028+0000 7fc2c8ff9640 1 -- 10.20.193.16:0/4095534177 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fc2c000ccd0 con 0x7fc2cc1536f0 2026-02-17T05:56:50.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.028+0000 7fc2c8ff9640 10 client.4619.objecter ms_dispatch2 0x7fc2cc000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:50.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.028+0000 7fc2c8ff9640 3 client.4619.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:50.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.028+0000 7fc2c8ff9640 3 client.4619.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:50.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.028+0000 7fc2c8ff9640 20 client.4619.objecter dump_active .. 0 homeless 2026-02-17T05:56:50.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.034+0000 7fc2c8ff9640 1 -- 10.20.193.16:0/4095534177 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc2c00624b0 con 0x7fc2cc1536f0 2026-02-17T05:56:50.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.034+0000 7fc2c8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.034+0000 7fc2c8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:50.038 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-17T05:56:50.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.093+0000 7feb9ffff640 1 -- 10.20.193.16:0/3620083531 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7febb80050d0 con 0x7febc41536f0 2026-02-17T05:56:50.094 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-17T05:56:50.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.094+0000 7feb9ffff640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7feb94079170 0x7feb9407b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.094+0000 7febc37fe640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7feb94079170 0x7feb9407b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.094+0000 7febc37fe640 10 monclient: get_auth_request con 0x7feb94079170 auth_method 0 2026-02-17T05:56:50.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.094+0000 7febc37fe640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7feb94079170 0x7feb9407b630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7febc40c9b90 tx=0x7febb0031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.094+0000 7feb9ffff640 10 client.4570.objecter ms_handle_connect 0x7feb94079170 2026-02-17T05:56:50.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.096+0000 7feb9ffff640 1 -- 10.20.193.16:0/3620083531 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7febb8098230 con 0x7febc41536f0 2026-02-17T05:56:50.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.096+0000 7feb9ffff640 10 client.4570.objecter ms_dispatch2 0x7febc4000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:50.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.096+0000 7feb9ffff640 3 client.4570.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:50.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.096+0000 7feb9ffff640 3 client.4570.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:50.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.096+0000 7feb9ffff640 20 client.4570.objecter dump_active .. 0 homeless 2026-02-17T05:56:50.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.104+0000 7feb9ffff640 1 -- 10.20.193.16:0/3620083531 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7febb8062eb0 con 0x7febc41536f0 2026-02-17T05:56:50.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.104+0000 7feb9ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.104+0000 7feb9ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:50.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.105+0000 7f4e82ffd640 1 -- 10.20.193.16:0/366864926 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f4e8c061f50 con 0x7f4e9c0700f0 2026-02-17T05:56:50.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.105+0000 7f4e82ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-17T05:56:50.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.105+0000 7f4e82ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:50.106 INFO:teuthology.orchestra.run.trial016.stdout:98784247814 2026-02-17T05:56:50.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f8d49eda640 10 monclient: get_monmap_and_config 2026-02-17T05:56:50.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f8d49eda640 10 monclient: build_initial_monmap 2026-02-17T05:56:50.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f8d49eda640 10 monclient: monmap: 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:50.108277+0000 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:50.108277+0000 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:50.108 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:50.109 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:50.109 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 10 librados: watch_flush enter 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 10 librados: watch_flush exit 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 20 client.4564.objecter shutdown clearing up homeless session... 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 10 client.4564.objecter successfully canceled tick 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 -- 10.20.193.16:0/366864926 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4e6c079170 msgr2=0x7f4e6c07b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4e6c079170 0x7f4e6c07b630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0c67c0 tx=0x7f4e90009190 comp rx=0 tx=0).stop 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 10 monclient: shutdown 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 -- 10.20.193.16:0/366864926 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4e9c0700f0 msgr2=0x7f4e9c150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4e9c0700f0 0x7f4e9c150020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00b500 tx=0x7f4e8c00b9d0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 -- 10.20.193.16:0/366864926 shutdown_connections 2026-02-17T05:56:50.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4e6c079170 0x7f4e6c07b630 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4e9c1536f0 0x7f4e9c1f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4e9c0c7fe0 0x7f4e9c1f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 --2- 10.20.193.16:0/366864926 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4e9c0700f0 0x7f4e9c150020 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 -- 10.20.193.16:0/366864926 >> 10.20.193.16:0/366864926 conn(0x7f4e9c0cd770 msgr2=0x7f4e9c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 -- 10.20.193.16:0/366864926 shutdown_connections 2026-02-17T05:56:50.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 -- 10.20.193.16:0/366864926 wait complete. 2026-02-17T05:56:50.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f4ea44bf640 1 librados: shutdown 2026-02-17T05:56:50.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f8d49eda640 1 Processor -- start 2026-02-17T05:56:50.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f8d49eda640 1 -- start start 2026-02-17T05:56:50.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.107+0000 7f8d49eda640 10 monclient: init 2026-02-17T05:56:50.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d49eda640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d49eda640 10 monclient: _add_conns ranks=[0,2,1,4,3,5] 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d49eda640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8d4414f020 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d49eda640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8d4414f400 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d49eda640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f8d4415caf0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d49eda640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d440c5b10 con 0x7f8d4414f020 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d49eda640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d4406e620 con 0x7f8d4415caf0 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d49eda640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d4406e900 con 0x7f8d4414f400 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d49eda640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d49eda640 10 monclient(hunting): authenticate will time out at 1062.677329s 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d42ffd640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f8d4414f400 0x7f8d4415c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46640/0 (socket says 10.20.193.16:46640) 2026-02-17T05:56:50.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d42ffd640 1 -- 10.20.193.16:0/2050433725 learned_addr learned my addr 10.20.193.16:0/2050433725 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.108+0000 7f8d437fe640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f8d4414f020 0x7f8d4406dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:56210/0 (socket says 10.20.193.16:56210) 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 617604445 0 0) 0x7f8d4406e900 con 0x7f8d4414f400 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): _negotiate 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): my global_id is 4622 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d28003620 con 0x7f8d4414f400 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962298789 0 0) 0x7f8d440c5b10 con 0x7f8d4414f020 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): _negotiate 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): my global_id is 4488 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d4406e900 con 0x7f8d4414f020 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3533256840 0 0) 0x7f8d4406e620 con 0x7f8d4415caf0 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): _negotiate 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 10 monclient(hunting): my global_id is 4573 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.110+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d440c5b10 con 0x7f8d4415caf0 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 <== mon.2 v1:10.20.193.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 105911455 0 0) 0x7f8d28003620 con 0x7f8d4414f400 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 >> v1:10.20.193.128:6789/0 conn(0x7f8d4415caf0 legacy=0x7f8d440c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 >> v1:10.20.193.16:6789/0 conn(0x7f8d4414f020 legacy=0x7f8d4406dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.148:6789/0 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 --> v1:10.20.193.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4406ebe0 con 0x7f8d4414f400 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 <== mon.2 v1:10.20.193.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f8d34002f00 con 0x7f8d4414f400 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient: dump: 2026-02-17T05:56:50.122 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] but i'm connected to v1:10.20.193.148:6789/0, reconnecting 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 >> v1:10.20.193.148:6789/0 conn(0x7f8d4414f400 legacy=0x7f8d4415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8d28003fb0 0x7f8d28004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient(hunting): picked mon.c con 0x7f8d28003fb0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8d280044c0 0x7f8d28004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient(hunting): picked mon.a con 0x7f8d280044c0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d2800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient(hunting): picked mon.b con 0x7f8d28009ab0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2800c490 con 0x7f8d280044c0 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2800c610 con 0x7f8d28009ab0 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2800c810 con 0x7f8d28003fb0 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.111+0000 7f8d427fc640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.112+0000 7f8d42ffd640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8d280044c0 0x7f8d28004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.112+0000 7f8d43fff640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d2800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.112+0000 7f8d43fff640 10 monclient(hunting): get_auth_request con 0x7f8d28009ab0 auth_method 0 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.112+0000 7f8d43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.112+0000 7f8d43fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.112+0000 7f8d43fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d42ffd640 10 monclient(hunting): get_auth_request con 0x7f8d280044c0 auth_method 0 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d42ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d42ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d43fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d437fe640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8d28003fb0 0x7f8d28004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d437fe640 10 monclient(hunting): get_auth_request con 0x7f8d28003fb0 auth_method 0 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d437fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d437fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d437fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:50.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.113+0000 7f8d42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 10 monclient(hunting): handle_auth_done global_id 4622 payload 293 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 1 -- 10.20.193.16:0/2050433725 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8d28003fb0 msgr2=0x7f8d28004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8d28003fb0 0x7f8d28004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 1 -- 10.20.193.16:0/2050433725 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8d280044c0 msgr2=0x7f8d28004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8d280044c0 0x7f8d28004ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 1 monclient: found mon.b 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 1 -- 10.20.193.16:0/2050433725 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d28003620 con 0x7f8d28009ab0 2026-02-17T05:56:50.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d49eda640 5 monclient: authenticate success, global_id 4622 2026-02-17T05:56:50.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d49eda640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:50.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 1 -- 10.20.193.16:0/2050433725 reap_dead start 2026-02-17T05:56:50.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.114+0000 7f8d43fff640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d2800bea0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800c320 tx=0x7f8d3802f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e097bc1d9123318a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.129 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-17T05:56:50.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247814 for osd.11 2026-02-17T05:56:50.131 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:50.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38003860 con 0x7f8d28009ab0 2026-02-17T05:56:50.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d427fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:50.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d427fc640 10 monclient: dump: 2026-02-17T05:56:50.135 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.135 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d38003c70 con 0x7f8d28009ab0 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d427fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d427fc640 1 -- 10.20.193.16:0/2050433725 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38005310 con 0x7f8d28009ab0 2026-02-17T05:56:50.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d427fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d427fc640 10 monclient: dump: 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 4 set_mon_vals no callback set 2026-02-17T05:56:50.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 10 monclient: shutdown 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 1 -- 10.20.193.16:0/2050433725 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 msgr2=0x7f8d2800bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d2800bea0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800c320 tx=0x7f8d3802f610 comp rx=0 tx=0).stop 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 1 -- 10.20.193.16:0/2050433725 shutdown_connections 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 1 --2- 10.20.193.16:0/2050433725 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d2800bea0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 1 -- 10.20.193.16:0/2050433725 >> 10.20.193.16:0/2050433725 conn(0x7f8d440cd770 msgr2=0x7f8d440cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 1 -- 10.20.193.16:0/2050433725 shutdown_connections 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.116+0000 7f8d49eda640 1 -- 10.20.193.16:0/2050433725 wait complete. 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.117+0000 7f8d49eda640 10 monclient: build_initial_monmap 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.117+0000 7f8d49eda640 10 monclient: monmap: 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:50.138 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.117+0000 7f8d49eda640 1 librados: starting msgr at 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.117+0000 7f8d49eda640 1 librados: starting objecter 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.117+0000 7f8d49eda640 1 Processor -- start 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.117+0000 7f8d49eda640 1 -- start start 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.117+0000 7f8d49eda640 1 librados: setting wanted keys 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.117+0000 7f8d49eda640 1 librados: calling monclient init 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.117+0000 7f8d49eda640 10 monclient: init 2026-02-17T05:56:50.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d441f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 10 monclient(hunting): picked mon.noname-b con 0x7f8d28009ab0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8d44155280 0x7f8d441579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 10 monclient(hunting): picked mon.noname-c con 0x7f8d44155280 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8d44157f10 0x7f8d44143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 10 monclient(hunting): picked mon.noname-a con 0x7f8d44157f10 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4415c3e0 con 0x7f8d44157f10 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4406f1a0 con 0x7f8d28009ab0 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4415c560 con 0x7f8d44155280 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d49eda640 10 monclient(hunting): authenticate will time out at 1062.687110s 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d437fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d441f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d42ffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8d44155280 0x7f8d441579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d43fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8d44157f10 0x7f8d44143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d437fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d441f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:50788/0 (socket says 10.20.193.16:50788) 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d437fe640 1 -- 10.20.193.16:0/1851418541 learned_addr learned my addr 10.20.193.16:0/1851418541 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d437fe640 10 monclient(hunting): get_auth_request con 0x7f8d28009ab0 auth_method 0 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d437fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d42ffd640 10 monclient(hunting): get_auth_request con 0x7f8d44155280 auth_method 0 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d42ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d42ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d43fff640 10 monclient(hunting): get_auth_request con 0x7f8d44157f10 auth_method 0 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d43fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d43fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.118+0000 7f8d42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.119+0000 7f8d43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.119+0000 7f8d43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.119+0000 7f8d43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7f8d437fe640 10 monclient(hunting): handle_auth_done global_id 4576 payload 824 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7f8d437fe640 1 -- 10.20.193.16:0/1851418541 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8d44155280 msgr2=0x7f8d441579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7f8d437fe640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8d44155280 0x7f8d441579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7f8d437fe640 1 -- 10.20.193.16:0/1851418541 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8d44157f10 msgr2=0x7f8d44143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7f8d437fe640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8d44157f10 0x7f8d44143f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7f8d437fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7f8d437fe640 1 monclient: found mon.noname-b 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7f8d437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7f8d437fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7f8d437fe640 1 -- 10.20.193.16:0/1851418541 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d44144ee0 con 0x7f8d28009ab0 2026-02-17T05:56:50.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7fc2d29e7640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:56:50.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.120+0000 7fc2d29e7640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:56:50.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.121+0000 7fc2d29e7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:50.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.121+0000 7fc2d29e7640 1 -- 10.20.193.16:0/4095534177 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc2cc145310 con 0x7fc2cc1536f0 2026-02-17T05:56:50.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.121+0000 7f8d437fe640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d441f5240 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d30002e30 tx=0x7f8d30003300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.121+0000 7f8d1ffff640 10 client.?.objecter ms_handle_connect 0x7f8d28009ab0 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.121+0000 7f8d1ffff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 1 -- 10.20.193.16:0/1851418541 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30003560 con 0x7f8d28009ab0 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 10 monclient: dump: 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.145 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d49eda640 5 monclient: authenticate success, global_id 4576 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d49eda640 10 monclient: _renew_subs 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d49eda640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d49eda640 1 -- 10.20.193.16:0/1851418541 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d44155790 con 0x7f8d28009ab0 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d49eda640 10 client.4576.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d49eda640 10 monclient: _renew_subs 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d49eda640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d49eda640 1 -- 10.20.193.16:0/1851418541 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d44155d50 con 0x7f8d28009ab0 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d49eda640 1 librados: init done 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 1 -- 10.20.193.16:0/1851418541 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d30018070 con 0x7f8d28009ab0 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 1 -- 10.20.193.16:0/1851418541 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3000fbc0 con 0x7f8d28009ab0 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d1ffff640 10 monclient: dump: 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.122+0000 7f8d41ffb640 4 set_mon_vals no callback set 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.123+0000 7f8d49eda640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.123+0000 7f8d49eda640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.123+0000 7f8d49eda640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.123+0000 7f8d49eda640 1 -- 10.20.193.16:0/1851418541 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4406dbe0 con 0x7f8d28009ab0 2026-02-17T05:56:50.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.124+0000 7fc2c8ff9640 1 -- 10.20.193.16:0/4095534177 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc2c0061db0 con 0x7fc2cc1536f0 2026-02-17T05:56:50.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.124+0000 7fc2c8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:56:50.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.124+0000 7fc2c8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:50.148 INFO:teuthology.orchestra.run.trial016.stdout:120259084296 2026-02-17T05:56:50.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 10 librados: watch_flush enter 2026-02-17T05:56:50.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 10 librados: watch_flush exit 2026-02-17T05:56:50.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 20 client.4619.objecter shutdown clearing up homeless session... 2026-02-17T05:56:50.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 10 client.4619.objecter successfully canceled tick 2026-02-17T05:56:50.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 -- 10.20.193.16:0/4095534177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc29c079140 msgr2=0x7fc29c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc29c079140 0x7fc29c07b600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8004660 tx=0x7fc2b8009110 comp rx=0 tx=0).stop 2026-02-17T05:56:50.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 10 monclient: shutdown 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 -- 10.20.193.16:0/4095534177 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc1536f0 msgr2=0x7fc2cc072e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc1536f0 0x7fc2cc072e50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000b9a0 tx=0x7fc2c000be70 comp rx=0 tx=0).stop 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 -- 10.20.193.16:0/4095534177 shutdown_connections 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc29c079140 0x7fc29c07b600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc2cc0c86f0 0x7fc2cc1f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc2cc1536f0 0x7fc2cc072e50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 --2- 10.20.193.16:0/4095534177 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc2cc0700f0 0x7fc2cc1425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 -- 10.20.193.16:0/4095534177 >> 10.20.193.16:0/4095534177 conn(0x7fc2cc0cd770 msgr2=0x7fc2cc1457c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 -- 10.20.193.16:0/4095534177 shutdown_connections 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 -- 10.20.193.16:0/4095534177 wait complete. 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.126+0000 7fc2d29e7640 1 librados: shutdown 2026-02-17T05:56:50.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.134+0000 7f8d1ffff640 1 -- 10.20.193.16:0/1851418541 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f8d3001d460 con 0x7f8d28009ab0 2026-02-17T05:56:50.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084295 got 120259084296 for osd.0 2026-02-17T05:56:50.151 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:50.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.134+0000 7f8d1ffff640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d140792e0 0x7f8d1407b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.134+0000 7f8d42ffd640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d140792e0 0x7f8d1407b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.134+0000 7f8d42ffd640 10 monclient: get_auth_request con 0x7f8d140792e0 auth_method 0 2026-02-17T05:56:50.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.134+0000 7f8d42ffd640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d140792e0 0x7f8d1407b7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8d34002410 tx=0x7f8d340029c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.134+0000 7f8d1ffff640 10 client.4576.objecter ms_handle_connect 0x7f8d140792e0 2026-02-17T05:56:50.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.135+0000 7f8d1ffff640 1 -- 10.20.193.16:0/1851418541 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f8d3000fd60 con 0x7f8d28009ab0 2026-02-17T05:56:50.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.135+0000 7f8d1ffff640 10 client.4576.objecter ms_dispatch2 0x7f8d44000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:50.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.135+0000 7f8d1ffff640 3 client.4576.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:50.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.135+0000 7f8d1ffff640 3 client.4576.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:50.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.135+0000 7f8d1ffff640 20 client.4576.objecter dump_active .. 0 homeless 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.163+0000 7f58013ba640 10 monclient: get_monmap_and_config 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.163+0000 7f58013ba640 10 monclient: build_initial_monmap 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.163+0000 7f58013ba640 10 monclient: monmap: 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:50.164845+0000 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:50.164845+0000 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:50.164 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 1 Processor -- start 2026-02-17T05:56:50.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 1 -- start start 2026-02-17T05:56:50.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 10 monclient: init 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 10 monclient: _add_conns ranks=[2,4,3,0,5,1] 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f57fc0700f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f57fc0c86f0 0x7f57fc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 10 monclient(hunting): picked mon.noname-b con 0x7f57fc0c86f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f57fc1536f0 0x7f57fc153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 10 monclient(hunting): picked mon.noname-a con 0x7f57fc1536f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57fc0c4820 con 0x7f57fc0700f0 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f57fc0c4b00 con 0x7f57fc1536f0 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f57fc0c4de0 con 0x7f57fc0c86f0 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.164+0000 7f58013ba640 10 monclient(hunting): authenticate will time out at 1062.733938s 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.165+0000 7f57faffd640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f57fc0700f0 0x7f57fc0c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46642/0 (socket says 10.20.193.16:46642) 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.165+0000 7f57faffd640 1 -- 10.20.193.16:0/3980046837 learned_addr learned my addr 10.20.193.16:0/3980046837 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.165+0000 7f57fb7fe640 1 --2- 10.20.193.16:0/3980046837 >> v2:10.20.193.16:3300/0 conn(0x7f57fc1536f0 0x7f57fc153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.165+0000 7f57fb7fe640 10 monclient(hunting): get_auth_request con 0x7f57fc1536f0 auth_method 0 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.165+0000 7f57fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.165+0000 7f57fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.165+0000 7f57fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57f9ffb640 1 -- 10.20.193.16:0/3980046837 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3120774844 0 0) 0x7f57fc0c4820 con 0x7f57fc0700f0 2026-02-17T05:56:50.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57f9ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:56:50.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57f9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57f9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57f9ffb640 10 monclient(hunting): my global_id is 4628 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57f9ffb640 1 -- 10.20.193.16:0/3980046837 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57e8003620 con 0x7f57fc0700f0 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 10 monclient(hunting): handle_auth_done global_id 4494 payload 274 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 1 -- 10.20.193.16:0/3980046837 >> v2:10.20.193.128:3300/0 conn(0x7f57fc0c86f0 msgr2=0x7f57fc0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 1 --2- 10.20.193.16:0/3980046837 >> v2:10.20.193.128:3300/0 conn(0x7f57fc0c86f0 0x7f57fc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 1 -- 10.20.193.16:0/3980046837 >> v1:10.20.193.148:6789/0 conn(0x7f57fc0700f0 legacy=0x7f57fc0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 1 monclient: found mon.noname-a 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.166+0000 7f57fb7fe640 1 -- 10.20.193.16:0/3980046837 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc0c50c0 con 0x7f57fc1536f0 2026-02-17T05:56:50.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.167+0000 7f57fb7fe640 1 --2- 10.20.193.16:0/3980046837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f57fc1536f0 0x7f57fc153b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0c5b80 tx=0x7f57ec02f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3df55828753e90a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 1 -- 10.20.193.16:0/3980046837 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec03c070 con 0x7f57fc1536f0 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 10 monclient: dump: 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.169 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f58013ba640 5 monclient: authenticate success, global_id 4494 2026-02-17T05:56:50.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f58013ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:50.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 1 -- 10.20.193.16:0/3980046837 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec02fe40 con 0x7f57fc1536f0 2026-02-17T05:56:50.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 1 -- 10.20.193.16:0/3980046837 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec037640 con 0x7f57fc1536f0 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f57f9ffb640 10 monclient: dump: 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f58013ba640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.169+0000 7f58013ba640 4 set_mon_vals no callback set 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 10 monclient: shutdown 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 -- 10.20.193.16:0/3980046837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f57fc1536f0 msgr2=0x7f57fc153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 --2- 10.20.193.16:0/3980046837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f57fc1536f0 0x7f57fc153b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0c5b80 tx=0x7f57ec02f130 comp rx=0 tx=0).stop 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 -- 10.20.193.16:0/3980046837 shutdown_connections 2026-02-17T05:56:50.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 --2- 10.20.193.16:0/3980046837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f57fc1536f0 0x7f57fc153b20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 --2- 10.20.193.16:0/3980046837 >> v2:10.20.193.128:3300/0 conn(0x7f57fc0c86f0 0x7f57fc0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 -- 10.20.193.16:0/3980046837 >> 10.20.193.16:0/3980046837 conn(0x7f57fc0cd770 msgr2=0x7f57fc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 -- 10.20.193.16:0/3980046837 shutdown_connections 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 -- 10.20.193.16:0/3980046837 wait complete. 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 10 monclient: build_initial_monmap 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 10 monclient: monmap: 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 librados: starting msgr at 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 librados: starting objecter 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 Processor -- start 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 -- start start 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 librados: setting wanted keys 2026-02-17T05:56:50.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 1 librados: calling monclient init 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.170+0000 7f58013ba640 10 monclient: init 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f57fc0700f0 0x7f57fc1f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 10 monclient(hunting): picked mon.noname-c con 0x7f57fc0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f57fc0c86f0 0x7f57fc06edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 10 monclient(hunting): picked mon.noname-b con 0x7f57fc0c86f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f57fc1536f0 0x7f57fc152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 10 monclient(hunting): picked mon.noname-a con 0x7f57fc1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc154060 con 0x7f57fc1536f0 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc0c5680 con 0x7f57fc0c86f0 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc074c00 con 0x7f57fc0700f0 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f58013ba640 10 monclient(hunting): authenticate will time out at 1062.740228s 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57fb7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f57fc1536f0 0x7f57fc152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57fb7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f57fc1536f0 0x7f57fc152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44438/0 (socket says 10.20.193.16:44438) 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57fb7fe640 1 -- 10.20.193.16:0/239438520 learned_addr learned my addr 10.20.193.16:0/239438520 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57fb7fe640 10 monclient(hunting): get_auth_request con 0x7f57fc1536f0 auth_method 0 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57faffd640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f57fc0700f0 0x7f57fc1f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57faffd640 10 monclient(hunting): get_auth_request con 0x7f57fc0700f0 auth_method 0 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.171+0000 7f57faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.172+0000 7f57fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.172+0000 7f57fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.172+0000 7f57fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.172+0000 7f57faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.172+0000 7f57faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.172+0000 7f57faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.173+0000 7f57faffd640 10 monclient(hunting): handle_auth_done global_id 4631 payload 824 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.173+0000 7f57faffd640 1 -- 10.20.193.16:0/239438520 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f57fc0c86f0 msgr2=0x7f57fc06edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.173+0000 7f57faffd640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f57fc0c86f0 0x7f57fc06edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.173+0000 7f57faffd640 1 -- 10.20.193.16:0/239438520 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f57fc1536f0 msgr2=0x7f57fc152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.173+0000 7f57faffd640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f57fc1536f0 0x7f57fc152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.173+0000 7f57faffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.173+0000 7f57faffd640 1 monclient: found mon.noname-c 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.173+0000 7f57faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.173+0000 7f57faffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.173+0000 7f57faffd640 1 -- 10.20.193.16:0/239438520 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc06e330 con 0x7f57fc0700f0 2026-02-17T05:56:50.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.174+0000 7f57faffd640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f57fc0700f0 0x7f57fc1f6ac0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c450 tx=0x7f57f000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.174+0000 7f57db7fe640 10 client.?.objecter ms_handle_connect 0x7f57fc0700f0 2026-02-17T05:56:50.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.174+0000 7f57db7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:50.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 1 -- 10.20.193.16:0/239438520 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0017070 con 0x7f57fc0700f0 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 10 monclient: dump: 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 1 -- 10.20.193.16:0/239438520 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f0005de0 con 0x7f57fc0700f0 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 1 -- 10.20.193.16:0/239438520 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0007270 con 0x7f57fc0700f0 2026-02-17T05:56:50.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 5 monclient: authenticate success, global_id 4631 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 10 monclient: _renew_subs 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57f97fa640 4 set_mon_vals no callback set 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 1 -- 10.20.193.16:0/239438520 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57fc1f7000 con 0x7f57fc0700f0 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f57db7fe640 10 monclient: dump: 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 10 client.4631.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:50.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 10 monclient: _renew_subs 2026-02-17T05:56:50.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:50.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 1 -- 10.20.193.16:0/239438520 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57fc1f75b0 con 0x7f57fc0700f0 2026-02-17T05:56:50.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 1 librados: init done 2026-02-17T05:56:50.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:50.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.176+0000 7f58013ba640 1 -- 10.20.193.16:0/239438520 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57fc0704d0 con 0x7f57fc0700f0 2026-02-17T05:56:50.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.188+0000 7f57db7fe640 1 -- 10.20.193.16:0/239438520 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f57f0007410 con 0x7f57fc0700f0 2026-02-17T05:56:50.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.188+0000 7f57db7fe640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f57c8079190 0x7f57c807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.188+0000 7f57fa7fc640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f57c8079190 0x7f57c807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.188+0000 7f57fa7fc640 10 monclient: get_auth_request con 0x7f57c8079190 auth_method 0 2026-02-17T05:56:50.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.189+0000 7f57fa7fc640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f57c8079190 0x7f57c807b650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57fc151100 tx=0x7f57e4006cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.189+0000 7f57db7fe640 10 client.4631.objecter ms_handle_connect 0x7f57c8079190 2026-02-17T05:56:50.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.189+0000 7f57db7fe640 1 -- 10.20.193.16:0/239438520 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f57f0007730 con 0x7f57fc0700f0 2026-02-17T05:56:50.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.189+0000 7f57db7fe640 10 client.4631.objecter ms_dispatch2 0x7f57fc000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:50.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.189+0000 7f57db7fe640 3 client.4631.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:50.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.189+0000 7f57db7fe640 3 client.4631.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:50.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.189+0000 7f57db7fe640 20 client.4631.objecter dump_active .. 0 homeless 2026-02-17T05:56:50.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.192+0000 7febc9e66640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:56:50.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.192+0000 7febc9e66640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:56:50.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.192+0000 7febc9e66640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.192+0000 7febc9e66640 1 -- 10.20.193.16:0/3620083531 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7febc41573f0 con 0x7febc41536f0 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.218+0000 7f5b7169f640 10 monclient: get_monmap_and_config 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.218+0000 7f5b7169f640 10 monclient: build_initial_monmap 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.218+0000 7f5b7169f640 10 monclient: monmap: 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:50.219270+0000 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:50.219270+0000 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:50.218 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.218+0000 7f5b7169f640 1 Processor -- start 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.218+0000 7f5b7169f640 1 -- start start 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.218+0000 7f5b7169f640 10 monclient: init 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 10 monclient: _add_conns ranks=[1,3,5,0,2,4] 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5b6c155ac0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f5b6c070590 0x7f5b6c073890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 10 monclient(hunting): picked mon.noname-a con 0x7f5b6c070590 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f5b6c155ea0 0x7f5b6c15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 10 monclient(hunting): picked mon.noname-c con 0x7f5b6c155ea0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b6c072510 con 0x7f5b6c155ac0 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5b6c0c8d10 con 0x7f5b6c070590 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f5b6c0727f0 con 0x7f5b6c155ea0 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b7169f640 10 monclient(hunting): authenticate will time out at 1062.788328s 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.219+0000 7f5b6a7fc640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f5b6c070590 0x7f5b6c073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.220+0000 7f5b6a7fc640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f5b6c070590 0x7f5b6c073890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44442/0 (socket says 10.20.193.16:44442) 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.220+0000 7f5b6a7fc640 1 -- 10.20.193.16:0/1802377596 learned_addr learned my addr 10.20.193.16:0/1802377596 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.220+0000 7f5b6a7fc640 10 monclient(hunting): get_auth_request con 0x7f5b6c070590 auth_method 0 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.220+0000 7f5b6a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.220+0000 7f5b6a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.220+0000 7f5b6a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.220+0000 7f5b6a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.220+0000 7f5b6a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.220+0000 7f5b6a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.221+0000 7f5b6a7fc640 10 monclient(hunting): handle_auth_done global_id 4500 payload 274 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.221+0000 7f5b6a7fc640 1 -- 10.20.193.16:0/1802377596 >> v2:10.20.193.148:3300/0 conn(0x7f5b6c155ea0 msgr2=0x7f5b6c15d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.221+0000 7f5b6a7fc640 1 --2- 10.20.193.16:0/1802377596 >> v2:10.20.193.148:3300/0 conn(0x7f5b6c155ea0 0x7f5b6c15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.221+0000 7f5b6a7fc640 1 -- 10.20.193.16:0/1802377596 >> v1:10.20.193.128:6789/0 conn(0x7f5b6c155ac0 legacy=0x7f5b6c06fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:50.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.221+0000 7f5b6a7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.221+0000 7f5b6a7fc640 1 monclient: found mon.noname-a 2026-02-17T05:56:50.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.221+0000 7f5b6a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.221+0000 7f5b6a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.221+0000 7f5b6a7fc640 1 -- 10.20.193.16:0/1802377596 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c072ad0 con 0x7f5b6c070590 2026-02-17T05:56:50.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.222+0000 7f5b6a7fc640 1 --2- 10.20.193.16:0/1802377596 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c070590 0x7f5b6c073890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c06f440 tx=0x7f5b5402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d37280c8b714eea server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 1 -- 10.20.193.16:0/1802377596 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5403c070 con 0x7f5b6c070590 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 10 monclient: dump: 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 1 -- 10.20.193.16:0/1802377596 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b5402fa90 con 0x7f5b6c070590 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 1 -- 10.20.193.16:0/1802377596 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5402fe60 con 0x7f5b6c070590 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:50.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b69ffb640 10 monclient: dump: 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 5 monclient: authenticate success, global_id 4500 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 4 set_mon_vals no callback set 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 10 monclient: shutdown 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 1 -- 10.20.193.16:0/1802377596 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c070590 msgr2=0x7f5b6c073890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 1 --2- 10.20.193.16:0/1802377596 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c070590 0x7f5b6c073890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c06f440 tx=0x7f5b5402ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 1 -- 10.20.193.16:0/1802377596 shutdown_connections 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 1 --2- 10.20.193.16:0/1802377596 >> v2:10.20.193.148:3300/0 conn(0x7f5b6c155ea0 0x7f5b6c15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 1 --2- 10.20.193.16:0/1802377596 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c070590 0x7f5b6c073890 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 1 -- 10.20.193.16:0/1802377596 >> 10.20.193.16:0/1802377596 conn(0x7f5b6c0cd770 msgr2=0x7f5b6c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 1 -- 10.20.193.16:0/1802377596 shutdown_connections 2026-02-17T05:56:50.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.224+0000 7f5b7169f640 1 -- 10.20.193.16:0/1802377596 wait complete. 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient: build_initial_monmap 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient: monmap: 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 librados: starting msgr at 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 librados: starting objecter 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 Processor -- start 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 -- start start 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 librados: setting wanted keys 2026-02-17T05:56:50.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 librados: calling monclient init 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient: init 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5b6c070590 0x7f5b6c158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient(hunting): picked mon.noname-b con 0x7f5b6c070590 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c155ac0 0x7f5b6c156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient(hunting): picked mon.noname-a con 0x7f5b6c155ac0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5b6c155ea0 0x7f5b6c1f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient(hunting): picked mon.noname-c con 0x7f5b6c155ea0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c073370 con 0x7f5b6c155ac0 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c073090 con 0x7f5b6c070590 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c15cb30 con 0x7f5b6c155ea0 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.225+0000 7f5b7169f640 10 monclient(hunting): authenticate will time out at 1062.794783s 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.226+0000 7f5b6a7fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c155ac0 0x7f5b6c156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.226+0000 7f5b6a7fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c155ac0 0x7f5b6c156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44444/0 (socket says 10.20.193.16:44444) 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.226+0000 7f5b6a7fc640 1 -- 10.20.193.16:0/3503574030 learned_addr learned my addr 10.20.193.16:0/3503574030 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.226+0000 7f5b6a7fc640 10 monclient(hunting): get_auth_request con 0x7f5b6c155ac0 auth_method 0 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.226+0000 7f5b6a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.226+0000 7f5b6a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.226+0000 7f5b6a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.226+0000 7f5b6a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.226+0000 7f5b6a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.226+0000 7f5b6a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.228+0000 7f5b6a7fc640 10 monclient(hunting): handle_auth_done global_id 4503 payload 824 2026-02-17T05:56:50.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.228+0000 7f5b6a7fc640 1 -- 10.20.193.16:0/3503574030 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5b6c155ea0 msgr2=0x7f5b6c1f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.228+0000 7f5b6a7fc640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5b6c155ea0 0x7f5b6c1f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.228+0000 7f5b6a7fc640 1 -- 10.20.193.16:0/3503574030 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5b6c070590 msgr2=0x7f5b6c158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.228+0000 7f5b6a7fc640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5b6c070590 0x7f5b6c158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.228+0000 7f5b6a7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.228+0000 7f5b6a7fc640 1 monclient: found mon.noname-a 2026-02-17T05:56:50.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.228+0000 7f5b6a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.228+0000 7f5b6a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.228+0000 7f5b6a7fc640 1 -- 10.20.193.16:0/3503574030 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c1593a0 con 0x7f5b6c155ac0 2026-02-17T05:56:50.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.229+0000 7f5b6a7fc640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c155ac0 0x7f5b6c156fd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009a70 tx=0x7f5b540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.229+0000 7f5b4b7fe640 10 client.?.objecter ms_handle_connect 0x7f5b6c155ac0 2026-02-17T05:56:50.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.229+0000 7f5b4b7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 1 -- 10.20.193.16:0/3503574030 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54044070 con 0x7f5b6c155ac0 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 10 monclient: dump: 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 1 -- 10.20.193.16:0/3503574030 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b54040630 con 0x7f5b6c155ac0 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 1 -- 10.20.193.16:0/3503574030 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5403c040 con 0x7f5b6c155ac0 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b4b7fe640 10 monclient: dump: 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b697fa640 4 set_mon_vals no callback set 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 5 monclient: authenticate success, global_id 4503 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 10 monclient: _renew_subs 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 1 -- 10.20.193.16:0/3503574030 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c159680 con 0x7f5b6c155ac0 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 10 client.4503.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:50.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 10 monclient: _renew_subs 2026-02-17T05:56:50.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 1 -- 10.20.193.16:0/3503574030 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c159c40 con 0x7f5b6c155ac0 2026-02-17T05:56:50.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 1 librados: init done 2026-02-17T05:56:50.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.231+0000 7f5b7169f640 1 -- 10.20.193.16:0/3503574030 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c073890 con 0x7f5b6c155ac0 2026-02-17T05:56:50.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.239+0000 7f8d1ffff640 1 -- 10.20.193.16:0/1851418541 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8d30062630 con 0x7f8d28009ab0 2026-02-17T05:56:50.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.239+0000 7f8d1ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.239+0000 7f8d1ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:50.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.240+0000 7feb9ffff640 1 -- 10.20.193.16:0/3620083531 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7febb80627b0 con 0x7febc41536f0 2026-02-17T05:56:50.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.240+0000 7feb9ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:56:50.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.240+0000 7feb9ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:50.240 INFO:teuthology.orchestra.run.trial016.stdout:98784247813 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 10 librados: watch_flush enter 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 10 librados: watch_flush exit 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 20 client.4570.objecter shutdown clearing up homeless session... 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 10 client.4570.objecter successfully canceled tick 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 -- 10.20.193.16:0/3620083531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7feb94079170 msgr2=0x7feb9407b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7feb94079170 0x7feb9407b630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7febc40c9b90 tx=0x7febb0031000 comp rx=0 tx=0).stop 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 10 monclient: shutdown 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 -- 10.20.193.16:0/3620083531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7febc41536f0 msgr2=0x7febc4072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7febc41536f0 0x7febc4072400 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7febb800cae0 tx=0x7febb800a060 comp rx=0 tx=0).stop 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 -- 10.20.193.16:0/3620083531 shutdown_connections 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7feb94079170 0x7feb9407b630 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7febc41536f0 0x7febc4072400 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7febc40c7fe0 0x7febc4146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 --2- 10.20.193.16:0/3620083531 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7febc40700f0 0x7febc4155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 -- 10.20.193.16:0/3620083531 >> 10.20.193.16:0/3620083531 conn(0x7febc40cd770 msgr2=0x7febc4147ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 -- 10.20.193.16:0/3620083531 shutdown_connections 2026-02-17T05:56:50.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 -- 10.20.193.16:0/3620083531 wait complete. 2026-02-17T05:56:50.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.242+0000 7febc9e66640 1 librados: shutdown 2026-02-17T05:56:50.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.243+0000 7f5b4b7fe640 1 -- 10.20.193.16:0/3503574030 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f5b5403f050 con 0x7f5b6c155ac0 2026-02-17T05:56:50.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.243+0000 7f5b4b7fe640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5b38079140 0x7f5b3807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.243+0000 7f5b6affd640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5b38079140 0x7f5b3807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.243+0000 7f5b6affd640 10 monclient: get_auth_request con 0x7f5b38079140 auth_method 0 2026-02-17T05:56:50.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.243+0000 7f5b6affd640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5b38079140 0x7f5b3807b600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009750 tx=0x7f5b60006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.244+0000 7f5b4b7fe640 10 client.4503.objecter ms_handle_connect 0x7f5b38079140 2026-02-17T05:56:50.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.245+0000 7f5b4b7fe640 1 -- 10.20.193.16:0/3503574030 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f5b54054cf0 con 0x7f5b6c155ac0 2026-02-17T05:56:50.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.245+0000 7f5b4b7fe640 10 client.4503.objecter ms_dispatch2 0x7f5b6c000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:50.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.245+0000 7f5b4b7fe640 3 client.4503.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:50.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.245+0000 7f5b4b7fe640 3 client.4503.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:50.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.245+0000 7f5b4b7fe640 20 client.4503.objecter dump_active .. 0 homeless 2026-02-17T05:56:50.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247813 for osd.3 2026-02-17T05:56:50.251 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:50.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.291+0000 7f57db7fe640 1 -- 10.20.193.16:0/239438520 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f57f0022020 con 0x7f57fc0700f0 2026-02-17T05:56:50.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.291+0000 7f57db7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.291+0000 7f57db7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:50.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.328+0000 7f8d49eda640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:56:50.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.328+0000 7f8d49eda640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:56:50.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.328+0000 7f8d49eda640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:50.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.328+0000 7f8d49eda640 1 -- 10.20.193.16:0/1851418541 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8d441562c0 con 0x7f8d28009ab0 2026-02-17T05:56:50.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.330+0000 7f8d1ffff640 1 -- 10.20.193.16:0/1851418541 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8d30066930 con 0x7f8d28009ab0 2026-02-17T05:56:50.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.330+0000 7f8d1ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:56:50.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.330+0000 7f8d1ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:50.331 INFO:teuthology.orchestra.run.trial016.stdout:115964117001 2026-02-17T05:56:50.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 10 librados: watch_flush enter 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 10 librados: watch_flush exit 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 20 client.4576.objecter shutdown clearing up homeless session... 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 10 client.4576.objecter successfully canceled tick 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 -- 10.20.193.16:0/1851418541 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d140792e0 msgr2=0x7f8d1407b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d140792e0 0x7f8d1407b7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8d34002410 tx=0x7f8d340029c0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 10 monclient: shutdown 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 -- 10.20.193.16:0/1851418541 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 msgr2=0x7f8d441f5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d441f5240 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d30002e30 tx=0x7f8d30003300 comp rx=0 tx=0).stop 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 -- 10.20.193.16:0/1851418541 shutdown_connections 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d140792e0 0x7f8d1407b7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8d44157f10 0x7f8d44143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8d44155280 0x7f8d441579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 --2- 10.20.193.16:0/1851418541 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8d28009ab0 0x7f8d441f5240 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 -- 10.20.193.16:0/1851418541 >> 10.20.193.16:0/1851418541 conn(0x7f8d440cd770 msgr2=0x7f8d440cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 -- 10.20.193.16:0/1851418541 shutdown_connections 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 -- 10.20.193.16:0/1851418541 wait complete. 2026-02-17T05:56:50.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.332+0000 7f8d49eda640 1 librados: shutdown 2026-02-17T05:56:50.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964117001 got 115964117001 for osd.5 2026-02-17T05:56:50.341 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:50.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.353+0000 7f5b4b7fe640 1 -- 10.20.193.16:0/3503574030 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5b540878c0 con 0x7f5b6c155ac0 2026-02-17T05:56:50.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.354+0000 7f5b4b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.354+0000 7f5b4b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:50.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.376+0000 7f58013ba640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-17T05:56:50.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.376+0000 7f58013ba640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-17T05:56:50.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.376+0000 7f58013ba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:50.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.376+0000 7f58013ba640 1 -- 10.20.193.16:0/239438520 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7f57fc1573f0 con 0x7f57fc0700f0 2026-02-17T05:56:50.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.379+0000 7f57db7fe640 1 -- 10.20.193.16:0/239438520 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f57f0061620 con 0x7f57fc0700f0 2026-02-17T05:56:50.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.379+0000 7f57db7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-17T05:56:50.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.379+0000 7f57db7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:50.379 INFO:teuthology.orchestra.run.trial016.stdout:94489280519 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 10 librados: watch_flush enter 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 10 librados: watch_flush exit 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 20 client.4631.objecter shutdown clearing up homeless session... 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 10 client.4631.objecter successfully canceled tick 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 -- 10.20.193.16:0/239438520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f57c8079190 msgr2=0x7f57c807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f57c8079190 0x7f57c807b650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57fc151100 tx=0x7f57e4006cf0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 10 monclient: shutdown 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 -- 10.20.193.16:0/239438520 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f57fc0700f0 msgr2=0x7f57fc1f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f57fc0700f0 0x7f57fc1f6ac0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c450 tx=0x7f57f000c920 comp rx=0 tx=0).stop 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 -- 10.20.193.16:0/239438520 shutdown_connections 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f57c8079190 0x7f57c807b650 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f57fc0c86f0 0x7f57fc06edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f57fc1536f0 0x7f57fc152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 --2- 10.20.193.16:0/239438520 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f57fc0700f0 0x7f57fc1f6ac0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 -- 10.20.193.16:0/239438520 >> 10.20.193.16:0/239438520 conn(0x7f57fc0cd770 msgr2=0x7f57fc14dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 -- 10.20.193.16:0/239438520 shutdown_connections 2026-02-17T05:56:50.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 -- 10.20.193.16:0/239438520 wait complete. 2026-02-17T05:56:50.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.381+0000 7f58013ba640 1 librados: shutdown 2026-02-17T05:56:50.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280519 got 94489280519 for osd.15 2026-02-17T05:56:50.390 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:50.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.438+0000 7f5b7169f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:56:50.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.439+0000 7f5b7169f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:56:50.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.439+0000 7f5b7169f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.439+0000 7f5b7169f640 1 -- 10.20.193.16:0/3503574030 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5b6c15f030 con 0x7f5b6c155ac0 2026-02-17T05:56:50.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.470+0000 7f5b4b7fe640 1 -- 10.20.193.16:0/3503574030 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b540871c0 con 0x7f5b6c155ac0 2026-02-17T05:56:50.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.470+0000 7f5b4b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:56:50.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.470+0000 7f5b4b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:50.471 INFO:teuthology.orchestra.run.trial016.stdout:90194313223 2026-02-17T05:56:50.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 10 librados: watch_flush enter 2026-02-17T05:56:50.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 10 librados: watch_flush exit 2026-02-17T05:56:50.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 20 client.4503.objecter shutdown clearing up homeless session... 2026-02-17T05:56:50.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 10 client.4503.objecter successfully canceled tick 2026-02-17T05:56:50.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 -- 10.20.193.16:0/3503574030 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5b38079140 msgr2=0x7f5b3807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5b38079140 0x7f5b3807b600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009750 tx=0x7f5b60006ca0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 10 monclient: shutdown 2026-02-17T05:56:50.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 -- 10.20.193.16:0/3503574030 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c155ac0 msgr2=0x7f5b6c156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c155ac0 0x7f5b6c156fd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009a70 tx=0x7f5b540047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 -- 10.20.193.16:0/3503574030 shutdown_connections 2026-02-17T05:56:50.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5b38079140 0x7f5b3807b600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5b6c155ea0 0x7f5b6c1f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5b6c155ac0 0x7f5b6c156fd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 --2- 10.20.193.16:0/3503574030 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5b6c070590 0x7f5b6c158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 -- 10.20.193.16:0/3503574030 >> 10.20.193.16:0/3503574030 conn(0x7f5b6c0cd770 msgr2=0x7f5b6c15f980 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 -- 10.20.193.16:0/3503574030 shutdown_connections 2026-02-17T05:56:50.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 -- 10.20.193.16:0/3503574030 wait complete. 2026-02-17T05:56:50.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.472+0000 7f5b7169f640 1 librados: shutdown 2026-02-17T05:56:50.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313223 got 90194313223 for osd.2 2026-02-17T05:56:50.482 DEBUG:teuthology.parallel:result is None 2026-02-17T05:56:50.482 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-17T05:56:50.482 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T05:56:50.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.548+0000 7ff8c12e1640 10 monclient: get_monmap_and_config 2026-02-17T05:56:50.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.548+0000 7ff8c12e1640 10 monclient: build_initial_monmap 2026-02-17T05:56:50.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.548+0000 7ff8c12e1640 10 monclient: monmap: 2026-02-17T05:56:50.550 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:50.550 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.550 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:50.549540+0000 2026-02-17T05:56:50.550 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:50.549540+0000 2026-02-17T05:56:50.550 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:50.550 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.550 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:50.550 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:50.550 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:50.551 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:50.551 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:50.551 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:50.551 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.548+0000 7ff8c12e1640 1 Processor -- start 2026-02-17T05:56:50.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.548+0000 7ff8c12e1640 1 -- start start 2026-02-17T05:56:50.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 10 monclient: init 2026-02-17T05:56:50.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 10 monclient: _add_conns ranks=[2,5,3,1,0,4] 2026-02-17T05:56:50.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff8bc0700f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:50.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7ff8bc0c86f0 0x7ff8bc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 10 monclient(hunting): picked mon.noname-c con 0x7ff8bc0c86f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:50.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7ff8bc1536f0 0x7ff8bc153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 10 monclient(hunting): picked mon.noname-a con 0x7ff8bc1536f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:50.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8bc0c4820 con 0x7ff8bc0700f0 2026-02-17T05:56:50.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff8bc0c4b00 con 0x7ff8bc1536f0 2026-02-17T05:56:50.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7ff8bc0c4de0 con 0x7ff8bc0c86f0 2026-02-17T05:56:50.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8c12e1640 10 monclient(hunting): authenticate will time out at 1063.118617s 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.549+0000 7ff8bb7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7ff8bc1536f0 0x7ff8bc153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8ba7fc640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7ff8bc0c86f0 0x7ff8bc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8bb7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7ff8bc1536f0 0x7ff8bc153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44450/0 (socket says 10.20.193.16:44450) 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8bb7fe640 1 -- 10.20.193.16:0/2703879544 learned_addr learned my addr 10.20.193.16:0/2703879544 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8bb7fe640 10 monclient(hunting): get_auth_request con 0x7ff8bc1536f0 auth_method 0 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8bb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8bb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8ba7fc640 10 monclient(hunting): get_auth_request con 0x7ff8bc0c86f0 auth_method 0 2026-02-17T05:56:50.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8ba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.550+0000 7ff8ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.551+0000 7ff8bb7fe640 10 monclient(hunting): handle_auth_done global_id 4506 payload 274 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.551+0000 7ff8bb7fe640 1 -- 10.20.193.16:0/2703879544 >> v2:10.20.193.148:3300/0 conn(0x7ff8bc0c86f0 msgr2=0x7ff8bc0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.551+0000 7ff8bb7fe640 1 --2- 10.20.193.16:0/2703879544 >> v2:10.20.193.148:3300/0 conn(0x7ff8bc0c86f0 0x7ff8bc0704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.551+0000 7ff8bb7fe640 1 -- 10.20.193.16:0/2703879544 >> v1:10.20.193.148:6789/0 conn(0x7ff8bc0700f0 legacy=0x7ff8bc0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.551+0000 7ff8bb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.551+0000 7ff8bb7fe640 1 monclient: found mon.noname-a 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.551+0000 7ff8bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.551+0000 7ff8bb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.551+0000 7ff8bb7fe640 1 -- 10.20.193.16:0/2703879544 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc0c50c0 con 0x7ff8bc1536f0 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.552+0000 7ff8bb7fe640 1 --2- 10.20.193.16:0/2703879544 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff8bc1536f0 0x7ff8bc153b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0c5b80 tx=0x7ff8ac02f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2b621e01c3199d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 1 -- 10.20.193.16:0/2703879544 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac02fcf0 con 0x7ff8bc1536f0 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 10 monclient: dump: 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.563 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 1 -- 10.20.193.16:0/2703879544 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ac0373f0 con 0x7ff8bc1536f0 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 5 monclient: authenticate success, global_id 4506 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 1 -- 10.20.193.16:0/2703879544 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0377c0 con 0x7ff8bc1536f0 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8b9ffb640 10 monclient: dump: 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.565 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 4 set_mon_vals no callback set 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 10 monclient: shutdown 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 1 -- 10.20.193.16:0/2703879544 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff8bc1536f0 msgr2=0x7ff8bc153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 1 --2- 10.20.193.16:0/2703879544 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff8bc1536f0 0x7ff8bc153b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0c5b80 tx=0x7ff8ac02f230 comp rx=0 tx=0).stop 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 1 -- 10.20.193.16:0/2703879544 shutdown_connections 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 1 --2- 10.20.193.16:0/2703879544 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff8bc1536f0 0x7ff8bc153b20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 1 --2- 10.20.193.16:0/2703879544 >> v2:10.20.193.148:3300/0 conn(0x7ff8bc0c86f0 0x7ff8bc0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 1 -- 10.20.193.16:0/2703879544 >> 10.20.193.16:0/2703879544 conn(0x7ff8bc0cd770 msgr2=0x7ff8bc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 1 -- 10.20.193.16:0/2703879544 shutdown_connections 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.554+0000 7ff8c12e1640 1 -- 10.20.193.16:0/2703879544 wait complete. 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.555+0000 7ff8c12e1640 10 monclient: build_initial_monmap 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.555+0000 7ff8c12e1640 10 monclient: monmap: 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:50.567 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.555+0000 7ff8c12e1640 1 librados: starting msgr at 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.555+0000 7ff8c12e1640 1 librados: starting objecter 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.555+0000 7ff8c12e1640 1 Processor -- start 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.555+0000 7ff8c12e1640 1 -- start start 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.555+0000 7ff8c12e1640 1 librados: setting wanted keys 2026-02-17T05:56:50.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.555+0000 7ff8c12e1640 1 librados: calling monclient init 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.555+0000 7ff8c12e1640 10 monclient: init 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff8bc0700f0 0x7ff8bc142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 10 monclient(hunting): picked mon.noname-c con 0x7ff8bc0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff8bc0c86f0 0x7ff8bc149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 10 monclient(hunting): picked mon.noname-b con 0x7ff8bc0c86f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff8bc1536f0 0x7ff8bc06da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 10 monclient(hunting): picked mon.noname-a con 0x7ff8bc1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc154060 con 0x7ff8bc1536f0 2026-02-17T05:56:50.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc0c5680 con 0x7ff8bc0c86f0 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc074c00 con 0x7ff8bc0700f0 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8c12e1640 10 monclient(hunting): authenticate will time out at 1063.125119s 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8bb7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff8bc1536f0 0x7ff8bc06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8ba7fc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff8bc0c86f0 0x7ff8bc149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8baffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff8bc0700f0 0x7ff8bc142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8bb7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff8bc1536f0 0x7ff8bc06da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44454/0 (socket says 10.20.193.16:44454) 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8bb7fe640 1 -- 10.20.193.16:0/1570760316 learned_addr learned my addr 10.20.193.16:0/1570760316 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8bb7fe640 10 monclient(hunting): get_auth_request con 0x7ff8bc1536f0 auth_method 0 2026-02-17T05:56:50.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8bb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8bb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8ba7fc640 10 monclient(hunting): get_auth_request con 0x7ff8bc0c86f0 auth_method 0 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8ba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8baffd640 10 monclient(hunting): get_auth_request con 0x7ff8bc0700f0 auth_method 0 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8baffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.556+0000 7ff8bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.557+0000 7ff8ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.557+0000 7ff8ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.557+0000 7ff8ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.557+0000 7ff8baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.557+0000 7ff8baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.557+0000 7ff8baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.558+0000 7ff8baffd640 10 monclient(hunting): handle_auth_done global_id 4637 payload 824 2026-02-17T05:56:50.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.558+0000 7ff8baffd640 1 -- 10.20.193.16:0/1570760316 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff8bc0c86f0 msgr2=0x7ff8bc149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.558+0000 7ff8baffd640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff8bc0c86f0 0x7ff8bc149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.558+0000 7ff8baffd640 1 -- 10.20.193.16:0/1570760316 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff8bc1536f0 msgr2=0x7ff8bc06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.558+0000 7ff8baffd640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff8bc1536f0 0x7ff8bc06da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.558+0000 7ff8baffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.558+0000 7ff8baffd640 1 monclient: found mon.noname-c 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.558+0000 7ff8baffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.558+0000 7ff8baffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.558+0000 7ff8baffd640 1 -- 10.20.193.16:0/1570760316 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc140f80 con 0x7ff8bc0700f0 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.559+0000 7ff8baffd640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff8bc0700f0 0x7ff8bc142300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c4e0 tx=0x7ff8b000c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.559+0000 7ff89b7fe640 10 client.?.objecter ms_handle_connect 0x7ff8bc0700f0 2026-02-17T05:56:50.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.559+0000 7ff89b7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:50.576 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff89b7fe640 1 -- 10.20.193.16:0/1570760316 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00052d0 con 0x7ff8bc0700f0 2026-02-17T05:56:50.576 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff89b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.576 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff89b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff89b7fe640 10 monclient: dump: 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.577 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff89b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff89b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff89b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff8c12e1640 5 monclient: authenticate success, global_id 4637 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff8c12e1640 10 monclient: _renew_subs 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff8c12e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff8c12e1640 1 -- 10.20.193.16:0/1570760316 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8bc141260 con 0x7ff8bc0700f0 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff8c12e1640 10 client.4637.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff8c12e1640 10 monclient: _renew_subs 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff8c12e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff8c12e1640 1 -- 10.20.193.16:0/1570760316 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8bc141820 con 0x7ff8bc0700f0 2026-02-17T05:56:50.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.560+0000 7ff8c12e1640 1 librados: init done 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff89b7fe640 1 -- 10.20.193.16:0/1570760316 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b0005470 con 0x7ff8bc0700f0 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff89b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff89b7fe640 1 -- 10.20.193.16:0/1570760316 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00038f0 con 0x7ff8bc0700f0 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff89b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff89b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff8b97fa640 4 set_mon_vals no callback set 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff89b7fe640 10 monclient: dump: 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.579 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff8c12e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff8c12e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff8c12e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.561+0000 7ff8c12e1640 1 -- 10.20.193.16:0/1570760316 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8bc0704d0 con 0x7ff8bc0700f0 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.572+0000 7ff89b7fe640 1 -- 10.20.193.16:0/1570760316 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7ff8b0003a90 con 0x7ff8bc0700f0 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.572+0000 7ff89b7fe640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff888079140 0x7ff88807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.572+0000 7ff8ba7fc640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff888079140 0x7ff88807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.572+0000 7ff8ba7fc640 10 monclient: get_auth_request con 0x7ff888079140 auth_method 0 2026-02-17T05:56:50.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.573+0000 7ff89b7fe640 10 client.4637.objecter ms_handle_connect 0x7ff888079140 2026-02-17T05:56:50.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.573+0000 7ff8ba7fc640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff888079140 0x7ff88807b600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0c4420 tx=0x7ff8a4009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.573+0000 7ff89b7fe640 1 -- 10.20.193.16:0/1570760316 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7ff8b0097c80 con 0x7ff8bc0700f0 2026-02-17T05:56:50.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.573+0000 7ff89b7fe640 10 client.4637.objecter ms_dispatch2 0x7ff8bc000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:50.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.573+0000 7ff89b7fe640 3 client.4637.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:50.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.573+0000 7ff89b7fe640 3 client.4637.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:50.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.573+0000 7ff89b7fe640 20 client.4637.objecter dump_active .. 0 homeless 2026-02-17T05:56:50.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.673+0000 7ff89b7fe640 1 -- 10.20.193.16:0/1570760316 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff8b0062230 con 0x7ff8bc0700f0 2026-02-17T05:56:50.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.673+0000 7ff89b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.673+0000 7ff89b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:50.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.756+0000 7ff8c12e1640 1 -- 10.20.193.16:0/1570760316 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8bc06dfc0 con 0x7ff888079140 2026-02-17T05:56:50.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.757+0000 7ff89b7fe640 1 -- 10.20.193.16:0/1570760316 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+45054 (secure 0 0 0) 0x7ff8bc06dfc0 con 0x7ff888079140 2026-02-17T05:56:50.759 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:56:50.761 INFO:teuthology.orchestra.run.trial016.stderr:dumped all 2026-02-17T05:56:50.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 10 librados: watch_flush enter 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 10 librados: watch_flush exit 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 20 client.4637.objecter shutdown clearing up homeless session... 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 10 client.4637.objecter successfully canceled tick 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 -- 10.20.193.16:0/1570760316 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff888079140 msgr2=0x7ff88807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff888079140 0x7ff88807b600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0c4420 tx=0x7ff8a4009290 comp rx=0 tx=0).stop 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 10 monclient: shutdown 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 -- 10.20.193.16:0/1570760316 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff8bc0700f0 msgr2=0x7ff8bc142300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff8bc0700f0 0x7ff8bc142300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c4e0 tx=0x7ff8b000c9b0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 -- 10.20.193.16:0/1570760316 shutdown_connections 2026-02-17T05:56:50.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff888079140 0x7ff88807b600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff8bc0c86f0 0x7ff8bc149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff8bc1536f0 0x7ff8bc06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 --2- 10.20.193.16:0/1570760316 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff8bc0700f0 0x7ff8bc142300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 -- 10.20.193.16:0/1570760316 >> 10.20.193.16:0/1570760316 conn(0x7ff8bc0cd770 msgr2=0x7ff8bc14fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 -- 10.20.193.16:0/1570760316 shutdown_connections 2026-02-17T05:56:50.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 -- 10.20.193.16:0/1570760316 wait complete. 2026-02-17T05:56:50.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.759+0000 7ff8c12e1640 1 librados: shutdown 2026-02-17T05:56:50.768 INFO:teuthology.orchestra.run.trial016.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2026-02-17T05:56:49.879715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437812,"kb_used_data":6232,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509511628,"statfs":{"total":1546188226560,"available":1545739907072,"internally_reserved":0,"allocated":6381568,"data_stored":3249102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25505,"internal_metadata":441424991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131107,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":18,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.978658"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:45.946554+0000","last_change":"2026-02-17T05:56:45.946554+0000","last_active":"2026-02-17T05:56:45.946554+0000","last_peered":"2026-02-17T05:56:45.946554+0000","last_clean":"2026-02-17T05:56:45.946554+0000","last_became_active":"2026-02-17T05:56:43.852478+0000","last_became_peered":"2026-02-17T05:56:43.852478+0000","last_unstale":"2026-02-17T05:56:45.946554+0000","last_undegraded":"2026-02-17T05:56:45.946554+0000","last_fullsized":"2026-02-17T05:56:45.946554+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T08:21:55.695169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031385049999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:46.676975+0000","last_change":"2026-02-17T05:56:46.700244+0000","last_active":"2026-02-17T05:56:46.676975+0000","last_peered":"2026-02-17T05:56:46.676975+0000","last_clean":"2026-02-17T05:56:46.676975+0000","last_became_active":"2026-02-17T05:56:43.182525+0000","last_became_peered":"2026-02-17T05:56:43.182525+0000","last_unstale":"2026-02-17T05:56:46.676975+0000","last_undegraded":"2026-02-17T05:56:46.676975+0000","last_fullsized":"2026-02-17T05:56:46.676975+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:38:44.190148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038112767999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:45.481770+0000","last_change":"2026-02-17T05:56:45.501489+0000","last_active":"2026-02-17T05:56:45.481770+0000","last_peered":"2026-02-17T05:56:45.481770+0000","last_clean":"2026-02-17T05:56:45.481770+0000","last_became_active":"2026-02-17T05:56:43.381684+0000","last_became_peered":"2026-02-17T05:56:43.381684+0000","last_unstale":"2026-02-17T05:56:45.481770+0000","last_undegraded":"2026-02-17T05:56:45.481770+0000","last_fullsized":"2026-02-17T05:56:45.481770+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:07:04.503400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031884497999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-17T05:56:44.771676+0000","last_change":"2026-02-17T05:56:43.309643+0000","last_active":"2026-02-17T05:56:44.771676+0000","last_peered":"2026-02-17T05:56:44.771676+0000","last_clean":"2026-02-17T05:56:44.771676+0000","last_became_active":"2026-02-17T05:56:43.290452+0000","last_became_peered":"2026-02-17T05:56:43.290452+0000","last_unstale":"2026-02-17T05:56:44.771676+0000","last_undegraded":"2026-02-17T05:56:44.771676+0000","last_fullsized":"2026-02-17T05:56:44.771676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T10:31:53.023851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"33'2","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:45.527362+0000","last_change":"2026-02-17T05:56:45.527362+0000","last_active":"2026-02-17T05:56:45.527362+0000","last_peered":"2026-02-17T05:56:45.527362+0000","last_clean":"2026-02-17T05:56:45.527362+0000","last_became_active":"2026-02-17T05:56:43.262979+0000","last_became_peered":"2026-02-17T05:56:43.262979+0000","last_unstale":"2026-02-17T05:56:45.527362+0000","last_undegraded":"2026-02-17T05:56:45.527362+0000","last_fullsized":"2026-02-17T05:56:45.527362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:30:08.337496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.056715723000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:46.677559+0000","last_change":"2026-02-17T05:56:46.695943+0000","last_active":"2026-02-17T05:56:46.677559+0000","last_peered":"2026-02-17T05:56:46.677559+0000","last_clean":"2026-02-17T05:56:46.677559+0000","last_became_active":"2026-02-17T05:56:43.075477+0000","last_became_peered":"2026-02-17T05:56:43.075477+0000","last_unstale":"2026-02-17T05:56:46.677559+0000","last_undegraded":"2026-02-17T05:56:46.677559+0000","last_fullsized":"2026-02-17T05:56:46.677559+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T07:20:23.191397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.030611670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:45.500912+0000","last_change":"2026-02-17T05:56:45.522106+0000","last_active":"2026-02-17T05:56:45.500912+0000","last_peered":"2026-02-17T05:56:45.500912+0000","last_clean":"2026-02-17T05:56:45.500912+0000","last_became_active":"2026-02-17T05:56:43.142816+0000","last_became_peered":"2026-02-17T05:56:43.142816+0000","last_unstale":"2026-02-17T05:56:45.500912+0000","last_undegraded":"2026-02-17T05:56:45.500912+0000","last_fullsized":"2026-02-17T05:56:45.500912+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:47:19.171322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.043200879999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"31'1","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:46.674477+0000","last_change":"2026-02-17T05:56:46.694818+0000","last_active":"2026-02-17T05:56:46.674477+0000","last_peered":"2026-02-17T05:56:46.674477+0000","last_clean":"2026-02-17T05:56:46.674477+0000","last_became_active":"2026-02-17T05:56:43.121999+0000","last_became_peered":"2026-02-17T05:56:43.121999+0000","last_unstale":"2026-02-17T05:56:46.674477+0000","last_undegraded":"2026-02-17T05:56:46.674477+0000","last_fullsized":"2026-02-17T05:56:46.674477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T17:12:50.512061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.034338280999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"32'32","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:46.658317+0000","last_change":"2026-02-17T05:56:37.927084+0000","last_active":"2026-02-17T05:56:46.658317+0000","last_peered":"2026-02-17T05:56:46.658317+0000","last_clean":"2026-02-17T05:56:46.658317+0000","last_became_active":"2026-02-17T05:56:37.919763+0000","last_became_peered":"2026-02-17T05:56:37.919763+0000","last_unstale":"2026-02-17T05:56:46.658317+0000","last_undegraded":"2026-02-17T05:56:46.658317+0000","last_fullsized":"2026-02-17T05:56:46.658317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:30.809269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:30.809269+0000","last_clean_scrub_stamp":"2026-02-17T05:56:30.809269+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:55:23.546716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":28,"seq":120259084297,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":28,"seq":120259084296,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":27,"seq":115964116998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,3,4,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313223,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":335872,"data_stored":146183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280519,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":335872,"data_stored":146183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[4,5,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313223,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":356352,"data_stored":155740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[3,4,5,6,7,8,9,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":25,"seq":107374182410,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":356352,"data_stored":155740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[0,1,2,4,5,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27312,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,6,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":25,"seq":107374182405,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":335872,"data_stored":146183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,5,6,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":25,"seq":107374182405,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":753664,"data_stored":553561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,4,5,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":27,"seq":115964117001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":753664,"data_stored":553561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":27,"seq":115964117001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27292,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":335872,"data_stored":146183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[1,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T05:56:50.770 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T05:56:50.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.836+0000 7fd6c340c640 10 monclient: get_monmap_and_config 2026-02-17T05:56:50.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.836+0000 7fd6c340c640 10 monclient: build_initial_monmap 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.836+0000 7fd6c340c640 10 monclient: monmap: 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:50.837202+0000 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:50.837202+0000 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:50.837 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:50.838 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:50.838 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.836+0000 7fd6c340c640 1 Processor -- start 2026-02-17T05:56:50.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.836+0000 7fd6c340c640 1 -- start start 2026-02-17T05:56:50.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.836+0000 7fd6c340c640 10 monclient: init 2026-02-17T05:56:50.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 10 monclient: _add_conns ranks=[3,1,0,5,2,4] 2026-02-17T05:56:50.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fd6bc0700f0 0x7fd6bc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 10 monclient(hunting): picked mon.noname-a con 0x7fd6bc0700f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fd6bc0c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd6bc1536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6bc0c4820 con 0x7fd6bc1536f0 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6bc0c4b00 con 0x7fd6bc0c7fe0 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd6bc0c4de0 con 0x7fd6bc0700f0 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c340c640 10 monclient(hunting): authenticate will time out at 1063.406256s 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c0980640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fd6bc0c7fe0 0x7fd6bc0c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:58560/0 (socket says 10.20.193.16:58560) 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c0980640 1 -- 10.20.193.16:0/2594687140 learned_addr learned my addr 10.20.193.16:0/2594687140 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.837+0000 7fd6c1181640 1 --2- 10.20.193.16:0/2594687140 >> v2:10.20.193.16:3300/0 conn(0x7fd6bc0700f0 0x7fd6bc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.838+0000 7fd6c1181640 10 monclient(hunting): get_auth_request con 0x7fd6bc0700f0 auth_method 0 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.838+0000 7fd6c1181640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.838+0000 7fd6c1181640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.838+0000 7fd6c1181640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.838+0000 7fd6c1181640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.838+0000 7fd6c1181640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.838+0000 7fd6c1181640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6abfff640 1 -- 10.20.193.16:0/2594687140 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365927087 0 0) 0x7fd6bc0c4b00 con 0x7fd6bc0c7fe0 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6abfff640 10 monclient(hunting): _negotiate 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6abfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6abfff640 10 monclient(hunting): my global_id is 4582 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6abfff640 1 -- 10.20.193.16:0/2594687140 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6a4003620 con 0x7fd6bc0c7fe0 2026-02-17T05:56:50.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6c1181640 10 monclient(hunting): handle_auth_done global_id 4512 payload 274 2026-02-17T05:56:50.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6c1181640 1 -- 10.20.193.16:0/2594687140 >> v1:10.20.193.128:6789/0 conn(0x7fd6bc0c7fe0 legacy=0x7fd6bc0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6c1181640 1 -- 10.20.193.16:0/2594687140 >> v1:10.20.193.16:6789/0 conn(0x7fd6bc1536f0 legacy=0x7fd6bc153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6c1181640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6c1181640 1 monclient: found mon.noname-a 2026-02-17T05:56:50.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6c1181640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6c1181640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.839+0000 7fd6c1181640 1 -- 10.20.193.16:0/2594687140 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc0c50c0 con 0x7fd6bc0700f0 2026-02-17T05:56:50.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.841+0000 7fd6c1181640 1 --2- 10.20.193.16:0/2594687140 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd6bc0700f0 0x7fd6bc0704d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0098a0 tx=0x7fd6ac02edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc298e6a27e77f14 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 1 -- 10.20.193.16:0/2594687140 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac03c070 con 0x7fd6bc0700f0 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 10 monclient: dump: 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.848 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 1 -- 10.20.193.16:0/2594687140 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ac02fae0 con 0x7fd6bc0700f0 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 5 monclient: authenticate success, global_id 4512 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 1 -- 10.20.193.16:0/2594687140 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac037440 con 0x7fd6bc0700f0 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6abfff640 10 monclient: dump: 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.849 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 4 set_mon_vals no callback set 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 10 monclient: shutdown 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 1 -- 10.20.193.16:0/2594687140 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd6bc0700f0 msgr2=0x7fd6bc0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 1 --2- 10.20.193.16:0/2594687140 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd6bc0700f0 0x7fd6bc0704d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0098a0 tx=0x7fd6ac02edd0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 1 -- 10.20.193.16:0/2594687140 shutdown_connections 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 1 --2- 10.20.193.16:0/2594687140 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd6bc0700f0 0x7fd6bc0704d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 1 -- 10.20.193.16:0/2594687140 >> 10.20.193.16:0/2594687140 conn(0x7fd6bc0cd770 msgr2=0x7fd6bc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 1 -- 10.20.193.16:0/2594687140 shutdown_connections 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.844+0000 7fd6c340c640 1 -- 10.20.193.16:0/2594687140 wait complete. 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient: build_initial_monmap 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient: monmap: 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:50.851 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 librados: starting msgr at 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 librados: starting objecter 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 Processor -- start 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 -- start start 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 librados: setting wanted keys 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 librados: calling monclient init 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient: init 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:50.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd6bc0700f0 0x7fd6bc06edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient(hunting): picked mon.noname-a con 0x7fd6bc0700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd6bc0c7fe0 0x7fd6bc1f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient(hunting): picked mon.noname-c con 0x7fd6bc0c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd6bc1536f0 0x7fd6bc06de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient(hunting): picked mon.noname-b con 0x7fd6bc1536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc074c00 con 0x7fd6bc0700f0 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc0c5680 con 0x7fd6bc1536f0 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc074d80 con 0x7fd6bc0c7fe0 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:50.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.845+0000 7fd6c340c640 10 monclient(hunting): authenticate will time out at 1063.414908s 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1181640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd6bc0700f0 0x7fd6bc06edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1982640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd6bc1536f0 0x7fd6bc06de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c0980640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd6bc0c7fe0 0x7fd6bc1f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1982640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd6bc1536f0 0x7fd6bc06de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:56782/0 (socket says 10.20.193.16:56782) 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1982640 1 -- 10.20.193.16:0/1892743467 learned_addr learned my addr 10.20.193.16:0/1892743467 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1982640 10 monclient(hunting): get_auth_request con 0x7fd6bc1536f0 auth_method 0 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1982640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1982640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1982640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1181640 10 monclient(hunting): get_auth_request con 0x7fd6bc0700f0 auth_method 0 2026-02-17T05:56:50.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1181640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1181640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1181640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c0980640 10 monclient(hunting): get_auth_request con 0x7fd6bc0c7fe0 auth_method 0 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c0980640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c0980640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c0980640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1181640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1181640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1181640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1982640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1982640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c1982640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c0980640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c0980640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:50.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.846+0000 7fd6c0980640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:50.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.848+0000 7fd6c1181640 10 monclient(hunting): handle_auth_done global_id 4518 payload 824 2026-02-17T05:56:50.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.848+0000 7fd6c1181640 1 -- 10.20.193.16:0/1892743467 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd6bc0c7fe0 msgr2=0x7fd6bc1f4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.848+0000 7fd6c1181640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd6bc0c7fe0 0x7fd6bc1f4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.848+0000 7fd6c1181640 1 -- 10.20.193.16:0/1892743467 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd6bc1536f0 msgr2=0x7fd6bc06de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:50.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.848+0000 7fd6c1181640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd6bc1536f0 0x7fd6bc06de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:50.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.848+0000 7fd6c1181640 10 monclient: _finish_hunting 0 2026-02-17T05:56:50.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.848+0000 7fd6c1181640 1 monclient: found mon.noname-a 2026-02-17T05:56:50.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.848+0000 7fd6c1181640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:50.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.848+0000 7fd6c1181640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.848+0000 7fd6c1181640 1 -- 10.20.193.16:0/1892743467 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc151740 con 0x7fd6bc0700f0 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.849+0000 7fd6c1181640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd6bc0700f0 0x7fd6bc06edd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac036000 tx=0x7fd6ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.849+0000 7fd6a9ffb640 10 client.?.objecter ms_handle_connect 0x7fd6bc0700f0 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.849+0000 7fd6a9ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 1 -- 10.20.193.16:0/1892743467 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac03c040 con 0x7fd6bc0700f0 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 10 monclient: dump: 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.859 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 1 -- 10.20.193.16:0/1892743467 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ac042630 con 0x7fd6bc0700f0 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 5 monclient: authenticate success, global_id 4518 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 1 -- 10.20.193.16:0/1892743467 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac042a20 con 0x7fd6bc0700f0 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:50.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6a9ffb640 10 monclient: dump: 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6ab7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 10 monclient: _renew_subs 2026-02-17T05:56:50.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 1 -- 10.20.193.16:0/1892743467 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc151a20 con 0x7fd6bc0700f0 2026-02-17T05:56:50.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 10 client.4518.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:50.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 10 monclient: _renew_subs 2026-02-17T05:56:50.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 1 -- 10.20.193.16:0/1892743467 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc151fe0 con 0x7fd6bc0700f0 2026-02-17T05:56:50.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 1 librados: init done 2026-02-17T05:56:50.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:56:50.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.851+0000 7fd6c340c640 1 -- 10.20.193.16:0/1892743467 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6bc06f310 con 0x7fd6bc0700f0 2026-02-17T05:56:50.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.862+0000 7fd6a9ffb640 1 -- 10.20.193.16:0/1892743467 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fd6ac04b450 con 0x7fd6bc0700f0 2026-02-17T05:56:50.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.863+0000 7fd6a9ffb640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd68c0791c0 0x7fd68c07b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:50.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.863+0000 7fd6c0980640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd68c0791c0 0x7fd68c07b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:50.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.863+0000 7fd6c0980640 10 monclient: get_auth_request con 0x7fd68c0791c0 auth_method 0 2026-02-17T05:56:50.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.863+0000 7fd6c0980640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd68c0791c0 0x7fd68c07b680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc1ee2c0 tx=0x7fd6b0007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:50.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.863+0000 7fd6a9ffb640 10 client.4518.objecter ms_handle_connect 0x7fd68c0791c0 2026-02-17T05:56:50.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.865+0000 7fd6a9ffb640 1 -- 10.20.193.16:0/1892743467 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7fd6ac02fa20 con 0x7fd6bc0700f0 2026-02-17T05:56:50.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.865+0000 7fd6a9ffb640 10 client.4518.objecter ms_dispatch2 0x7fd6bc000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:50.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.865+0000 7fd6a9ffb640 3 client.4518.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:50.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.865+0000 7fd6a9ffb640 3 client.4518.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:50.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.865+0000 7fd6a9ffb640 20 client.4518.objecter dump_active .. 0 homeless 2026-02-17T05:56:50.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.965+0000 7fd6a9ffb640 1 -- 10.20.193.16:0/1892743467 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd6ac086990 con 0x7fd6bc0700f0 2026-02-17T05:56:50.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.965+0000 7fd6a9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:50.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:50.965+0000 7fd6a9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:51.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.051+0000 7fd6c340c640 1 -- 10.20.193.16:0/1892743467 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6bc0704d0 con 0x7fd68c0791c0 2026-02-17T05:56:51.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.052+0000 7fd6a9ffb640 1 -- 10.20.193.16:0/1892743467 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+45054 (secure 0 0 0) 0x7fd6bc0704d0 con 0x7fd68c0791c0 2026-02-17T05:56:51.053 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:56:51.054 INFO:teuthology.orchestra.run.trial016.stderr:dumped all 2026-02-17T05:56:51.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 10 librados: watch_flush enter 2026-02-17T05:56:51.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 10 librados: watch_flush exit 2026-02-17T05:56:51.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 20 client.4518.objecter shutdown clearing up homeless session... 2026-02-17T05:56:51.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 10 client.4518.objecter successfully canceled tick 2026-02-17T05:56:51.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 -- 10.20.193.16:0/1892743467 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd68c0791c0 msgr2=0x7fd68c07b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:51.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd68c0791c0 0x7fd68c07b680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc1ee2c0 tx=0x7fd6b0007dd0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 10 monclient: shutdown 2026-02-17T05:56:51.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:51.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 -- 10.20.193.16:0/1892743467 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd6bc0700f0 msgr2=0x7fd6bc06edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:51.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd6bc0700f0 0x7fd6bc06edd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac036000 tx=0x7fd6ac0047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 -- 10.20.193.16:0/1892743467 shutdown_connections 2026-02-17T05:56:51.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd68c0791c0 0x7fd68c07b680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd6bc1536f0 0x7fd6bc06de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd6bc0c7fe0 0x7fd6bc1f4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 --2- 10.20.193.16:0/1892743467 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd6bc0700f0 0x7fd6bc06edd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 -- 10.20.193.16:0/1892743467 >> 10.20.193.16:0/1892743467 conn(0x7fd6bc0cd770 msgr2=0x7fd6bc14e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:51.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 -- 10.20.193.16:0/1892743467 shutdown_connections 2026-02-17T05:56:51.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 -- 10.20.193.16:0/1892743467 wait complete. 2026-02-17T05:56:51.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.054+0000 7fd6c340c640 1 librados: shutdown 2026-02-17T05:56:51.063 INFO:teuthology.orchestra.run.trial016.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2026-02-17T05:56:49.879715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437812,"kb_used_data":6232,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509511628,"statfs":{"total":1546188226560,"available":1545739907072,"internally_reserved":0,"allocated":6381568,"data_stored":3249102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25505,"internal_metadata":441424991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131107,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":18,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.978658"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:45.946554+0000","last_change":"2026-02-17T05:56:45.946554+0000","last_active":"2026-02-17T05:56:45.946554+0000","last_peered":"2026-02-17T05:56:45.946554+0000","last_clean":"2026-02-17T05:56:45.946554+0000","last_became_active":"2026-02-17T05:56:43.852478+0000","last_became_peered":"2026-02-17T05:56:43.852478+0000","last_unstale":"2026-02-17T05:56:45.946554+0000","last_undegraded":"2026-02-17T05:56:45.946554+0000","last_fullsized":"2026-02-17T05:56:45.946554+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T08:21:55.695169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031385049999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:46.676975+0000","last_change":"2026-02-17T05:56:46.700244+0000","last_active":"2026-02-17T05:56:46.676975+0000","last_peered":"2026-02-17T05:56:46.676975+0000","last_clean":"2026-02-17T05:56:46.676975+0000","last_became_active":"2026-02-17T05:56:43.182525+0000","last_became_peered":"2026-02-17T05:56:43.182525+0000","last_unstale":"2026-02-17T05:56:46.676975+0000","last_undegraded":"2026-02-17T05:56:46.676975+0000","last_fullsized":"2026-02-17T05:56:46.676975+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:38:44.190148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038112767999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:45.481770+0000","last_change":"2026-02-17T05:56:45.501489+0000","last_active":"2026-02-17T05:56:45.481770+0000","last_peered":"2026-02-17T05:56:45.481770+0000","last_clean":"2026-02-17T05:56:45.481770+0000","last_became_active":"2026-02-17T05:56:43.381684+0000","last_became_peered":"2026-02-17T05:56:43.381684+0000","last_unstale":"2026-02-17T05:56:45.481770+0000","last_undegraded":"2026-02-17T05:56:45.481770+0000","last_fullsized":"2026-02-17T05:56:45.481770+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:07:04.503400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031884497999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-17T05:56:44.771676+0000","last_change":"2026-02-17T05:56:43.309643+0000","last_active":"2026-02-17T05:56:44.771676+0000","last_peered":"2026-02-17T05:56:44.771676+0000","last_clean":"2026-02-17T05:56:44.771676+0000","last_became_active":"2026-02-17T05:56:43.290452+0000","last_became_peered":"2026-02-17T05:56:43.290452+0000","last_unstale":"2026-02-17T05:56:44.771676+0000","last_undegraded":"2026-02-17T05:56:44.771676+0000","last_fullsized":"2026-02-17T05:56:44.771676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T10:31:53.023851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"33'2","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:45.527362+0000","last_change":"2026-02-17T05:56:45.527362+0000","last_active":"2026-02-17T05:56:45.527362+0000","last_peered":"2026-02-17T05:56:45.527362+0000","last_clean":"2026-02-17T05:56:45.527362+0000","last_became_active":"2026-02-17T05:56:43.262979+0000","last_became_peered":"2026-02-17T05:56:43.262979+0000","last_unstale":"2026-02-17T05:56:45.527362+0000","last_undegraded":"2026-02-17T05:56:45.527362+0000","last_fullsized":"2026-02-17T05:56:45.527362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:30:08.337496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.056715723000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:46.677559+0000","last_change":"2026-02-17T05:56:46.695943+0000","last_active":"2026-02-17T05:56:46.677559+0000","last_peered":"2026-02-17T05:56:46.677559+0000","last_clean":"2026-02-17T05:56:46.677559+0000","last_became_active":"2026-02-17T05:56:43.075477+0000","last_became_peered":"2026-02-17T05:56:43.075477+0000","last_unstale":"2026-02-17T05:56:46.677559+0000","last_undegraded":"2026-02-17T05:56:46.677559+0000","last_fullsized":"2026-02-17T05:56:46.677559+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T07:20:23.191397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.030611670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:45.500912+0000","last_change":"2026-02-17T05:56:45.522106+0000","last_active":"2026-02-17T05:56:45.500912+0000","last_peered":"2026-02-17T05:56:45.500912+0000","last_clean":"2026-02-17T05:56:45.500912+0000","last_became_active":"2026-02-17T05:56:43.142816+0000","last_became_peered":"2026-02-17T05:56:43.142816+0000","last_unstale":"2026-02-17T05:56:45.500912+0000","last_undegraded":"2026-02-17T05:56:45.500912+0000","last_fullsized":"2026-02-17T05:56:45.500912+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:47:19.171322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.043200879999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"31'1","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:46.674477+0000","last_change":"2026-02-17T05:56:46.694818+0000","last_active":"2026-02-17T05:56:46.674477+0000","last_peered":"2026-02-17T05:56:46.674477+0000","last_clean":"2026-02-17T05:56:46.674477+0000","last_became_active":"2026-02-17T05:56:43.121999+0000","last_became_peered":"2026-02-17T05:56:43.121999+0000","last_unstale":"2026-02-17T05:56:46.674477+0000","last_undegraded":"2026-02-17T05:56:46.674477+0000","last_fullsized":"2026-02-17T05:56:46.674477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T17:12:50.512061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.034338280999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"32'32","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-17T05:56:46.658317+0000","last_change":"2026-02-17T05:56:37.927084+0000","last_active":"2026-02-17T05:56:46.658317+0000","last_peered":"2026-02-17T05:56:46.658317+0000","last_clean":"2026-02-17T05:56:46.658317+0000","last_became_active":"2026-02-17T05:56:37.919763+0000","last_became_peered":"2026-02-17T05:56:37.919763+0000","last_unstale":"2026-02-17T05:56:46.658317+0000","last_undegraded":"2026-02-17T05:56:46.658317+0000","last_fullsized":"2026-02-17T05:56:46.658317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:30.809269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:30.809269+0000","last_clean_scrub_stamp":"2026-02-17T05:56:30.809269+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:55:23.546716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":28,"seq":120259084297,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":28,"seq":120259084296,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":27,"seq":115964116998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,3,4,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313223,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":335872,"data_stored":146183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280519,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":335872,"data_stored":146183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[4,5,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313223,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":356352,"data_stored":155740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[3,4,5,6,7,8,9,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":25,"seq":107374182410,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":356352,"data_stored":155740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[0,1,2,4,5,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27312,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":352256,"data_stored":155721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,6,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":25,"seq":107374182405,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":335872,"data_stored":146183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,5,6,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":25,"seq":107374182405,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":753664,"data_stored":553561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,4,5,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":27,"seq":115964117001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":753664,"data_stored":553561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":27,"seq":115964117001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27292,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":335872,"data_stored":146183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[1,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T05:56:51.065 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-17T05:56:51.065 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-17T05:56:51.065 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-17T05:56:51.066 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:56:51.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.132+0000 7f35d80c7640 10 monclient: get_monmap_and_config 2026-02-17T05:56:51.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.132+0000 7f35d80c7640 10 monclient: build_initial_monmap 2026-02-17T05:56:51.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.132+0000 7f35d80c7640 10 monclient: monmap: 2026-02-17T05:56:51.133 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:51.133 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:51.133 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:51.133164+0000 2026-02-17T05:56:51.133 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:51.133164+0000 2026-02-17T05:56:51.133 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.132+0000 7f35d80c7640 1 Processor -- start 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.132+0000 7f35d80c7640 1 -- start start 2026-02-17T05:56:51.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.132+0000 7f35d80c7640 10 monclient: init 2026-02-17T05:56:51.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:51.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 10 monclient: _add_conns ranks=[0,1,5,3,4,2] 2026-02-17T05:56:51.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f35d00700f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:51.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f35d00c86f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:56:51.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f35d0153e00 0x7f35d00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:51.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 10 monclient(hunting): picked mon.noname-c con 0x7f35d0153e00 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:51.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d00c4820 con 0x7f35d00700f0 2026-02-17T05:56:51.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d00c4b00 con 0x7f35d00c86f0 2026-02-17T05:56:51.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:51.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f35d00c4de0 con 0x7f35d0153e00 2026-02-17T05:56:51.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:51.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d80c7640 10 monclient(hunting): authenticate will time out at 1063.702137s 2026-02-17T05:56:51.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d563b640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f35d00c86f0 0x7f35d01536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:58570/0 (socket says 10.20.193.16:58570) 2026-02-17T05:56:51.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d563b640 1 -- 10.20.193.16:0/2998069150 learned_addr learned my addr 10.20.193.16:0/2998069150 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:51.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d663d640 1 --2- 10.20.193.16:0/2998069150 >> v2:10.20.193.148:3300/0 conn(0x7f35d0153e00 0x7f35d00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:51.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d663d640 10 monclient(hunting): get_auth_request con 0x7f35d0153e00 auth_method 0 2026-02-17T05:56:51.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d663d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:51.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d663d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:51.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.133+0000 7f35d663d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:51.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d663d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:51.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d663d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:51.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d663d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:51.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 1 -- 10.20.193.16:0/2998069150 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379759483 0 0) 0x7f35d00c4b00 con 0x7f35d00c86f0 2026-02-17T05:56:51.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 10 monclient(hunting): _negotiate 2026-02-17T05:56:51.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:51.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:51.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 10 monclient(hunting): my global_id is 4588 2026-02-17T05:56:51.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 1 -- 10.20.193.16:0/2998069150 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35b8003620 con 0x7f35d00c86f0 2026-02-17T05:56:51.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 1 -- 10.20.193.16:0/2998069150 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 250994518 0 0) 0x7f35d00c4820 con 0x7f35d00700f0 2026-02-17T05:56:51.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 10 monclient(hunting): _negotiate 2026-02-17T05:56:51.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:51.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:51.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 10 monclient(hunting): my global_id is 4521 2026-02-17T05:56:51.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.134+0000 7f35d4e3a640 1 -- 10.20.193.16:0/2998069150 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35d00c4b00 con 0x7f35d00700f0 2026-02-17T05:56:51.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.135+0000 7f35d663d640 10 monclient(hunting): handle_auth_done global_id 4643 payload 274 2026-02-17T05:56:51.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.135+0000 7f35d663d640 1 -- 10.20.193.16:0/2998069150 >> v1:10.20.193.128:6789/0 conn(0x7f35d00c86f0 legacy=0x7f35d01536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:51.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.135+0000 7f35d663d640 1 -- 10.20.193.16:0/2998069150 >> v1:10.20.193.16:6789/0 conn(0x7f35d00700f0 legacy=0x7f35d00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:51.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.135+0000 7f35d663d640 10 monclient: _finish_hunting 0 2026-02-17T05:56:51.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.135+0000 7f35d663d640 1 monclient: found mon.noname-c 2026-02-17T05:56:51.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.135+0000 7f35d663d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:51.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.135+0000 7f35d663d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:51.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.135+0000 7f35d663d640 1 -- 10.20.193.16:0/2998069150 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d00c50c0 con 0x7f35d0153e00 2026-02-17T05:56:51.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.135+0000 7f35d663d640 1 --2- 10.20.193.16:0/2998069150 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35d0153e00 0x7f35d00704d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009d90 tx=0x7f35cc02f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f88632b282db823 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:51.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 1 -- 10.20.193.16:0/2998069150 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc03c070 con 0x7f35d0153e00 2026-02-17T05:56:51.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:51.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:51.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 10 monclient: dump: 2026-02-17T05:56:51.160 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:51.160 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:51.160 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:51.160 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:51.160 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:51.160 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:51.161 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:51.161 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:51.161 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:51.161 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:51.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 10 monclient: _finish_auth 0 2026-02-17T05:56:51.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 10 monclient: _check_auth_tickets 2026-02-17T05:56:51.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:51.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 5 monclient: authenticate success, global_id 4643 2026-02-17T05:56:51.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:51.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 1 -- 10.20.193.16:0/2998069150 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35cc037440 con 0x7f35d0153e00 2026-02-17T05:56:51.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:51.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 1 -- 10.20.193.16:0/2998069150 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0377f0 con 0x7f35d0153e00 2026-02-17T05:56:51.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:51.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 4 set_mon_vals no callback set 2026-02-17T05:56:51.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:51.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:51.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d4e3a640 10 monclient: dump: 2026-02-17T05:56:51.164 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:51.164 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:51.164 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:51.164 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:51.164 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:51.164 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:51.165 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:51.165 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:51.165 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:51.165 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:51.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 10 monclient: shutdown 2026-02-17T05:56:51.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:51.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 1 -- 10.20.193.16:0/2998069150 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35d0153e00 msgr2=0x7f35d00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:51.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 1 --2- 10.20.193.16:0/2998069150 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35d0153e00 0x7f35d00704d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009d90 tx=0x7f35cc02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 1 -- 10.20.193.16:0/2998069150 shutdown_connections 2026-02-17T05:56:51.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 1 --2- 10.20.193.16:0/2998069150 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35d0153e00 0x7f35d00704d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.137+0000 7f35d80c7640 1 -- 10.20.193.16:0/2998069150 >> 10.20.193.16:0/2998069150 conn(0x7f35d00cd770 msgr2=0x7f35d00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:51.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 1 -- 10.20.193.16:0/2998069150 shutdown_connections 2026-02-17T05:56:51.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 1 -- 10.20.193.16:0/2998069150 wait complete. 2026-02-17T05:56:51.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 10 monclient: build_initial_monmap 2026-02-17T05:56:51.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 10 monclient: monmap: 2026-02-17T05:56:51.167 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:51.167 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:51.167 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:51.167 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:51.168 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:51.168 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:51.168 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:51.168 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:51.168 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:51.168 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:51.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 1 librados: starting msgr at 2026-02-17T05:56:51.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 1 librados: starting objecter 2026-02-17T05:56:51.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 1 Processor -- start 2026-02-17T05:56:51.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 1 -- start start 2026-02-17T05:56:51.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 1 librados: setting wanted keys 2026-02-17T05:56:51.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 1 librados: calling monclient init 2026-02-17T05:56:51.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.138+0000 7f35d80c7640 10 monclient: init 2026-02-17T05:56:51.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:51.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:51.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f35d00700f0 0x7f35d0151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:51.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 10 monclient(hunting): picked mon.noname-a con 0x7f35d00700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:51.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35d00c86f0 0x7f35d013c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:51.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 10 monclient(hunting): picked mon.noname-c con 0x7f35d00c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:51.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f35d0153e00 0x7f35d0141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:51.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 10 monclient(hunting): picked mon.noname-b con 0x7f35d0153e00 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:51.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:51.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35d0074c00 con 0x7f35d00700f0 2026-02-17T05:56:51.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:51.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f35d00c5680 con 0x7f35d0153e00 2026-02-17T05:56:51.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:51.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35d0074d80 con 0x7f35d00c86f0 2026-02-17T05:56:51.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:51.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d80c7640 10 monclient(hunting): authenticate will time out at 1063.708303s 2026-02-17T05:56:51.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d5e3c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f35d00700f0 0x7f35d0151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:51.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d663d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f35d0153e00 0x7f35d0141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:51.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d563b640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35d00c86f0 0x7f35d013c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:51.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d5e3c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f35d00700f0 0x7f35d0151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44484/0 (socket says 10.20.193.16:44484) 2026-02-17T05:56:51.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d5e3c640 1 -- 10.20.193.16:0/1679167800 learned_addr learned my addr 10.20.193.16:0/1679167800 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:51.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d5e3c640 10 monclient(hunting): get_auth_request con 0x7f35d00700f0 auth_method 0 2026-02-17T05:56:51.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d5e3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:51.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d5e3c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:51.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d5e3c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:51.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d563b640 10 monclient(hunting): get_auth_request con 0x7f35d00c86f0 auth_method 0 2026-02-17T05:56:51.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d563b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:51.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d563b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:51.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d563b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:51.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d663d640 10 monclient(hunting): get_auth_request con 0x7f35d0153e00 auth_method 0 2026-02-17T05:56:51.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d663d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:51.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d663d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:51.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.139+0000 7f35d663d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:51.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.140+0000 7f35d5e3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:51.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.140+0000 7f35d5e3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:51.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.140+0000 7f35d5e3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:51.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.140+0000 7f35d563b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:51.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.140+0000 7f35d563b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:51.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.140+0000 7f35d563b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:51.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.140+0000 7f35d663d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:51.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.140+0000 7f35d663d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:51.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.140+0000 7f35d663d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:51.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.141+0000 7f35d563b640 10 monclient(hunting): handle_auth_done global_id 4646 payload 824 2026-02-17T05:56:51.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.141+0000 7f35d563b640 1 -- 10.20.193.16:0/1679167800 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f35d0153e00 msgr2=0x7f35d0141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:51.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.141+0000 7f35d563b640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f35d0153e00 0x7f35d0141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.141+0000 7f35d563b640 1 -- 10.20.193.16:0/1679167800 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f35d00700f0 msgr2=0x7f35d0151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:51.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.141+0000 7f35d563b640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f35d00700f0 0x7f35d0151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.141+0000 7f35d563b640 10 monclient: _finish_hunting 0 2026-02-17T05:56:51.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.141+0000 7f35d563b640 1 monclient: found mon.noname-c 2026-02-17T05:56:51.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.141+0000 7f35d563b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:51.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.141+0000 7f35d563b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:51.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.141+0000 7f35d563b640 1 -- 10.20.193.16:0/1679167800 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d01517e0 con 0x7f35d00c86f0 2026-02-17T05:56:51.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.142+0000 7f35d563b640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35d00c86f0 0x7f35d013c6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35c400c480 tx=0x7f35c400c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:51.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.142+0000 7f35be7fc640 10 client.?.objecter ms_handle_connect 0x7f35d00c86f0 2026-02-17T05:56:51.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.142+0000 7f35be7fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:51.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 1 -- 10.20.193.16:0/1679167800 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4017070 con 0x7f35d00c86f0 2026-02-17T05:56:51.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:51.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:51.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 10 monclient: dump: 2026-02-17T05:56:51.177 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:51.177 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:51.177 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:51.177 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:51.177 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:51.177 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:51.178 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:51.178 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:51.178 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:51.178 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:51.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:51.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:51.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:51.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35d80c7640 5 monclient: authenticate success, global_id 4646 2026-02-17T05:56:51.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35d80c7640 10 monclient: _renew_subs 2026-02-17T05:56:51.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35d80c7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35d80c7640 1 -- 10.20.193.16:0/1679167800 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d0151ac0 con 0x7f35d00c86f0 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35d80c7640 10 client.4646.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35d80c7640 10 monclient: _renew_subs 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35d80c7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35d80c7640 1 -- 10.20.193.16:0/1679167800 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d0152080 con 0x7f35d00c86f0 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35d80c7640 1 librados: init done 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 1 -- 10.20.193.16:0/1679167800 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c4005de0 con 0x7f35d00c86f0 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 1 -- 10.20.193.16:0/1679167800 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4021060 con 0x7f35d00c86f0 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35bffff640 4 set_mon_vals no callback set 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35be7fc640 10 monclient: dump: 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:51.179 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.143+0000 7f35d80c7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.144+0000 7f35d80c7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.144+0000 7f35d80c7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:51.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.144+0000 7f35d80c7640 1 -- 10.20.193.16:0/1679167800 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d00704d0 con 0x7f35d00c86f0 2026-02-17T05:56:51.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.217+0000 7f35be7fc640 1 -- 10.20.193.16:0/1679167800 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f35c4024080 con 0x7f35d00c86f0 2026-02-17T05:56:51.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.217+0000 7f35be7fc640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f35a0079140 0x7f35a007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:51.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.217+0000 7f35d5e3c640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f35a0079140 0x7f35a007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:51.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.217+0000 7f35d5e3c640 10 monclient: get_auth_request con 0x7f35a0079140 auth_method 0 2026-02-17T05:56:51.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.217+0000 7f35d5e3c640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f35a0079140 0x7f35a007b600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35c0002b80 tx=0x7f35c0002b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:51.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.217+0000 7f35be7fc640 10 client.4646.objecter ms_handle_connect 0x7f35a0079140 2026-02-17T05:56:51.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.253+0000 7f35be7fc640 1 -- 10.20.193.16:0/1679167800 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 8972+0+0 (secure 0 0 0) 0x7f35c405d700 con 0x7f35d00c86f0 2026-02-17T05:56:51.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.253+0000 7f35be7fc640 10 client.4646.objecter ms_dispatch2 0x7f35d0000b90 osd_map(33..33 src has 1..33) 2026-02-17T05:56:51.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.253+0000 7f35be7fc640 3 client.4646.objecter handle_osd_map got epochs [33,33] > 0 2026-02-17T05:56:51.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.253+0000 7f35be7fc640 3 client.4646.objecter handle_osd_map decoding full epoch 33 2026-02-17T05:56:51.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.253+0000 7f35be7fc640 20 client.4646.objecter dump_active .. 0 homeless 2026-02-17T05:56:51.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.260+0000 7f35be7fc640 1 -- 10.20.193.16:0/1679167800 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f35c40616c0 con 0x7f35d00c86f0 2026-02-17T05:56:51.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.260+0000 7f35be7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:51.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.260+0000 7f35be7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:51.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.360+0000 7f35d80c7640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:56:51.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.360+0000 7f35d80c7640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:56:51.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.360+0000 7f35d80c7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:51.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.360+0000 7f35d80c7640 1 -- 10.20.193.16:0/1679167800 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f35d01573f0 con 0x7f35d00c86f0 2026-02-17T05:56:51.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.365+0000 7f35be7fc640 1 -- 10.20.193.16:0/1679167800 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f35c40659c0 con 0x7f35d00c86f0 2026-02-17T05:56:51.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.365+0000 7f35be7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:56:51.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.365+0000 7f35be7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:51.366 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:56:51.367 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:56:51.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 10 librados: watch_flush enter 2026-02-17T05:56:51.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 10 librados: watch_flush exit 2026-02-17T05:56:51.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 20 client.4646.objecter shutdown clearing up homeless session... 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 10 client.4646.objecter successfully canceled tick 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 -- 10.20.193.16:0/1679167800 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f35a0079140 msgr2=0x7f35a007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f35a0079140 0x7f35a007b600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35c0002b80 tx=0x7f35c0002b10 comp rx=0 tx=0).stop 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 10 monclient: shutdown 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 -- 10.20.193.16:0/1679167800 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35d00c86f0 msgr2=0x7f35d013c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35d00c86f0 0x7f35d013c6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35c400c480 tx=0x7f35c400c950 comp rx=0 tx=0).stop 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 -- 10.20.193.16:0/1679167800 shutdown_connections 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f35a0079140 0x7f35a007b600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f35d0153e00 0x7f35d0141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35d00c86f0 0x7f35d013c6c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 --2- 10.20.193.16:0/1679167800 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f35d00700f0 0x7f35d0151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:51.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 -- 10.20.193.16:0/1679167800 >> 10.20.193.16:0/1679167800 conn(0x7f35d00cd770 msgr2=0x7f35d014e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:51.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 -- 10.20.193.16:0/1679167800 shutdown_connections 2026-02-17T05:56:51.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 -- 10.20.193.16:0/1679167800 wait complete. 2026-02-17T05:56:51.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:51.366+0000 7f35d80c7640 1 librados: shutdown 2026-02-17T05:56:54.376 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:56:54.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.442+0000 7ff4f433e640 10 monclient: get_monmap_and_config 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.442+0000 7ff4f433e640 10 monclient: build_initial_monmap 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.442+0000 7ff4f433e640 10 monclient: monmap: 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:54.443623+0000 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:54.443623+0000 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 1 Processor -- start 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 1 -- start start 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 10 monclient: init 2026-02-17T05:56:54.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 10 monclient: _add_conns ranks=[4,0,2,1,5,3] 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7ff4ec0700f0 0x7ff4ec0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 10 monclient(hunting): picked mon.noname-b con 0x7ff4ec0700f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff4ec0c7fe0 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff4ec1536f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ec0c4820 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ec0c4b00 con 0x7ff4ec1536f0 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ec0c4de0 con 0x7ff4ec0700f0 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.443+0000 7ff4f433e640 10 monclient(hunting): authenticate will time out at 1067.012616s 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.444+0000 7ff4f18b2640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7ff4ec0c7fe0 0x7ff4ec0c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:46358/0 (socket says 10.20.193.16:46358) 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.444+0000 7ff4f18b2640 1 -- 10.20.193.16:0/3908328035 learned_addr learned my addr 10.20.193.16:0/3908328035 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.444+0000 7ff4f20b3640 1 --2- 10.20.193.16:0/3908328035 >> v2:10.20.193.128:3300/0 conn(0x7ff4ec0700f0 0x7ff4ec0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.444+0000 7ff4f20b3640 10 monclient(hunting): get_auth_request con 0x7ff4ec0700f0 auth_method 0 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.444+0000 7ff4f20b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.444+0000 7ff4f20b3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.444+0000 7ff4f20b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.444+0000 7ff4f20b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.444+0000 7ff4f20b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:54.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.444+0000 7ff4f20b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 1 -- 10.20.193.16:0/3908328035 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3227499680 0 0) 0x7ff4ec0c4820 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 10 monclient(hunting): _negotiate 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 10 monclient(hunting): my global_id is 4527 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 1 -- 10.20.193.16:0/3908328035 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4d4003620 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 1 -- 10.20.193.16:0/3908328035 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61399449 0 0) 0x7ff4ec0c4b00 con 0x7ff4ec1536f0 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 10 monclient(hunting): _negotiate 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 10 monclient(hunting): my global_id is 4649 2026-02-17T05:56:54.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f10b1640 1 -- 10.20.193.16:0/3908328035 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4ec0c4820 con 0x7ff4ec1536f0 2026-02-17T05:56:54.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f20b3640 10 monclient(hunting): handle_auth_done global_id 4594 payload 274 2026-02-17T05:56:54.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f20b3640 1 -- 10.20.193.16:0/3908328035 >> v1:10.20.193.148:6789/0 conn(0x7ff4ec1536f0 legacy=0x7ff4ec153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:54.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f20b3640 1 -- 10.20.193.16:0/3908328035 >> v1:10.20.193.16:6789/0 conn(0x7ff4ec0c7fe0 legacy=0x7ff4ec0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:54.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f20b3640 10 monclient: _finish_hunting 0 2026-02-17T05:56:54.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f20b3640 1 monclient: found mon.noname-b 2026-02-17T05:56:54.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f20b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:54.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f20b3640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:54.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.445+0000 7ff4f20b3640 1 -- 10.20.193.16:0/3908328035 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec0c50c0 con 0x7ff4ec0700f0 2026-02-17T05:56:54.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.446+0000 7ff4f20b3640 1 --2- 10.20.193.16:0/3908328035 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0700f0 0x7ff4ec0704d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0098a0 tx=0x7ff4dc02edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ba176c59c74053b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:54.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 1 -- 10.20.193.16:0/3908328035 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc03c070 con 0x7ff4ec0700f0 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 10 monclient: dump: 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 10 monclient: _finish_auth 0 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 10 monclient: _check_auth_tickets 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 1 -- 10.20.193.16:0/3908328035 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4dc02fae0 con 0x7ff4ec0700f0 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f433e640 5 monclient: authenticate success, global_id 4594 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f433e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 1 -- 10.20.193.16:0/3908328035 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc037440 con 0x7ff4ec0700f0 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f10b1640 10 monclient: dump: 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f433e640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:54.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.448+0000 7ff4f433e640 4 set_mon_vals no callback set 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 10 monclient: shutdown 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 1 -- 10.20.193.16:0/3908328035 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0700f0 msgr2=0x7ff4ec0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 1 --2- 10.20.193.16:0/3908328035 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0700f0 0x7ff4ec0704d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0098a0 tx=0x7ff4dc02edd0 comp rx=0 tx=0).stop 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 1 -- 10.20.193.16:0/3908328035 shutdown_connections 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 1 --2- 10.20.193.16:0/3908328035 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0700f0 0x7ff4ec0704d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 1 -- 10.20.193.16:0/3908328035 >> 10.20.193.16:0/3908328035 conn(0x7ff4ec0cd770 msgr2=0x7ff4ec0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 1 -- 10.20.193.16:0/3908328035 shutdown_connections 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 1 -- 10.20.193.16:0/3908328035 wait complete. 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 10 monclient: build_initial_monmap 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 10 monclient: monmap: 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 1 librados: starting msgr at 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.449+0000 7ff4f433e640 1 librados: starting objecter 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 1 Processor -- start 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 1 -- start start 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 1 librados: setting wanted keys 2026-02-17T05:56:54.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 1 librados: calling monclient init 2026-02-17T05:56:54.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient: init 2026-02-17T05:56:54.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:54.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:54.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff4ec0700f0 0x7ff4ec06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:54.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient(hunting): picked mon.noname-c con 0x7ff4ec0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:54.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0c7fe0 0x7ff4ec1f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:54.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient(hunting): picked mon.noname-b con 0x7ff4ec0c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:54.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4ec1536f0 0x7ff4ec1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:54.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient(hunting): picked mon.noname-a con 0x7ff4ec1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:54.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec074c00 con 0x7ff4ec1536f0 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec0c5680 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec074d80 con 0x7ff4ec0700f0 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f433e640 10 monclient(hunting): authenticate will time out at 1067.019479s 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f28b4640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4ec1536f0 0x7ff4ec1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f18b2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0c7fe0 0x7ff4ec1f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f20b3640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff4ec0700f0 0x7ff4ec06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f18b2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0c7fe0 0x7ff4ec1f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:56828/0 (socket says 10.20.193.16:56828) 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f18b2640 1 -- 10.20.193.16:0/2378627649 learned_addr learned my addr 10.20.193.16:0/2378627649 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f18b2640 10 monclient(hunting): get_auth_request con 0x7ff4ec0c7fe0 auth_method 0 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f18b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f18b2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f18b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f28b4640 10 monclient(hunting): get_auth_request con 0x7ff4ec1536f0 auth_method 0 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f28b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f28b4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.450+0000 7ff4f28b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f20b3640 10 monclient(hunting): get_auth_request con 0x7ff4ec0700f0 auth_method 0 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f20b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f20b3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f20b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f18b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f18b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f18b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f28b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f28b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f28b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f20b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f20b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:54.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.451+0000 7ff4f20b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.452+0000 7ff4f18b2640 10 monclient(hunting): handle_auth_done global_id 4597 payload 824 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.452+0000 7ff4f18b2640 1 -- 10.20.193.16:0/2378627649 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff4ec0700f0 msgr2=0x7ff4ec06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.452+0000 7ff4f18b2640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff4ec0700f0 0x7ff4ec06e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.452+0000 7ff4f18b2640 1 -- 10.20.193.16:0/2378627649 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4ec1536f0 msgr2=0x7ff4ec1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.452+0000 7ff4f18b2640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4ec1536f0 0x7ff4ec1f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.452+0000 7ff4f18b2640 10 monclient: _finish_hunting 0 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.452+0000 7ff4f18b2640 1 monclient: found mon.noname-b 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.452+0000 7ff4f18b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.452+0000 7ff4f18b2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.452+0000 7ff4f18b2640 1 -- 10.20.193.16:0/2378627649 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec1f24c0 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.453+0000 7ff4f18b2640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0c7fe0 0x7ff4ec1f3c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000c520 tx=0x7ff4e000c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.453+0000 7ff4d2ffd640 10 client.?.objecter ms_handle_connect 0x7ff4ec0c7fe0 2026-02-17T05:56:54.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.453+0000 7ff4d2ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:54.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4d2ffd640 1 -- 10.20.193.16:0/2378627649 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0017070 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4d2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4d2ffd640 10 monclient: dump: 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4d2ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4d2ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 5 monclient: authenticate success, global_id 4597 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 10 monclient: _renew_subs 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 1 -- 10.20.193.16:0/2378627649 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ec1f27a0 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 10 client.4597.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:54.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 10 monclient: _renew_subs 2026-02-17T05:56:54.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:54.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 1 -- 10.20.193.16:0/2378627649 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ec1f2d60 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 1 librados: init done 2026-02-17T05:56:54.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:54.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.454+0000 7ff4f433e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:54.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.455+0000 7ff4f433e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.455+0000 7ff4f433e640 1 -- 10.20.193.16:0/2378627649 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ec0704d0 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.455+0000 7ff4d2ffd640 1 -- 10.20.193.16:0/2378627649 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e0005de0 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.455+0000 7ff4d2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.455+0000 7ff4d2ffd640 1 -- 10.20.193.16:0/2378627649 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0021060 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.455+0000 7ff4d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.455+0000 7ff4d2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.455+0000 7ff4f08b0640 4 set_mon_vals no callback set 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.455+0000 7ff4d2ffd640 10 monclient: dump: 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:54.457 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:54.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.561+0000 7ff4d2ffd640 1 -- 10.20.193.16:0/2378627649 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7ff4e0024080 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.561+0000 7ff4d2ffd640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff4bc079170 0x7ff4bc07b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:54.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.561+0000 7ff4f20b3640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff4bc079170 0x7ff4bc07b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:54.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.561+0000 7ff4f20b3640 10 monclient: get_auth_request con 0x7ff4bc079170 auth_method 0 2026-02-17T05:56:54.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.562+0000 7ff4f20b3640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff4bc079170 0x7ff4bc07b630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc039000 tx=0x7ff4dc033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:54.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.562+0000 7ff4d2ffd640 10 client.4597.objecter ms_handle_connect 0x7ff4bc079170 2026-02-17T05:56:54.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.562+0000 7ff4d2ffd640 1 -- 10.20.193.16:0/2378627649 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7ff4e00971a0 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.562+0000 7ff4d2ffd640 10 client.4597.objecter ms_dispatch2 0x7ff4ec000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:56:54.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.562+0000 7ff4d2ffd640 3 client.4597.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:56:54.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.562+0000 7ff4d2ffd640 3 client.4597.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:56:54.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.562+0000 7ff4d2ffd640 20 client.4597.objecter dump_active .. 0 homeless 2026-02-17T05:56:54.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.569+0000 7ff4d2ffd640 1 -- 10.20.193.16:0/2378627649 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff4e00616c0 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.569+0000 7ff4d2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:54.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.569+0000 7ff4d2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:54.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.669+0000 7ff4f433e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:56:54.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.669+0000 7ff4f433e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:56:54.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.669+0000 7ff4f433e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:56:54.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.669+0000 7ff4f433e640 1 -- 10.20.193.16:0/2378627649 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff4ec1573f0 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.672+0000 7ff4d2ffd640 1 -- 10.20.193.16:0/2378627649 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff4e00659c0 con 0x7ff4ec0c7fe0 2026-02-17T05:56:54.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.672+0000 7ff4d2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:56:54.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.672+0000 7ff4d2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:54.674 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:56:54.674 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:56:54.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 10 librados: watch_flush enter 2026-02-17T05:56:54.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 10 librados: watch_flush exit 2026-02-17T05:56:54.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 20 client.4597.objecter shutdown clearing up homeless session... 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 10 client.4597.objecter successfully canceled tick 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 -- 10.20.193.16:0/2378627649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff4bc079170 msgr2=0x7ff4bc07b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff4bc079170 0x7ff4bc07b630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc039000 tx=0x7ff4dc033000 comp rx=0 tx=0).stop 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 10 monclient: shutdown 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 -- 10.20.193.16:0/2378627649 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0c7fe0 msgr2=0x7ff4ec1f3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0c7fe0 0x7ff4ec1f3c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000c520 tx=0x7ff4e000c9f0 comp rx=0 tx=0).stop 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 -- 10.20.193.16:0/2378627649 shutdown_connections 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff4bc079170 0x7ff4bc07b630 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4ec0c7fe0 0x7ff4ec1f3c20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:54.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4ec1536f0 0x7ff4ec1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:54.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 --2- 10.20.193.16:0/2378627649 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff4ec0700f0 0x7ff4ec06e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:54.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 -- 10.20.193.16:0/2378627649 >> 10.20.193.16:0/2378627649 conn(0x7ff4ec0cd770 msgr2=0x7ff4ec0c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:54.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 -- 10.20.193.16:0/2378627649 shutdown_connections 2026-02-17T05:56:54.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 -- 10.20.193.16:0/2378627649 wait complete. 2026-02-17T05:56:54.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:54.673+0000 7ff4f433e640 1 librados: shutdown 2026-02-17T05:56:57.683 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:56:57.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.751+0000 7fd2b4f51640 10 monclient: get_monmap_and_config 2026-02-17T05:56:57.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.751+0000 7fd2b4f51640 10 monclient: build_initial_monmap 2026-02-17T05:56:57.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.751+0000 7fd2b4f51640 10 monclient: monmap: 2026-02-17T05:56:57.753 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:57.753 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:57.753 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:56:57.752728+0000 2026-02-17T05:56:57.753 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:56:57.752728+0000 2026-02-17T05:56:57.753 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:57.753 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:57.753 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:56:57.754 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:56:57.754 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:56:57.754 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:56:57.754 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:56:57.754 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:56:57.754 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:57.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 1 Processor -- start 2026-02-17T05:56:57.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 1 -- start start 2026-02-17T05:56:57.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 10 monclient: init 2026-02-17T05:56:57.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:57.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 10 monclient: _add_conns ranks=[0,4,5,2,1,3] 2026-02-17T05:56:57.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd2b014f020 addr v1:10.20.193.16:6789/0 2026-02-17T05:56:57.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd2b014f400 0x7fd2b015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:57.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 10 monclient(hunting): picked mon.noname-b con 0x7fd2b014f400 addr v2:10.20.193.128:3300/0 2026-02-17T05:56:57.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd2b015c920 0x7fd2b00c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:57.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 10 monclient(hunting): picked mon.noname-c con 0x7fd2b015c920 addr v2:10.20.193.148:3300/0 2026-02-17T05:56:57.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b00c5b10 con 0x7fd2b014f020 2026-02-17T05:56:57.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:57.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fd2b015cd20 con 0x7fd2b014f400 2026-02-17T05:56:57.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:57.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd2b006e620 con 0x7fd2b015c920 2026-02-17T05:56:57.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:57.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.752+0000 7fd2b4f51640 10 monclient(hunting): authenticate will time out at 1070.321803s 2026-02-17T05:56:57.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2ae575640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fd2b014f020 0x7fd2b006dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:46364/0 (socket says 10.20.193.16:46364) 2026-02-17T05:56:57.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2ae575640 1 -- 10.20.193.16:0/375223797 learned_addr learned my addr 10.20.193.16:0/375223797 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:56:57.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2aed76640 1 --2- 10.20.193.16:0/375223797 >> v2:10.20.193.148:3300/0 conn(0x7fd2b015c920 0x7fd2b00c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:57.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2add74640 1 --2- 10.20.193.16:0/375223797 >> v2:10.20.193.128:3300/0 conn(0x7fd2b014f400 0x7fd2b015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:57.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2aed76640 10 monclient(hunting): get_auth_request con 0x7fd2b015c920 auth_method 0 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2aed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2aed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2aed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2add74640 10 monclient(hunting): get_auth_request con 0x7fd2b014f400 auth_method 0 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2add74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2add74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2add74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2aed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2aed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2aed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:57.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2add74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2add74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.753+0000 7fd2add74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2ad573640 1 -- 10.20.193.16:0/375223797 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3611945888 0 0) 0x7fd2b00c5b10 con 0x7fd2b014f020 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2ad573640 10 monclient(hunting): _negotiate 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2ad573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2ad573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2ad573640 10 monclient(hunting): my global_id is 4533 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2ad573640 1 -- 10.20.193.16:0/375223797 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd29c003620 con 0x7fd2b014f020 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2aed76640 10 monclient(hunting): handle_auth_done global_id 4655 payload 274 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2aed76640 1 -- 10.20.193.16:0/375223797 >> v2:10.20.193.128:3300/0 conn(0x7fd2b014f400 msgr2=0x7fd2b015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:57.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2aed76640 1 --2- 10.20.193.16:0/375223797 >> v2:10.20.193.128:3300/0 conn(0x7fd2b014f400 0x7fd2b015c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:57.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2aed76640 1 -- 10.20.193.16:0/375223797 >> v1:10.20.193.16:6789/0 conn(0x7fd2b014f020 legacy=0x7fd2b006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:57.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2aed76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:57.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2aed76640 1 monclient: found mon.noname-c 2026-02-17T05:56:57.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2aed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:57.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2aed76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:57.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.754+0000 7fd2aed76640 1 -- 10.20.193.16:0/375223797 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b006e900 con 0x7fd2b015c920 2026-02-17T05:56:57.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.755+0000 7fd2aed76640 1 --2- 10.20.193.16:0/375223797 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd2b015c920 0x7fd2b00c6010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00c9d30 tx=0x7fd2a002f250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c72b30162c1a8525 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:57.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 1 -- 10.20.193.16:0/375223797 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a003c070 con 0x7fd2b015c920 2026-02-17T05:56:57.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:57.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:57.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 10 monclient: dump: 2026-02-17T05:56:57.770 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:57.770 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:57.770 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:57.770 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:57.770 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:57.770 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:57.770 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:57.770 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:57.770 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:57.771 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:57.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 10 monclient: _finish_auth 0 2026-02-17T05:56:57.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 10 monclient: _check_auth_tickets 2026-02-17T05:56:57.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:57.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 5 monclient: authenticate success, global_id 4655 2026-02-17T05:56:57.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:57.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 1 -- 10.20.193.16:0/375223797 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a0037440 con 0x7fd2b015c920 2026-02-17T05:56:57.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:57.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:57.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 1 -- 10.20.193.16:0/375223797 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0037810 con 0x7fd2b015c920 2026-02-17T05:56:57.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 4 set_mon_vals no callback set 2026-02-17T05:56:57.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:57.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:57.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2ad573640 10 monclient: dump: 2026-02-17T05:56:57.772 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:57.772 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:57.772 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:57.772 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:57.772 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:57.772 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:57.773 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:57.773 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:57.773 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:57.773 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:57.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 10 monclient: shutdown 2026-02-17T05:56:57.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:57.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 1 -- 10.20.193.16:0/375223797 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd2b015c920 msgr2=0x7fd2b00c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:57.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 1 --2- 10.20.193.16:0/375223797 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd2b015c920 0x7fd2b00c6010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00c9d30 tx=0x7fd2a002f250 comp rx=0 tx=0).stop 2026-02-17T05:56:57.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 1 -- 10.20.193.16:0/375223797 shutdown_connections 2026-02-17T05:56:57.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 1 --2- 10.20.193.16:0/375223797 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd2b015c920 0x7fd2b00c6010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:57.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 1 --2- 10.20.193.16:0/375223797 >> v2:10.20.193.128:3300/0 conn(0x7fd2b014f400 0x7fd2b015c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:57.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 1 -- 10.20.193.16:0/375223797 >> 10.20.193.16:0/375223797 conn(0x7fd2b00cd770 msgr2=0x7fd2b00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:57.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 1 -- 10.20.193.16:0/375223797 shutdown_connections 2026-02-17T05:56:57.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.757+0000 7fd2b4f51640 1 -- 10.20.193.16:0/375223797 wait complete. 2026-02-17T05:56:57.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient: build_initial_monmap 2026-02-17T05:56:57.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient: monmap: 2026-02-17T05:56:57.774 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:56:57.774 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 librados: starting msgr at 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 librados: starting objecter 2026-02-17T05:56:57.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 Processor -- start 2026-02-17T05:56:57.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 -- start start 2026-02-17T05:56:57.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 librados: setting wanted keys 2026-02-17T05:56:57.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 librados: calling monclient init 2026-02-17T05:56:57.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient: init 2026-02-17T05:56:57.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:57.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:56:57.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd2b014f020 0x7fd2b014c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:57.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient(hunting): picked mon.noname-a con 0x7fd2b014f020 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:56:57.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2b014f400 0x7fd2b01579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:57.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient(hunting): picked mon.noname-b con 0x7fd2b014f400 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:56:57.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd2b015c920 0x7fd2b0149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:57.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient(hunting): picked mon.noname-c con 0x7fd2b015c920 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:56:57.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:57.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b006f1a0 con 0x7fd2b014f020 2026-02-17T05:56:57.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:57.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b006eec0 con 0x7fd2b014f400 2026-02-17T05:56:57.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:57.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b006e2f0 con 0x7fd2b015c920 2026-02-17T05:56:57.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.758+0000 7fd2b4f51640 10 monclient(hunting): authenticate will time out at 1070.327977s 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2ae575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd2b014f020 0x7fd2b014c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2aed76640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd2b015c920 0x7fd2b0149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2add74640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2b014f400 0x7fd2b01579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2ae575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd2b014f020 0x7fd2b014c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44498/0 (socket says 10.20.193.16:44498) 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2ae575640 1 -- 10.20.193.16:0/3780887159 learned_addr learned my addr 10.20.193.16:0/3780887159 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2ae575640 10 monclient(hunting): get_auth_request con 0x7fd2b014f020 auth_method 0 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2ae575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2ae575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2ae575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:57.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2aed76640 10 monclient(hunting): get_auth_request con 0x7fd2b015c920 auth_method 0 2026-02-17T05:56:57.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2aed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:57.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2aed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:57.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2aed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:57.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2add74640 10 monclient(hunting): get_auth_request con 0x7fd2b014f400 auth_method 0 2026-02-17T05:56:57.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2add74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:57.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2add74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:57.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2add74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:57.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2ae575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:57.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2ae575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:57.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2ae575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:57.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2aed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:57.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2aed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:57.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2aed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:57.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2add74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:57.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2add74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:57.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.759+0000 7fd2add74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:57.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.760+0000 7fd2aed76640 10 monclient(hunting): handle_auth_done global_id 4658 payload 824 2026-02-17T05:56:57.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.760+0000 7fd2aed76640 1 -- 10.20.193.16:0/3780887159 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2b014f400 msgr2=0x7fd2b01579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:57.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.760+0000 7fd2aed76640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2b014f400 0x7fd2b01579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:57.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.760+0000 7fd2aed76640 1 -- 10.20.193.16:0/3780887159 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd2b014f020 msgr2=0x7fd2b014c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:57.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.760+0000 7fd2aed76640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd2b014f020 0x7fd2b014c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:57.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.760+0000 7fd2aed76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:57.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.760+0000 7fd2aed76640 1 monclient: found mon.noname-c 2026-02-17T05:56:57.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.760+0000 7fd2aed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:57.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.760+0000 7fd2aed76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:56:57.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.760+0000 7fd2aed76640 1 -- 10.20.193.16:0/3780887159 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b014cd80 con 0x7fd2b015c920 2026-02-17T05:56:57.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.761+0000 7fd2aed76640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd2b015c920 0x7fd2b0149c60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0036000 tx=0x7fd2a0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:57.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.761+0000 7fd296ffd640 10 client.?.objecter ms_handle_connect 0x7fd2b015c920 2026-02-17T05:56:57.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.761+0000 7fd296ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd296ffd640 1 -- 10.20.193.16:0/3780887159 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a003c050 con 0x7fd2b015c920 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd296ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd296ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd296ffd640 10 monclient: dump: 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:57.782 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd296ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd296ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd296ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd2b4f51640 5 monclient: authenticate success, global_id 4658 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd2b4f51640 10 monclient: _renew_subs 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd2b4f51640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd2b4f51640 1 -- 10.20.193.16:0/3780887159 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b014d050 con 0x7fd2b015c920 2026-02-17T05:56:57.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd2b4f51640 10 client.4658.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:56:57.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd2b4f51640 10 monclient: _renew_subs 2026-02-17T05:56:57.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd2b4f51640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:57.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd2b4f51640 1 -- 10.20.193.16:0/3780887159 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b014d610 con 0x7fd2b015c920 2026-02-17T05:56:57.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.762+0000 7fd2b4f51640 1 librados: init done 2026-02-17T05:56:57.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd2b4f51640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:57.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd2b4f51640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:57.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd2b4f51640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:57.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd2b4f51640 1 -- 10.20.193.16:0/3780887159 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b0050aa0 con 0x7fd2b015c920 2026-02-17T05:56:57.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd296ffd640 1 -- 10.20.193.16:0/3780887159 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a0048070 con 0x7fd2b015c920 2026-02-17T05:56:57.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd296ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd2acd72640 4 set_mon_vals no callback set 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd296ffd640 1 -- 10.20.193.16:0/3780887159 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0041a10 con 0x7fd2b015c920 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd296ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd296ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.763+0000 7fd296ffd640 10 monclient: dump: 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:57.785 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:56:57.786 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:56:57.786 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:56:57.786 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:56:57.786 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:56:57.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.868+0000 7fd296ffd640 1 -- 10.20.193.16:0/3780887159 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fd2a0041bb0 con 0x7fd2b015c920 2026-02-17T05:56:57.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.868+0000 7fd296ffd640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd27c079140 0x7fd27c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:57.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.869+0000 7fd2ae575640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd27c079140 0x7fd27c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:57.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.869+0000 7fd2ae575640 10 monclient: get_auth_request con 0x7fd27c079140 auth_method 0 2026-02-17T05:56:57.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.869+0000 7fd2ae575640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd27c079140 0x7fd27c07b600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4005980 tx=0x7fd2a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:57.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.869+0000 7fd296ffd640 10 client.4658.objecter ms_handle_connect 0x7fd27c079140 2026-02-17T05:56:57.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.871+0000 7fd296ffd640 1 -- 10.20.193.16:0/3780887159 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fd2a00be010 con 0x7fd2b015c920 2026-02-17T05:56:57.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.871+0000 7fd296ffd640 10 client.4658.objecter ms_dispatch2 0x7fd2b0000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:56:57.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.871+0000 7fd296ffd640 3 client.4658.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:56:57.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.871+0000 7fd296ffd640 3 client.4658.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:56:57.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.871+0000 7fd296ffd640 20 client.4658.objecter dump_active .. 0 homeless 2026-02-17T05:56:57.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.877+0000 7fd296ffd640 1 -- 10.20.193.16:0/3780887159 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd2b0050aa0 con 0x7fd2b015c920 2026-02-17T05:56:57.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.877+0000 7fd296ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:57.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.877+0000 7fd296ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:57.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.976+0000 7fd2b4f51640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:56:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.976+0000 7fd2b4f51640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:56:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.976+0000 7fd2b4f51640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:56:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.976+0000 7fd2b4f51640 1 -- 10.20.193.16:0/3780887159 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd2b015c3e0 con 0x7fd2b015c920 2026-02-17T05:56:57.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.979+0000 7fd296ffd640 1 -- 10.20.193.16:0/3780887159 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd2a0088530 con 0x7fd2b015c920 2026-02-17T05:56:57.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.979+0000 7fd296ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:56:57.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.979+0000 7fd296ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:57.979 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:56:57.979 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 10 librados: watch_flush enter 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 10 librados: watch_flush exit 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 20 client.4658.objecter shutdown clearing up homeless session... 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 10 client.4658.objecter successfully canceled tick 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 -- 10.20.193.16:0/3780887159 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd27c079140 msgr2=0x7fd27c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd27c079140 0x7fd27c07b600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4005980 tx=0x7fd2a4009040 comp rx=0 tx=0).stop 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 10 monclient: shutdown 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 -- 10.20.193.16:0/3780887159 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd2b015c920 msgr2=0x7fd2b0149c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd2b015c920 0x7fd2b0149c60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0036000 tx=0x7fd2a0004b40 comp rx=0 tx=0).stop 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 -- 10.20.193.16:0/3780887159 shutdown_connections 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd27c079140 0x7fd27c07b600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd2b014f400 0x7fd2b01579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd2b015c920 0x7fd2b0149c60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 --2- 10.20.193.16:0/3780887159 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd2b014f020 0x7fd2b014c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 -- 10.20.193.16:0/3780887159 >> 10.20.193.16:0/3780887159 conn(0x7fd2b00cd770 msgr2=0x7fd2b00c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 -- 10.20.193.16:0/3780887159 shutdown_connections 2026-02-17T05:56:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 -- 10.20.193.16:0/3780887159 wait complete. 2026-02-17T05:56:57.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:56:57.980+0000 7fd2b4f51640 1 librados: shutdown 2026-02-17T05:57:00.989 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:01.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.055+0000 7fdc6c2e1640 10 monclient: get_monmap_and_config 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.055+0000 7fdc6c2e1640 10 monclient: build_initial_monmap 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.055+0000 7fdc6c2e1640 10 monclient: monmap: 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:01.056696+0000 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:01.056696+0000 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 1 Processor -- start 2026-02-17T05:57:01.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 1 -- start start 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 10 monclient: init 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 10 monclient: _add_conns ranks=[5,2,0,1,4,3] 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fdc640700f0 0x7fdc640704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 10 monclient(hunting): picked mon.noname-c con 0x7fdc640700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdc640c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdc641536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc640c4820 con 0x7fdc641536f0 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc640c4b00 con 0x7fdc640c7fe0 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fdc640c4de0 con 0x7fdc640700f0 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.056+0000 7fdc6c2e1640 10 monclient(hunting): authenticate will time out at 1073.625793s 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.057+0000 7fdc6a857640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fdc641536f0 0x7fdc64153b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:46080/0 (socket says 10.20.193.16:46080) 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.057+0000 7fdc6a857640 1 -- 10.20.193.16:0/2491340297 learned_addr learned my addr 10.20.193.16:0/2491340297 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.057+0000 7fdc6a056640 1 --2- 10.20.193.16:0/2491340297 >> v2:10.20.193.148:3300/0 conn(0x7fdc640700f0 0x7fdc640704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.057+0000 7fdc6a056640 10 monclient(hunting): get_auth_request con 0x7fdc640700f0 auth_method 0 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.057+0000 7fdc6a056640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.057+0000 7fdc6a056640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:01.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.057+0000 7fdc6a056640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:01.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.058+0000 7fdc6a056640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:01.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.058+0000 7fdc6a056640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:01.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.058+0000 7fdc6a056640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:01.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.058+0000 7fdc69054640 1 -- 10.20.193.16:0/2491340297 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4287142018 0 0) 0x7fdc640c4820 con 0x7fdc641536f0 2026-02-17T05:57:01.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.058+0000 7fdc69054640 10 monclient(hunting): _negotiate 2026-02-17T05:57:01.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.058+0000 7fdc69054640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:01.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.058+0000 7fdc69054640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:01.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.058+0000 7fdc69054640 10 monclient(hunting): my global_id is 4539 2026-02-17T05:57:01.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.058+0000 7fdc69054640 1 -- 10.20.193.16:0/2491340297 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc4c003620 con 0x7fdc641536f0 2026-02-17T05:57:01.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.059+0000 7fdc6a056640 10 monclient(hunting): handle_auth_done global_id 4661 payload 274 2026-02-17T05:57:01.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.059+0000 7fdc6a056640 1 -- 10.20.193.16:0/2491340297 >> v1:10.20.193.148:6789/0 conn(0x7fdc640c7fe0 legacy=0x7fdc640c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:01.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.059+0000 7fdc6a056640 1 -- 10.20.193.16:0/2491340297 >> v1:10.20.193.16:6789/0 conn(0x7fdc641536f0 legacy=0x7fdc64153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:01.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.059+0000 7fdc6a056640 10 monclient: _finish_hunting 0 2026-02-17T05:57:01.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.059+0000 7fdc6a056640 1 monclient: found mon.noname-c 2026-02-17T05:57:01.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.059+0000 7fdc6a056640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:01.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.059+0000 7fdc6a056640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:01.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.059+0000 7fdc6a056640 1 -- 10.20.193.16:0/2491340297 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc640c50c0 con 0x7fdc640700f0 2026-02-17T05:57:01.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.061+0000 7fdc6a056640 1 --2- 10.20.193.16:0/2491340297 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640700f0 0x7fdc640704d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdc540098a0 tx=0x7fdc5402edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2357561de87d851 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 1 -- 10.20.193.16:0/2491340297 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5403c070 con 0x7fdc640700f0 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 10 monclient: dump: 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 10 monclient: _finish_auth 0 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 10 monclient: _check_auth_tickets 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 1 -- 10.20.193.16:0/2491340297 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc5402fae0 con 0x7fdc640700f0 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 5 monclient: authenticate success, global_id 4661 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:01.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 1 -- 10.20.193.16:0/2491340297 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54037440 con 0x7fdc640700f0 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 4 set_mon_vals no callback set 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc69054640 10 monclient: dump: 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 10 monclient: shutdown 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2491340297 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640700f0 msgr2=0x7fdc640704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 1 --2- 10.20.193.16:0/2491340297 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640700f0 0x7fdc640704d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdc540098a0 tx=0x7fdc5402edd0 comp rx=0 tx=0).stop 2026-02-17T05:57:01.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2491340297 shutdown_connections 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 1 --2- 10.20.193.16:0/2491340297 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640700f0 0x7fdc640704d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2491340297 >> 10.20.193.16:0/2491340297 conn(0x7fdc640cd770 msgr2=0x7fdc640cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2491340297 shutdown_connections 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.063+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2491340297 wait complete. 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient: build_initial_monmap 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient: monmap: 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 librados: starting msgr at 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 librados: starting objecter 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 Processor -- start 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 -- start start 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 librados: setting wanted keys 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 librados: calling monclient init 2026-02-17T05:57:01.065 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient: init 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdc640700f0 0x7fdc641f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient(hunting): picked mon.noname-b con 0x7fdc640700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640c7fe0 0x7fdc64144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient(hunting): picked mon.noname-c con 0x7fdc640c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdc641536f0 0x7fdc64073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient(hunting): picked mon.noname-a con 0x7fdc641536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64074c00 con 0x7fdc641536f0 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fdc640c5680 con 0x7fdc640700f0 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64074d80 con 0x7fdc640c7fe0 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.064+0000 7fdc6c2e1640 10 monclient(hunting): authenticate will time out at 1073.633995s 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a857640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdc641536f0 0x7fdc64073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc69855640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640c7fe0 0x7fdc64144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a056640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdc640700f0 0x7fdc641f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc69855640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640c7fe0 0x7fdc64144490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:59010/0 (socket says 10.20.193.16:59010) 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc69855640 1 -- 10.20.193.16:0/2793756250 learned_addr learned my addr 10.20.193.16:0/2793756250 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc69855640 10 monclient(hunting): get_auth_request con 0x7fdc640c7fe0 auth_method 0 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc69855640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc69855640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:01.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc69855640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a857640 10 monclient(hunting): get_auth_request con 0x7fdc641536f0 auth_method 0 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a857640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a857640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a857640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a056640 10 monclient(hunting): get_auth_request con 0x7fdc640700f0 auth_method 0 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a056640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a056640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a056640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc69855640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc69855640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc69855640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a857640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a857640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a857640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a056640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a056640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.065+0000 7fdc6a056640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.067+0000 7fdc69855640 10 monclient(hunting): handle_auth_done global_id 4667 payload 824 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.067+0000 7fdc69855640 1 -- 10.20.193.16:0/2793756250 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdc640700f0 msgr2=0x7fdc641f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.067+0000 7fdc69855640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdc640700f0 0x7fdc641f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.067+0000 7fdc69855640 1 -- 10.20.193.16:0/2793756250 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdc641536f0 msgr2=0x7fdc64073990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.067+0000 7fdc69855640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdc641536f0 0x7fdc64073990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.067+0000 7fdc69855640 10 monclient: _finish_hunting 0 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.067+0000 7fdc69855640 1 monclient: found mon.noname-c 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.067+0000 7fdc69855640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.067+0000 7fdc69855640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:01.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.067+0000 7fdc69855640 1 -- 10.20.193.16:0/2793756250 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc64144d30 con 0x7fdc640c7fe0 2026-02-17T05:57:01.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.068+0000 7fdc69855640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640c7fe0 0x7fdc64144490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c700 tx=0x7fdc5800cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:01.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.068+0000 7fdc4affd640 10 client.?.objecter ms_handle_connect 0x7fdc640c7fe0 2026-02-17T05:57:01.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.068+0000 7fdc4affd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.069+0000 7fdc4affd640 1 -- 10.20.193.16:0/2793756250 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580052d0 con 0x7fdc640c7fe0 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 10 monclient: dump: 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 10 monclient: _finish_auth 0 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 10 monclient: _check_auth_tickets 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 1 -- 10.20.193.16:0/2793756250 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc58005470 con 0x7fdc640c7fe0 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 5 monclient: authenticate success, global_id 4667 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 10 monclient: _renew_subs 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2793756250 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc64145010 con 0x7fdc640c7fe0 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 1 -- 10.20.193.16:0/2793756250 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5800a860 con 0x7fdc640c7fe0 2026-02-17T05:57:01.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4bfff640 4 set_mon_vals no callback set 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc4affd640 10 monclient: dump: 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 10 client.4667.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 10 monclient: _renew_subs 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2793756250 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc641455d0 con 0x7fdc640c7fe0 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 1 librados: init done 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:01.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.070+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2793756250 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc640704d0 con 0x7fdc640c7fe0 2026-02-17T05:57:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.176+0000 7fdc4affd640 1 -- 10.20.193.16:0/2793756250 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fdc5800aa00 con 0x7fdc640c7fe0 2026-02-17T05:57:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.177+0000 7fdc4affd640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdc34079140 0x7fdc3407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.177+0000 7fdc6a056640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdc34079140 0x7fdc3407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.177+0000 7fdc6a056640 10 monclient: get_auth_request con 0x7fdc34079140 auth_method 0 2026-02-17T05:57:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.177+0000 7fdc6a056640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdc34079140 0x7fdc3407b600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdc54039000 tx=0x7fdc54033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.177+0000 7fdc4affd640 10 client.4667.objecter ms_handle_connect 0x7fdc34079140 2026-02-17T05:57:01.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.179+0000 7fdc4affd640 1 -- 10.20.193.16:0/2793756250 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fdc5800ad20 con 0x7fdc640c7fe0 2026-02-17T05:57:01.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.179+0000 7fdc4affd640 10 client.4667.objecter ms_dispatch2 0x7fdc64000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:01.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.179+0000 7fdc4affd640 3 client.4667.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:01.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.179+0000 7fdc4affd640 3 client.4667.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:01.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.179+0000 7fdc4affd640 20 client.4667.objecter dump_active .. 0 homeless 2026-02-17T05:57:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.185+0000 7fdc4affd640 1 -- 10.20.193.16:0/2793756250 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdc58062250 con 0x7fdc640c7fe0 2026-02-17T05:57:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.185+0000 7fdc4affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.185+0000 7fdc4affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:01.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.284+0000 7fdc6c2e1640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:01.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.284+0000 7fdc6c2e1640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:01.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.284+0000 7fdc6c2e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:01.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.284+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2793756250 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdc641573f0 con 0x7fdc640c7fe0 2026-02-17T05:57:01.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.286+0000 7fdc4affd640 1 -- 10.20.193.16:0/2793756250 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fdc58066550 con 0x7fdc640c7fe0 2026-02-17T05:57:01.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.286+0000 7fdc4affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:01.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.286+0000 7fdc4affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:01.288 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:01.288 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:01.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 10 librados: watch_flush enter 2026-02-17T05:57:01.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 10 librados: watch_flush exit 2026-02-17T05:57:01.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 20 client.4667.objecter shutdown clearing up homeless session... 2026-02-17T05:57:01.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 10 client.4667.objecter successfully canceled tick 2026-02-17T05:57:01.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2793756250 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdc34079140 msgr2=0x7fdc3407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:01.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdc34079140 0x7fdc3407b600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdc54039000 tx=0x7fdc54033000 comp rx=0 tx=0).stop 2026-02-17T05:57:01.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 10 monclient: shutdown 2026-02-17T05:57:01.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:01.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2793756250 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640c7fe0 msgr2=0x7fdc64144490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:01.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640c7fe0 0x7fdc64144490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c700 tx=0x7fdc5800cbd0 comp rx=0 tx=0).stop 2026-02-17T05:57:01.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2793756250 shutdown_connections 2026-02-17T05:57:01.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdc34079140 0x7fdc3407b600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:01.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdc641536f0 0x7fdc64073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:01.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdc640c7fe0 0x7fdc64144490 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:01.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 --2- 10.20.193.16:0/2793756250 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdc640700f0 0x7fdc641f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:01.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2793756250 >> 10.20.193.16:0/2793756250 conn(0x7fdc640cd770 msgr2=0x7fdc64148cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:01.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2793756250 shutdown_connections 2026-02-17T05:57:01.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 -- 10.20.193.16:0/2793756250 wait complete. 2026-02-17T05:57:01.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:01.288+0000 7fdc6c2e1640 1 librados: shutdown 2026-02-17T05:57:03.741 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:02:42.568 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:03.768 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:02:42.594 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:03.770 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:02:42.596 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:03.770 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:02:42.598 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:03.783 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:02:42.610 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:03.787 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:02:42.613 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:03.804 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:02:42.593 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:03.815 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:02:42.604 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:03.828 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:02:42.617 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:04.297 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:04.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.366+0000 7f4f7b6d3640 10 monclient: get_monmap_and_config 2026-02-17T05:57:04.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.366+0000 7f4f7b6d3640 10 monclient: build_initial_monmap 2026-02-17T05:57:04.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.367+0000 7f4f7b6d3640 10 monclient: monmap: 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:04.368017+0000 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:04.368017+0000 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:04.368 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:04.369 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:04.369 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:04.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.367+0000 7f4f7b6d3640 1 Processor -- start 2026-02-17T05:57:04.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.367+0000 7f4f7b6d3640 1 -- start start 2026-02-17T05:57:04.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.367+0000 7f4f7b6d3640 10 monclient: init 2026-02-17T05:57:04.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:04.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 10 monclient: _add_conns ranks=[3,2,5,4,1,0] 2026-02-17T05:57:04.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f4f740700f0 0x7f4f740704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 10 monclient(hunting): picked mon.noname-a con 0x7f4f740700f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4f740c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f4f741536f0 0x7f4f74153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 10 monclient(hunting): picked mon.noname-c con 0x7f4f741536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f740c4820 con 0x7f4f740c7fe0 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4f740c4b00 con 0x7f4f740700f0 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f4f740c4de0 con 0x7f4f741536f0 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f7b6d3640 10 monclient(hunting): authenticate will time out at 1076.937098s 2026-02-17T05:57:04.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f79448640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f4f740700f0 0x7f4f740704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f79c49640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f4f741536f0 0x7f4f74153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f79448640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f4f740700f0 0x7f4f740704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51354/0 (socket says 10.20.193.16:51354) 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f79448640 1 -- 10.20.193.16:0/3412368971 learned_addr learned my addr 10.20.193.16:0/3412368971 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f79448640 10 monclient(hunting): get_auth_request con 0x7f4f740700f0 auth_method 0 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f79448640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f79448640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.368+0000 7f4f79448640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.369+0000 7f4f79448640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.369+0000 7f4f79448640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.369+0000 7f4f79448640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:04.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.369+0000 7f4f79c49640 10 monclient(hunting): get_auth_request con 0x7f4f741536f0 auth_method 0 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.369+0000 7f4f79c49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.369+0000 7f4f79c49640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.369+0000 7f4f79c49640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.369+0000 7f4f79c49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.369+0000 7f4f79c49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.369+0000 7f4f79c49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.370+0000 7f4f79448640 10 monclient(hunting): handle_auth_done global_id 4545 payload 274 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.370+0000 7f4f79448640 1 -- 10.20.193.16:0/3412368971 >> v2:10.20.193.148:3300/0 conn(0x7f4f741536f0 msgr2=0x7f4f74153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.370+0000 7f4f79448640 1 --2- 10.20.193.16:0/3412368971 >> v2:10.20.193.148:3300/0 conn(0x7f4f741536f0 0x7f4f74153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.370+0000 7f4f79448640 1 -- 10.20.193.16:0/3412368971 >> v1:10.20.193.148:6789/0 conn(0x7f4f740c7fe0 legacy=0x7f4f740c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.370+0000 7f4f79448640 10 monclient: _finish_hunting 0 2026-02-17T05:57:04.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.370+0000 7f4f79448640 1 monclient: found mon.noname-a 2026-02-17T05:57:04.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.370+0000 7f4f79448640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:04.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.370+0000 7f4f79448640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:04.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.370+0000 7f4f79448640 1 -- 10.20.193.16:0/3412368971 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f740c50c0 con 0x7f4f740700f0 2026-02-17T05:57:04.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.371+0000 7f4f79448640 1 --2- 10.20.193.16:0/3412368971 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740700f0 0x7f4f740704d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4f740c9540 tx=0x7f4f6402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2169a889b9c9b9d1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:04.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 1 -- 10.20.193.16:0/3412368971 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6403c070 con 0x7f4f740700f0 2026-02-17T05:57:04.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:04.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:04.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 10 monclient: dump: 2026-02-17T05:57:04.382 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 10 monclient: _finish_auth 0 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:04.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 1 -- 10.20.193.16:0/3412368971 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6402fa90 con 0x7f4f740700f0 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 5 monclient: authenticate success, global_id 4545 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 1 -- 10.20.193.16:0/3412368971 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6402fe60 con 0x7f4f740700f0 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 4 set_mon_vals no callback set 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f63fff640 10 monclient: dump: 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:04.384 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 10 monclient: shutdown 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/3412368971 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740700f0 msgr2=0x7f4f740704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 1 --2- 10.20.193.16:0/3412368971 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740700f0 0x7f4f740704d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4f740c9540 tx=0x7f4f6402ee60 comp rx=0 tx=0).stop 2026-02-17T05:57:04.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/3412368971 shutdown_connections 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 1 --2- 10.20.193.16:0/3412368971 >> v2:10.20.193.148:3300/0 conn(0x7f4f741536f0 0x7f4f74153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 1 --2- 10.20.193.16:0/3412368971 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740700f0 0x7f4f740704d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/3412368971 >> 10.20.193.16:0/3412368971 conn(0x7f4f740cd770 msgr2=0x7f4f740cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/3412368971 shutdown_connections 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.373+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/3412368971 wait complete. 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient: build_initial_monmap 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient: monmap: 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:04.386 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 librados: starting msgr at 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 librados: starting objecter 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 Processor -- start 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 -- start start 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 librados: setting wanted keys 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 librados: calling monclient init 2026-02-17T05:57:04.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient: init 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f740700f0 0x7f4f7406e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient(hunting): picked mon.noname-b con 0x7f4f740700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740c7fe0 0x7f4f740cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient(hunting): picked mon.noname-a con 0x7f4f740c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f741536f0 0x7f4f741f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient(hunting): picked mon.noname-c con 0x7f4f741536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74154060 con 0x7f4f740c7fe0 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:04.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f4f740c5680 con 0x7f4f740700f0 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74074c00 con 0x7f4f741536f0 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.374+0000 7f4f7b6d3640 10 monclient(hunting): authenticate will time out at 1076.943850s 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f78c47640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740c7fe0 0x7f4f740cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79c49640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f741536f0 0x7f4f741f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f78c47640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740c7fe0 0x7f4f740cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51358/0 (socket says 10.20.193.16:51358) 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f78c47640 1 -- 10.20.193.16:0/1007449981 learned_addr learned my addr 10.20.193.16:0/1007449981 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f78c47640 10 monclient(hunting): get_auth_request con 0x7f4f740c7fe0 auth_method 0 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f78c47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f78c47640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:04.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f78c47640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79c49640 10 monclient(hunting): get_auth_request con 0x7f4f741536f0 auth_method 0 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79c49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79c49640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79c49640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79448640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f740700f0 0x7f4f7406e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79c49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79c49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79c49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f78c47640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f78c47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f78c47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79448640 10 monclient(hunting): get_auth_request con 0x7f4f740700f0 auth_method 0 2026-02-17T05:57:04.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79448640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79448640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.375+0000 7f4f79448640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.376+0000 7f4f79448640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.376+0000 7f4f79448640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.376+0000 7f4f79448640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f79c49640 10 monclient(hunting): handle_auth_done global_id 4673 payload 824 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f79c49640 1 -- 10.20.193.16:0/1007449981 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f740700f0 msgr2=0x7f4f7406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f79c49640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f740700f0 0x7f4f7406e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f79c49640 1 -- 10.20.193.16:0/1007449981 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740c7fe0 msgr2=0x7f4f740cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f79c49640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740c7fe0 0x7f4f740cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f79c49640 10 monclient: _finish_hunting 0 2026-02-17T05:57:04.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f79c49640 1 monclient: found mon.noname-c 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f79c49640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f79c49640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f79c49640 1 -- 10.20.193.16:0/1007449981 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7406da10 con 0x7f4f741536f0 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.377+0000 7f4f78c47640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740c7fe0 0x7f4f740cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.378+0000 7f4f79c49640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f741536f0 0x7f4f741f36a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009d90 tx=0x7f4f7000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.378+0000 7f4f61ffb640 10 client.?.objecter ms_handle_connect 0x7f4f741536f0 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.378+0000 7f4f61ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f61ffb640 1 -- 10.20.193.16:0/1007449981 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70018020 con 0x7f4f741536f0 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f61ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f61ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f61ffb640 10 monclient: dump: 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:04.392 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f61ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f61ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f61ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:04.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f7b6d3640 5 monclient: authenticate success, global_id 4673 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f7b6d3640 10 monclient: _renew_subs 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f7b6d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/1007449981 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7406e9f0 con 0x7f4f741536f0 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f7b6d3640 10 client.4673.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f7b6d3640 10 monclient: _renew_subs 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f7b6d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/1007449981 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f741f4250 con 0x7f4f741536f0 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.379+0000 7f4f7b6d3640 1 librados: init done 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f61ffb640 1 -- 10.20.193.16:0/1007449981 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f70014480 con 0x7f4f741536f0 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f61ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f61ffb640 1 -- 10.20.193.16:0/1007449981 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70014830 con 0x7f4f741536f0 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f61ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:04.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f61ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f61ffb640 10 monclient: dump: 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:04.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f637fe640 4 set_mon_vals no callback set 2026-02-17T05:57:04.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f7b6d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:04.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f7b6d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:04.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f7b6d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:04.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.380+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/1007449981 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f740704d0 con 0x7f4f741536f0 2026-02-17T05:57:04.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.392+0000 7f4f61ffb640 1 -- 10.20.193.16:0/1007449981 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f4f700149d0 con 0x7f4f741536f0 2026-02-17T05:57:04.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.392+0000 7f4f61ffb640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f44079140 0x7f4f4407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:04.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.393+0000 7f4f79448640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f44079140 0x7f4f4407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:04.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.393+0000 7f4f79448640 10 monclient: get_auth_request con 0x7f4f44079140 auth_method 0 2026-02-17T05:57:04.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.393+0000 7f4f79448640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f44079140 0x7f4f4407b600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f740c5b80 tx=0x7f4f64033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:04.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.393+0000 7f4f61ffb640 10 client.4673.objecter ms_handle_connect 0x7f4f44079140 2026-02-17T05:57:04.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.487+0000 7f4f61ffb640 1 -- 10.20.193.16:0/1007449981 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f4f7005f1f0 con 0x7f4f741536f0 2026-02-17T05:57:04.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.487+0000 7f4f61ffb640 10 client.4673.objecter ms_dispatch2 0x7f4f74000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:04.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.487+0000 7f4f61ffb640 3 client.4673.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:04.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.487+0000 7f4f61ffb640 3 client.4673.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:04.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.487+0000 7f4f61ffb640 20 client.4673.objecter dump_active .. 0 homeless 2026-02-17T05:57:04.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.493+0000 7f4f61ffb640 1 -- 10.20.193.16:0/1007449981 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4f70022030 con 0x7f4f741536f0 2026-02-17T05:57:04.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.494+0000 7f4f61ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:04.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.494+0000 7f4f61ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:04.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.593+0000 7f4f7b6d3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:04.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.593+0000 7f4f7b6d3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:04.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.593+0000 7f4f7b6d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:04.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.593+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/1007449981 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4f7413db20 con 0x7f4f741536f0 2026-02-17T05:57:04.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.596+0000 7f4f61ffb640 1 -- 10.20.193.16:0/1007449981 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f4f70010040 con 0x7f4f741536f0 2026-02-17T05:57:04.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.596+0000 7f4f61ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:04.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.596+0000 7f4f61ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:04.596 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:04.596 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 10 librados: watch_flush enter 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 10 librados: watch_flush exit 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 20 client.4673.objecter shutdown clearing up homeless session... 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 10 client.4673.objecter successfully canceled tick 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/1007449981 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f44079140 msgr2=0x7f4f4407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f44079140 0x7f4f4407b600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f740c5b80 tx=0x7f4f64033000 comp rx=0 tx=0).stop 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 10 monclient: shutdown 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/1007449981 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f741536f0 msgr2=0x7f4f741f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f741536f0 0x7f4f741f36a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009d90 tx=0x7f4f7000c490 comp rx=0 tx=0).stop 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/1007449981 shutdown_connections 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f44079140 0x7f4f4407b600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f741536f0 0x7f4f741f36a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f740c7fe0 0x7f4f740cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 --2- 10.20.193.16:0/1007449981 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f740700f0 0x7f4f7406e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/1007449981 >> 10.20.193.16:0/1007449981 conn(0x7f4f740cd770 msgr2=0x7f4f7414dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/1007449981 shutdown_connections 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 -- 10.20.193.16:0/1007449981 wait complete. 2026-02-17T05:57:04.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:04.597+0000 7f4f7b6d3640 1 librados: shutdown 2026-02-17T05:57:07.606 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:07.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.672+0000 7fa83164d640 10 monclient: get_monmap_and_config 2026-02-17T05:57:07.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.672+0000 7fa83164d640 10 monclient: build_initial_monmap 2026-02-17T05:57:07.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.672+0000 7fa83164d640 10 monclient: monmap: 2026-02-17T05:57:07.672 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:07.672 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:07.672 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:07.673277+0000 2026-02-17T05:57:07.672 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:07.673277+0000 2026-02-17T05:57:07.672 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:07.673 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:07.673 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:07.673 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:07.673 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:07.673 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:07.673 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:07.673 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:07.673 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:07.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.672+0000 7fa83164d640 1 Processor -- start 2026-02-17T05:57:07.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.672+0000 7fa83164d640 1 -- start start 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.672+0000 7fa83164d640 10 monclient: init 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 10 monclient: _add_conns ranks=[2,5,3,1,0,4] 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa82c155ac0 addr v1:10.20.193.148:6789/0 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fa82c070590 0x7fa82c073890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 10 monclient(hunting): picked mon.noname-c con 0x7fa82c070590 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fa82c155ea0 0x7fa82c15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 10 monclient(hunting): picked mon.noname-a con 0x7fa82c155ea0 addr v2:10.20.193.16:3300/0 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa82c072510 con 0x7fa82c155ac0 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa82c0c8d10 con 0x7fa82c155ea0 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fa82c0727f0 con 0x7fa82c070590 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa83164d640 10 monclient(hunting): authenticate will time out at 1080.242265s 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa82b7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fa82c155ea0 0x7fa82c15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa82a7fc640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fa82c070590 0x7fa82c073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa82affd640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fa82c155ac0 0x7fa82c06fe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:46204/0 (socket says 10.20.193.16:46204) 2026-02-17T05:57:07.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa82affd640 1 -- 10.20.193.16:0/1213823307 learned_addr learned my addr 10.20.193.16:0/1213823307 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa82b7fe640 10 monclient(hunting): get_auth_request con 0x7fa82c155ea0 auth_method 0 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa82b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa82b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.673+0000 7fa82b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.674+0000 7fa82a7fc640 10 monclient(hunting): get_auth_request con 0x7fa82c070590 auth_method 0 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.674+0000 7fa82a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.674+0000 7fa82a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.674+0000 7fa82a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.674+0000 7fa82b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.674+0000 7fa82b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.674+0000 7fa82b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.674+0000 7fa82a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.674+0000 7fa82a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:07.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.674+0000 7fa82a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:07.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.675+0000 7fa82b7fe640 10 monclient(hunting): handle_auth_done global_id 4551 payload 274 2026-02-17T05:57:07.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.675+0000 7fa82b7fe640 1 -- 10.20.193.16:0/1213823307 >> v2:10.20.193.148:3300/0 conn(0x7fa82c070590 msgr2=0x7fa82c073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:07.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.675+0000 7fa82b7fe640 1 --2- 10.20.193.16:0/1213823307 >> v2:10.20.193.148:3300/0 conn(0x7fa82c070590 0x7fa82c073890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.675+0000 7fa82b7fe640 1 -- 10.20.193.16:0/1213823307 >> v1:10.20.193.148:6789/0 conn(0x7fa82c155ac0 legacy=0x7fa82c06fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:07.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.675+0000 7fa82b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:07.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.675+0000 7fa82b7fe640 1 monclient: found mon.noname-a 2026-02-17T05:57:07.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.675+0000 7fa82b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:07.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.675+0000 7fa82b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:07.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.675+0000 7fa82b7fe640 1 -- 10.20.193.16:0/1213823307 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c072ad0 con 0x7fa82c155ea0 2026-02-17T05:57:07.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.676+0000 7fa82b7fe640 1 --2- 10.20.193.16:0/1213823307 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa82c155ea0 0x7fa82c15d1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa82c070f10 tx=0x7fa81c02f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1924a6168595e172 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:07.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 1 -- 10.20.193.16:0/1213823307 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c03c070 con 0x7fa82c155ea0 2026-02-17T05:57:07.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:07.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:07.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 10 monclient: dump: 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 1 -- 10.20.193.16:0/1213823307 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c02fe40 con 0x7fa82c155ea0 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 5 monclient: authenticate success, global_id 4551 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 1 -- 10.20.193.16:0/1213823307 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c037660 con 0x7fa82c155ea0 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 4 set_mon_vals no callback set 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa829ffb640 10 monclient: dump: 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:07.679 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 10 monclient: shutdown 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 1 -- 10.20.193.16:0/1213823307 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa82c155ea0 msgr2=0x7fa82c15d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 1 --2- 10.20.193.16:0/1213823307 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa82c155ea0 0x7fa82c15d1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa82c070f10 tx=0x7fa81c02f130 comp rx=0 tx=0).stop 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 1 -- 10.20.193.16:0/1213823307 shutdown_connections 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 1 --2- 10.20.193.16:0/1213823307 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa82c155ea0 0x7fa82c15d1c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 1 --2- 10.20.193.16:0/1213823307 >> v2:10.20.193.148:3300/0 conn(0x7fa82c070590 0x7fa82c073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 1 -- 10.20.193.16:0/1213823307 >> 10.20.193.16:0/1213823307 conn(0x7fa82c0cd770 msgr2=0x7fa82c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 1 -- 10.20.193.16:0/1213823307 shutdown_connections 2026-02-17T05:57:07.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.678+0000 7fa83164d640 1 -- 10.20.193.16:0/1213823307 wait complete. 2026-02-17T05:57:07.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient: build_initial_monmap 2026-02-17T05:57:07.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient: monmap: 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 librados: starting msgr at 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 librados: starting objecter 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 Processor -- start 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 -- start start 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 librados: setting wanted keys 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 librados: calling monclient init 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient: init 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa82c070590 0x7fa82c1e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient(hunting): picked mon.noname-c con 0x7fa82c070590 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa82c155ac0 0x7fa82c1f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient(hunting): picked mon.noname-a con 0x7fa82c155ac0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa82c155ea0 0x7fa82c1ea510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient(hunting): picked mon.noname-b con 0x7fa82c155ea0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:07.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c073370 con 0x7fa82c155ac0 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c073090 con 0x7fa82c155ea0 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c15cb30 con 0x7fa82c070590 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.679+0000 7fa83164d640 10 monclient(hunting): authenticate will time out at 1080.248862s 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82affd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa82c070590 0x7fa82c1e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82a7fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa82c155ac0 0x7fa82c1f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82a7fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa82c155ac0 0x7fa82c1f5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51372/0 (socket says 10.20.193.16:51372) 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82a7fc640 1 -- 10.20.193.16:0/4263581403 learned_addr learned my addr 10.20.193.16:0/4263581403 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82a7fc640 10 monclient(hunting): get_auth_request con 0x7fa82c155ac0 auth_method 0 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82b7fe640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa82c155ea0 0x7fa82c1ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:07.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82affd640 10 monclient(hunting): get_auth_request con 0x7fa82c070590 auth_method 0 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82b7fe640 10 monclient(hunting): get_auth_request con 0x7fa82c155ea0 auth_method 0 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:07.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:07.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:07.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:07.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:07.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.680+0000 7fa82b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:07.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.681+0000 7fa82affd640 10 monclient(hunting): handle_auth_done global_id 4682 payload 824 2026-02-17T05:57:07.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.681+0000 7fa82affd640 1 -- 10.20.193.16:0/4263581403 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa82c155ea0 msgr2=0x7fa82c1ea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:07.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.681+0000 7fa82affd640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa82c155ea0 0x7fa82c1ea510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.681+0000 7fa82affd640 1 -- 10.20.193.16:0/4263581403 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa82c155ac0 msgr2=0x7fa82c1f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:07.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.681+0000 7fa82affd640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa82c155ac0 0x7fa82c1f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.681+0000 7fa82affd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:07.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.681+0000 7fa82affd640 1 monclient: found mon.noname-c 2026-02-17T05:57:07.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.681+0000 7fa82affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:07.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.681+0000 7fa82affd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:07.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.681+0000 7fa82affd640 1 -- 10.20.193.16:0/4263581403 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c1e8b80 con 0x7fa82c070590 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.682+0000 7fa82affd640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa82c070590 0x7fa82c1e84d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c410 tx=0x7fa82000c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.682+0000 7fa80b7fe640 10 client.?.objecter ms_handle_connect 0x7fa82c070590 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.682+0000 7fa80b7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa80b7fe640 1 -- 10.20.193.16:0/4263581403 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820017070 con 0x7fa82c070590 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa80b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa80b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa80b7fe640 10 monclient: dump: 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:07.689 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa80b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa80b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa80b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa83164d640 5 monclient: authenticate success, global_id 4682 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa83164d640 10 monclient: _renew_subs 2026-02-17T05:57:07.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa83164d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.683+0000 7fa83164d640 1 -- 10.20.193.16:0/4263581403 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c1e8e60 con 0x7fa82c070590 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa83164d640 10 client.4682.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa83164d640 10 monclient: _renew_subs 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa83164d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa83164d640 1 -- 10.20.193.16:0/4263581403 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c1e9420 con 0x7fa82c070590 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa83164d640 1 librados: init done 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa83164d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa83164d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa83164d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa83164d640 1 -- 10.20.193.16:0/4263581403 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82c073890 con 0x7fa82c070590 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa80b7fe640 1 -- 10.20.193.16:0/4263581403 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa820005de0 con 0x7fa82c070590 2026-02-17T05:57:07.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa80b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa8297fa640 4 set_mon_vals no callback set 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa80b7fe640 1 -- 10.20.193.16:0/4263581403 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820021060 con 0x7fa82c070590 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa80b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa80b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.684+0000 7fa80b7fe640 10 monclient: dump: 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:07.692 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:07.693 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:07.693 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:07.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.790+0000 7fa80b7fe640 1 -- 10.20.193.16:0/4263581403 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fa820024080 con 0x7fa82c070590 2026-02-17T05:57:07.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.790+0000 7fa80b7fe640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa7f8079170 0x7fa7f807b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:07.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.790+0000 7fa82a7fc640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa7f8079170 0x7fa7f807b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:07.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.791+0000 7fa82a7fc640 10 monclient: get_auth_request con 0x7fa7f8079170 auth_method 0 2026-02-17T05:57:07.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.791+0000 7fa80b7fe640 10 client.4682.objecter ms_handle_connect 0x7fa7f8079170 2026-02-17T05:57:07.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.791+0000 7fa82a7fc640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa7f8079170 0x7fa7f807b630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa82c1eb080 tx=0x7fa814009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:07.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.792+0000 7fa80b7fe640 1 -- 10.20.193.16:0/4263581403 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fa820097160 con 0x7fa82c070590 2026-02-17T05:57:07.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.792+0000 7fa80b7fe640 10 client.4682.objecter ms_dispatch2 0x7fa82c000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:07.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.792+0000 7fa80b7fe640 3 client.4682.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:07.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.792+0000 7fa80b7fe640 3 client.4682.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:07.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.792+0000 7fa80b7fe640 20 client.4682.objecter dump_active .. 0 homeless 2026-02-17T05:57:07.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.805+0000 7fa80b7fe640 1 -- 10.20.193.16:0/4263581403 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa820061680 con 0x7fa82c070590 2026-02-17T05:57:07.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.805+0000 7fa80b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:07.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.805+0000 7fa80b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:07.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.905+0000 7fa83164d640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:07.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.905+0000 7fa83164d640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:07.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.905+0000 7fa83164d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:07.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.905+0000 7fa83164d640 1 -- 10.20.193.16:0/4263581403 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa82c15f030 con 0x7fa82c070590 2026-02-17T05:57:07.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.908+0000 7fa80b7fe640 1 -- 10.20.193.16:0/4263581403 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fa820065980 con 0x7fa82c070590 2026-02-17T05:57:07.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.908+0000 7fa80b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:07.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.908+0000 7fa80b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:07.909 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:07.909 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:07.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 10 librados: watch_flush enter 2026-02-17T05:57:07.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 10 librados: watch_flush exit 2026-02-17T05:57:07.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 20 client.4682.objecter shutdown clearing up homeless session... 2026-02-17T05:57:07.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 10 client.4682.objecter successfully canceled tick 2026-02-17T05:57:07.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 -- 10.20.193.16:0/4263581403 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa7f8079170 msgr2=0x7fa7f807b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:07.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa7f8079170 0x7fa7f807b630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa82c1eb080 tx=0x7fa814009290 comp rx=0 tx=0).stop 2026-02-17T05:57:07.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 10 monclient: shutdown 2026-02-17T05:57:07.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:07.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 -- 10.20.193.16:0/4263581403 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa82c070590 msgr2=0x7fa82c1e84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:07.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa82c070590 0x7fa82c1e84d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c410 tx=0x7fa82000c8e0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 -- 10.20.193.16:0/4263581403 shutdown_connections 2026-02-17T05:57:07.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa7f8079170 0x7fa7f807b630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa82c155ea0 0x7fa82c1ea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa82c155ac0 0x7fa82c1f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 --2- 10.20.193.16:0/4263581403 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa82c070590 0x7fa82c1e84d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 -- 10.20.193.16:0/4263581403 >> 10.20.193.16:0/4263581403 conn(0x7fa82c0cd770 msgr2=0x7fa82c15f250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:07.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 -- 10.20.193.16:0/4263581403 shutdown_connections 2026-02-17T05:57:07.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 -- 10.20.193.16:0/4263581403 wait complete. 2026-02-17T05:57:07.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:07.909+0000 7fa83164d640 1 librados: shutdown 2026-02-17T05:57:10.920 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:10.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.987+0000 7f37d5b14640 10 monclient: get_monmap_and_config 2026-02-17T05:57:10.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.987+0000 7f37d5b14640 10 monclient: build_initial_monmap 2026-02-17T05:57:10.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.987+0000 7f37d5b14640 10 monclient: monmap: 2026-02-17T05:57:10.988 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:10.988499+0000 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:10.988499+0000 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:10.989 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:10.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.987+0000 7f37d5b14640 1 Processor -- start 2026-02-17T05:57:10.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.987+0000 7f37d5b14640 1 -- start start 2026-02-17T05:57:10.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.987+0000 7f37d5b14640 10 monclient: init 2026-02-17T05:57:10.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:10.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 10 monclient: _add_conns ranks=[5,0,3,2,4,1] 2026-02-17T05:57:10.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f37d00700f0 0x7f37d00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 10 monclient(hunting): picked mon.noname-c con 0x7f37d00700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f37d00c7fe0 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f37d01536f0 0x7f37d0153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 10 monclient(hunting): picked mon.noname-a con 0x7f37d01536f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37d00c4820 con 0x7f37d00c7fe0 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f37d00c4b00 con 0x7f37d01536f0 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f37d00c4de0 con 0x7f37d00700f0 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37d5b14640 10 monclient(hunting): authenticate will time out at 1083.557479s 2026-02-17T05:57:10.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.988+0000 7f37cffff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f37d01536f0 0x7f37d0153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cf7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f37d00700f0 0x7f37d00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cffff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f37d01536f0 0x7f37d0153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:53572/0 (socket says 10.20.193.16:53572) 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cffff640 1 -- 10.20.193.16:0/3392579512 learned_addr learned my addr 10.20.193.16:0/3392579512 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cffff640 10 monclient(hunting): get_auth_request con 0x7f37d01536f0 auth_method 0 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cf7fe640 10 monclient(hunting): get_auth_request con 0x7f37d00700f0 auth_method 0 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.989+0000 7f37cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.990+0000 7f37cffff640 10 monclient(hunting): handle_auth_done global_id 4557 payload 274 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.990+0000 7f37cffff640 1 -- 10.20.193.16:0/3392579512 >> v2:10.20.193.148:3300/0 conn(0x7f37d00700f0 msgr2=0x7f37d00704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.990+0000 7f37cffff640 1 --2- 10.20.193.16:0/3392579512 >> v2:10.20.193.148:3300/0 conn(0x7f37d00700f0 0x7f37d00704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.990+0000 7f37cffff640 1 -- 10.20.193.16:0/3392579512 >> v1:10.20.193.16:6789/0 conn(0x7f37d00c7fe0 legacy=0x7f37d00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.990+0000 7f37cffff640 10 monclient: _finish_hunting 0 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.990+0000 7f37cffff640 1 monclient: found mon.noname-a 2026-02-17T05:57:10.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.990+0000 7f37cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:10.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.990+0000 7f37cffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:10.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.990+0000 7f37cffff640 1 -- 10.20.193.16:0/3392579512 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d00c50c0 con 0x7f37d01536f0 2026-02-17T05:57:10.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.991+0000 7f37cffff640 1 --2- 10.20.193.16:0/3392579512 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d01536f0 0x7f37d0153b20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f37d0070b90 tx=0x7f37c402f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4c3c57eeedde90 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:11.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 1 -- 10.20.193.16:0/3392579512 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c402fc50 con 0x7f37d01536f0 2026-02-17T05:57:11.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:11.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 10 monclient: dump: 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:11.003 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:11.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 10 monclient: _finish_auth 0 2026-02-17T05:57:11.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:57:11.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:11.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 5 monclient: authenticate success, global_id 4557 2026-02-17T05:57:11.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 1 -- 10.20.193.16:0/3392579512 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c402fdf0 con 0x7f37d01536f0 2026-02-17T05:57:11.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:11.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:11.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 1 -- 10.20.193.16:0/3392579512 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4037610 con 0x7f37d01536f0 2026-02-17T05:57:11.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:11.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 4 set_mon_vals no callback set 2026-02-17T05:57:11.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:11.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:11.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37ce7fc640 10 monclient: dump: 2026-02-17T05:57:11.005 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:11.005 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:11.005 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:11.005 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:11.005 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:11.005 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:11.005 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 10 monclient: shutdown 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 1 -- 10.20.193.16:0/3392579512 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d01536f0 msgr2=0x7f37d0153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 1 --2- 10.20.193.16:0/3392579512 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d01536f0 0x7f37d0153b20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f37d0070b90 tx=0x7f37c402f230 comp rx=0 tx=0).stop 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 1 -- 10.20.193.16:0/3392579512 shutdown_connections 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 1 --2- 10.20.193.16:0/3392579512 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d01536f0 0x7f37d0153b20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 1 --2- 10.20.193.16:0/3392579512 >> v2:10.20.193.148:3300/0 conn(0x7f37d00700f0 0x7f37d00704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 1 -- 10.20.193.16:0/3392579512 >> 10.20.193.16:0/3392579512 conn(0x7f37d00cd770 msgr2=0x7f37d00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:11.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 1 -- 10.20.193.16:0/3392579512 shutdown_connections 2026-02-17T05:57:11.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.993+0000 7f37d5b14640 1 -- 10.20.193.16:0/3392579512 wait complete. 2026-02-17T05:57:11.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.994+0000 7f37d5b14640 10 monclient: build_initial_monmap 2026-02-17T05:57:11.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.994+0000 7f37d5b14640 10 monclient: monmap: 2026-02-17T05:57:11.007 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:11.007 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:11.007 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:11.007 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:11.007 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:11.008 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:11.008 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:11.008 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:11.008 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:11.008 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:11.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.994+0000 7f37d5b14640 1 librados: starting msgr at 2026-02-17T05:57:11.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.994+0000 7f37d5b14640 1 librados: starting objecter 2026-02-17T05:57:11.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.994+0000 7f37d5b14640 1 Processor -- start 2026-02-17T05:57:11.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.994+0000 7f37d5b14640 1 -- start start 2026-02-17T05:57:11.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.994+0000 7f37d5b14640 1 librados: setting wanted keys 2026-02-17T05:57:11.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.994+0000 7f37d5b14640 1 librados: calling monclient init 2026-02-17T05:57:11.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.994+0000 7f37d5b14640 10 monclient: init 2026-02-17T05:57:11.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:11.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:57:11.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f37d00700f0 0x7f37d0150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:11.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 10 monclient(hunting): picked mon.noname-b con 0x7f37d00700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:11.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d00c7fe0 0x7f37d0141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 10 monclient(hunting): picked mon.noname-a con 0x7f37d00c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f37d01536f0 0x7f37d01f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 10 monclient(hunting): picked mon.noname-c con 0x7f37d01536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0154060 con 0x7f37d00c7fe0 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f37d00c5680 con 0x7f37d00700f0 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0074c00 con 0x7f37d01536f0 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37d5b14640 10 monclient(hunting): authenticate will time out at 1083.564098s 2026-02-17T05:57:11.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37ceffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d00c7fe0 0x7f37d0141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37ceffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d00c7fe0 0x7f37d0141fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:53584/0 (socket says 10.20.193.16:53584) 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37ceffd640 1 -- 10.20.193.16:0/2035366409 learned_addr learned my addr 10.20.193.16:0/2035366409 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37ceffd640 10 monclient(hunting): get_auth_request con 0x7f37d00c7fe0 auth_method 0 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cffff640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f37d01536f0 0x7f37d01f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cf7fe640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f37d00700f0 0x7f37d0150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:11.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cffff640 10 monclient(hunting): get_auth_request con 0x7f37d01536f0 auth_method 0 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cf7fe640 10 monclient(hunting): get_auth_request con 0x7f37d00700f0 auth_method 0 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.995+0000 7f37cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:11.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37ceffd640 10 monclient(hunting): handle_auth_done global_id 4560 payload 824 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37ceffd640 1 -- 10.20.193.16:0/2035366409 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f37d01536f0 msgr2=0x7f37d01f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37ceffd640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f37d01536f0 0x7f37d01f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37ceffd640 1 -- 10.20.193.16:0/2035366409 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f37d00700f0 msgr2=0x7f37d0150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37ceffd640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f37d00700f0 0x7f37d0150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37ceffd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37ceffd640 1 monclient: found mon.noname-a 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37ceffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.996+0000 7f37ceffd640 1 -- 10.20.193.16:0/2035366409 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d0140c80 con 0x7f37d00c7fe0 2026-02-17T05:57:11.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.997+0000 7f37ceffd640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d00c7fe0 0x7f37d0141fc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f37c000b570 tx=0x7f37c000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.997+0000 7f37abfff640 10 client.?.objecter ms_handle_connect 0x7f37d00c7fe0 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.997+0000 7f37abfff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37abfff640 1 -- 10.20.193.16:0/2035366409 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0013020 con 0x7f37d00c7fe0 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37abfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37abfff640 10 monclient: dump: 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:11.014 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:11.015 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:11.015 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:11.015 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:11.015 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:11.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37abfff640 10 monclient: _finish_auth 0 2026-02-17T05:57:11.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37abfff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:11.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:11.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37d5b14640 5 monclient: authenticate success, global_id 4560 2026-02-17T05:57:11.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37d5b14640 10 monclient: _renew_subs 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37d5b14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37d5b14640 1 -- 10.20.193.16:0/2035366409 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d0140f20 con 0x7f37d00c7fe0 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37d5b14640 10 client.4560.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37d5b14640 10 monclient: _renew_subs 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37d5b14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37d5b14640 1 -- 10.20.193.16:0/2035366409 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d01417c0 con 0x7f37d00c7fe0 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.998+0000 7f37d5b14640 1 librados: init done 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37d5b14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37d5b14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37d5b14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37d5b14640 1 -- 10.20.193.16:0/2035366409 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d00704d0 con 0x7f37d00c7fe0 2026-02-17T05:57:11.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37abfff640 1 -- 10.20.193.16:0/2035366409 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c0004480 con 0x7f37d00c7fe0 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37abfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37abfff640 1 -- 10.20.193.16:0/2035366409 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0009380 con 0x7f37d00c7fe0 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37abfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37cdffb640 4 set_mon_vals no callback set 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:10.999+0000 7f37abfff640 10 monclient: dump: 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:11.017 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:11.018 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:11.018 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:11.018 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:11.018 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:11.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.069+0000 7f37abfff640 1 -- 10.20.193.16:0/2035366409 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f37c000f460 con 0x7f37d00c7fe0 2026-02-17T05:57:11.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.070+0000 7f37abfff640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37a0079140 0x7f37a007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:11.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.070+0000 7f37cf7fe640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37a0079140 0x7f37a007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.070+0000 7f37cf7fe640 10 monclient: get_auth_request con 0x7f37a0079140 auth_method 0 2026-02-17T05:57:11.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.070+0000 7f37cf7fe640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37a0079140 0x7f37a007b600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f37d0074800 tx=0x7f37bc009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:11.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.070+0000 7f37abfff640 10 client.4560.objecter ms_handle_connect 0x7f37a0079140 2026-02-17T05:57:11.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.108+0000 7f37abfff640 1 -- 10.20.193.16:0/2035366409 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f37c005e1a0 con 0x7f37d00c7fe0 2026-02-17T05:57:11.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.108+0000 7f37abfff640 10 client.4560.objecter ms_dispatch2 0x7f37d0000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:11.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.108+0000 7f37abfff640 3 client.4560.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:11.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.108+0000 7f37abfff640 3 client.4560.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:11.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.108+0000 7f37abfff640 20 client.4560.objecter dump_active .. 0 homeless 2026-02-17T05:57:11.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.116+0000 7f37abfff640 1 -- 10.20.193.16:0/2035366409 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f37c001d020 con 0x7f37d00c7fe0 2026-02-17T05:57:11.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.116+0000 7f37abfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:11.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.116+0000 7f37abfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:11.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.216+0000 7f37d5b14640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:11.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.216+0000 7f37d5b14640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:11.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.216+0000 7f37d5b14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:11.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.216+0000 7f37d5b14640 1 -- 10.20.193.16:0/2035366409 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f37d01573f0 con 0x7f37d00c7fe0 2026-02-17T05:57:11.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.220+0000 7f37abfff640 1 -- 10.20.193.16:0/2035366409 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f37c0062160 con 0x7f37d00c7fe0 2026-02-17T05:57:11.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.220+0000 7f37abfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:11.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.220+0000 7f37abfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:11.221 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:11.221 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:11.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 10 librados: watch_flush enter 2026-02-17T05:57:11.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 10 librados: watch_flush exit 2026-02-17T05:57:11.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 20 client.4560.objecter shutdown clearing up homeless session... 2026-02-17T05:57:11.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 10 client.4560.objecter successfully canceled tick 2026-02-17T05:57:11.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 -- 10.20.193.16:0/2035366409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37a0079140 msgr2=0x7f37a007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37a0079140 0x7f37a007b600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f37d0074800 tx=0x7f37bc009290 comp rx=0 tx=0).stop 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 10 monclient: shutdown 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 -- 10.20.193.16:0/2035366409 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d00c7fe0 msgr2=0x7f37d0141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d00c7fe0 0x7f37d0141fc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f37c000b570 tx=0x7f37c000ba40 comp rx=0 tx=0).stop 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 -- 10.20.193.16:0/2035366409 shutdown_connections 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37a0079140 0x7f37a007b600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f37d01536f0 0x7f37d01f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37d00c7fe0 0x7f37d0141fc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 --2- 10.20.193.16:0/2035366409 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f37d00700f0 0x7f37d0150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 -- 10.20.193.16:0/2035366409 >> 10.20.193.16:0/2035366409 conn(0x7f37d00cd770 msgr2=0x7f37d014e980 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:11.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 -- 10.20.193.16:0/2035366409 shutdown_connections 2026-02-17T05:57:11.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 -- 10.20.193.16:0/2035366409 wait complete. 2026-02-17T05:57:11.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:11.221+0000 7f37d5b14640 1 librados: shutdown 2026-02-17T05:57:14.232 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:14.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.298+0000 7fc7c9ab0640 10 monclient: get_monmap_and_config 2026-02-17T05:57:14.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.298+0000 7fc7c9ab0640 10 monclient: build_initial_monmap 2026-02-17T05:57:14.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.298+0000 7fc7c9ab0640 10 monclient: monmap: 2026-02-17T05:57:14.299 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:14.299449+0000 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:14.299449+0000 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:14.300 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:14.301 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:14.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.298+0000 7fc7c9ab0640 1 Processor -- start 2026-02-17T05:57:14.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.298+0000 7fc7c9ab0640 1 -- start start 2026-02-17T05:57:14.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.298+0000 7fc7c9ab0640 10 monclient: init 2026-02-17T05:57:14.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:14.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 10 monclient: _add_conns ranks=[1,4,5,0,3,2] 2026-02-17T05:57:14.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc7c40700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fc7c40c86f0 0x7fc7c40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 10 monclient(hunting): picked mon.noname-b con 0x7fc7c40c86f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fc7c41536f0 0x7fc7c4153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 10 monclient(hunting): picked mon.noname-c con 0x7fc7c41536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7c40c4820 con 0x7fc7c40700f0 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c40c4b00 con 0x7fc7c40c86f0 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c40c4de0 con 0x7fc7c41536f0 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:14.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c9ab0640 10 monclient(hunting): authenticate will time out at 1086.868497s 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.299+0000 7fc7c37fe640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fc7c40700f0 0x7fc7c40c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:54736/0 (socket says 10.20.193.16:54736) 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c37fe640 1 -- 10.20.193.16:0/2418212920 learned_addr learned my addr 10.20.193.16:0/2418212920 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c3fff640 1 --2- 10.20.193.16:0/2418212920 >> v2:10.20.193.148:3300/0 conn(0x7fc7c41536f0 0x7fc7c4153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c3fff640 10 monclient(hunting): get_auth_request con 0x7fc7c41536f0 auth_method 0 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c2ffd640 1 --2- 10.20.193.16:0/2418212920 >> v2:10.20.193.128:3300/0 conn(0x7fc7c40c86f0 0x7fc7c40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:14.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c2ffd640 10 monclient(hunting): get_auth_request con 0x7fc7c40c86f0 auth_method 0 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.300+0000 7fc7c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c3fff640 10 monclient(hunting): handle_auth_done global_id 4691 payload 274 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c3fff640 1 -- 10.20.193.16:0/2418212920 >> v2:10.20.193.128:3300/0 conn(0x7fc7c40c86f0 msgr2=0x7fc7c40704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c3fff640 1 --2- 10.20.193.16:0/2418212920 >> v2:10.20.193.128:3300/0 conn(0x7fc7c40c86f0 0x7fc7c40704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c3fff640 1 -- 10.20.193.16:0/2418212920 >> v1:10.20.193.128:6789/0 conn(0x7fc7c40700f0 legacy=0x7fc7c40c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:57:14.308 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c3fff640 1 monclient: found mon.noname-c 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.301+0000 7fc7c3fff640 1 -- 10.20.193.16:0/2418212920 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c40c50c0 con 0x7fc7c41536f0 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.302+0000 7fc7c3fff640 1 --2- 10.20.193.16:0/2418212920 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 0x7fc7c4153b20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40c9540 tx=0x7fc7b802f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac29863f03be811 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 1 -- 10.20.193.16:0/2418212920 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b803c070 con 0x7fc7c41536f0 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 10 monclient: dump: 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:14.309 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 10 monclient: _finish_auth 0 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c9ab0640 5 monclient: authenticate success, global_id 4691 2026-02-17T05:57:14.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c9ab0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 1 -- 10.20.193.16:0/2418212920 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b802fe40 con 0x7fc7c41536f0 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 1 -- 10.20.193.16:0/2418212920 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8037660 con 0x7fc7c41536f0 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.303+0000 7fc7c27fc640 10 monclient: dump: 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:14.311 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 4 set_mon_vals no callback set 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 10 monclient: shutdown 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2418212920 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 msgr2=0x7fc7c4153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 1 --2- 10.20.193.16:0/2418212920 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 0x7fc7c4153b20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40c9540 tx=0x7fc7b802f130 comp rx=0 tx=0).stop 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2418212920 shutdown_connections 2026-02-17T05:57:14.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 1 --2- 10.20.193.16:0/2418212920 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 0x7fc7c4153b20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 1 --2- 10.20.193.16:0/2418212920 >> v2:10.20.193.128:3300/0 conn(0x7fc7c40c86f0 0x7fc7c40704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2418212920 >> 10.20.193.16:0/2418212920 conn(0x7fc7c40cd770 msgr2=0x7fc7c40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2418212920 shutdown_connections 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2418212920 wait complete. 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 10 monclient: build_initial_monmap 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.304+0000 7fc7c9ab0640 10 monclient: monmap: 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:14.313 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:14.314 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:14.314 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:14.314 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:14.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 librados: starting msgr at 2026-02-17T05:57:14.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 librados: starting objecter 2026-02-17T05:57:14.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 Processor -- start 2026-02-17T05:57:14.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 -- start start 2026-02-17T05:57:14.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 librados: setting wanted keys 2026-02-17T05:57:14.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 librados: calling monclient init 2026-02-17T05:57:14.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient: init 2026-02-17T05:57:14.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:14.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:57:14.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc7c40700f0 0x7fc7c41f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient(hunting): picked mon.noname-a con 0x7fc7c40700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc7c40c86f0 0x7fc7c41f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient(hunting): picked mon.noname-b con 0x7fc7c40c86f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 0x7fc7c413c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient(hunting): picked mon.noname-c con 0x7fc7c41536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c4154060 con 0x7fc7c40700f0 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c40c5680 con 0x7fc7c40c86f0 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:14.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c4074c00 con 0x7fc7c41536f0 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c9ab0640 10 monclient(hunting): authenticate will time out at 1086.874645s 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c3fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 0x7fc7c413c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.305+0000 7fc7c37fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc7c40700f0 0x7fc7c41f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c2ffd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc7c40c86f0 0x7fc7c41f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c3fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 0x7fc7c413c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:50866/0 (socket says 10.20.193.16:50866) 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c3fff640 1 -- 10.20.193.16:0/2713463401 learned_addr learned my addr 10.20.193.16:0/2713463401 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c3fff640 10 monclient(hunting): get_auth_request con 0x7fc7c41536f0 auth_method 0 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:14.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c2ffd640 10 monclient(hunting): get_auth_request con 0x7fc7c40c86f0 auth_method 0 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c37fe640 10 monclient(hunting): get_auth_request con 0x7fc7c40700f0 auth_method 0 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:14.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.306+0000 7fc7c37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.307+0000 7fc7c3fff640 10 monclient(hunting): handle_auth_done global_id 4694 payload 824 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.307+0000 7fc7c3fff640 1 -- 10.20.193.16:0/2713463401 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc7c40c86f0 msgr2=0x7fc7c41f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.307+0000 7fc7c3fff640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc7c40c86f0 0x7fc7c41f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.307+0000 7fc7c3fff640 1 -- 10.20.193.16:0/2713463401 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc7c40700f0 msgr2=0x7fc7c41f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.307+0000 7fc7c3fff640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc7c40700f0 0x7fc7c41f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.307+0000 7fc7c3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.307+0000 7fc7c3fff640 1 monclient: found mon.noname-c 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.307+0000 7fc7c3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:14.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.307+0000 7fc7c3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.307+0000 7fc7c3fff640 1 -- 10.20.193.16:0/2713463401 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c41516b0 con 0x7fc7c41536f0 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.308+0000 7fc7c3fff640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 0x7fc7c413c980 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc7b802f660 tx=0x7fc7b8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.308+0000 7fc79ffff640 10 client.?.objecter ms_handle_connect 0x7fc7c41536f0 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.308+0000 7fc79ffff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc79ffff640 1 -- 10.20.193.16:0/2713463401 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8047020 con 0x7fc7c41536f0 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc79ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc79ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc79ffff640 10 monclient: dump: 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:14.324 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc79ffff640 10 monclient: _finish_auth 0 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc79ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc79ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc7c9ab0640 5 monclient: authenticate success, global_id 4694 2026-02-17T05:57:14.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc7c9ab0640 10 monclient: _renew_subs 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc7c9ab0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2713463401 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c4151990 con 0x7fc7c41536f0 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc7c9ab0640 10 client.4694.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc7c9ab0640 10 monclient: _renew_subs 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc7c9ab0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2713463401 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c4151f50 con 0x7fc7c41536f0 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.309+0000 7fc7c9ab0640 1 librados: init done 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc79ffff640 1 -- 10.20.193.16:0/2713463401 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b8042470 con 0x7fc7c41536f0 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc79ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc79ffff640 1 -- 10.20.193.16:0/2713463401 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b803c040 con 0x7fc7c41536f0 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc79ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc79ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:14.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc79ffff640 10 monclient: dump: 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc7c1ffb640 4 set_mon_vals no callback set 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc7c9ab0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:14.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc7c9ab0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:14.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc7c9ab0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:14.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.310+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2713463401 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c40698c0 con 0x7fc7c41536f0 2026-02-17T05:57:14.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.414+0000 7fc79ffff640 1 -- 10.20.193.16:0/2713463401 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fc7b8042820 con 0x7fc7c41536f0 2026-02-17T05:57:14.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.415+0000 7fc79ffff640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc794079170 0x7fc79407b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:14.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.415+0000 7fc7c37fe640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc794079170 0x7fc79407b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:14.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.415+0000 7fc7c37fe640 10 monclient: get_auth_request con 0x7fc794079170 auth_method 0 2026-02-17T05:57:14.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.415+0000 7fc7c37fe640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc794079170 0x7fc79407b630 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0002770 tx=0x7fc7b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:14.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.415+0000 7fc79ffff640 10 client.4694.objecter ms_handle_connect 0x7fc794079170 2026-02-17T05:57:14.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.417+0000 7fc79ffff640 1 -- 10.20.193.16:0/2713463401 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fc7b8083f90 con 0x7fc7c41536f0 2026-02-17T05:57:14.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.417+0000 7fc79ffff640 10 client.4694.objecter ms_dispatch2 0x7fc7c4000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:14.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.417+0000 7fc79ffff640 3 client.4694.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:14.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.417+0000 7fc79ffff640 3 client.4694.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:14.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.417+0000 7fc79ffff640 20 client.4694.objecter dump_active .. 0 homeless 2026-02-17T05:57:14.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.423+0000 7fc79ffff640 1 -- 10.20.193.16:0/2713463401 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc7c40698c0 con 0x7fc7c41536f0 2026-02-17T05:57:14.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.423+0000 7fc79ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:14.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.423+0000 7fc79ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:14.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.521+0000 7fc7c9ab0640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:14.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.521+0000 7fc7c9ab0640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:14.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.521+0000 7fc7c9ab0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:14.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.521+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2713463401 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc7c414dc00 con 0x7fc7c41536f0 2026-02-17T05:57:14.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.524+0000 7fc79ffff640 1 -- 10.20.193.16:0/2713463401 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc7c414dc00 con 0x7fc7c41536f0 2026-02-17T05:57:14.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.524+0000 7fc79ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:14.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.524+0000 7fc79ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:14.525 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:14.525 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:14.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 10 librados: watch_flush enter 2026-02-17T05:57:14.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 10 librados: watch_flush exit 2026-02-17T05:57:14.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 20 client.4694.objecter shutdown clearing up homeless session... 2026-02-17T05:57:14.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 10 client.4694.objecter successfully canceled tick 2026-02-17T05:57:14.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2713463401 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc794079170 msgr2=0x7fc79407b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:14.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc794079170 0x7fc79407b630 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0002770 tx=0x7fc7b0009040 comp rx=0 tx=0).stop 2026-02-17T05:57:14.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 10 monclient: shutdown 2026-02-17T05:57:14.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:14.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2713463401 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 msgr2=0x7fc7c413c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:14.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 0x7fc7c413c980 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc7b802f660 tx=0x7fc7b8004b40 comp rx=0 tx=0).stop 2026-02-17T05:57:14.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2713463401 shutdown_connections 2026-02-17T05:57:14.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc794079170 0x7fc79407b630 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:14.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc7c40c86f0 0x7fc7c41f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:14.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc7c41536f0 0x7fc7c413c980 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:14.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 --2- 10.20.193.16:0/2713463401 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc7c40700f0 0x7fc7c41f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:14.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2713463401 >> 10.20.193.16:0/2713463401 conn(0x7fc7c40cd770 msgr2=0x7fc7c414e650 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:14.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2713463401 shutdown_connections 2026-02-17T05:57:14.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 -- 10.20.193.16:0/2713463401 wait complete. 2026-02-17T05:57:14.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:14.525+0000 7fc7c9ab0640 1 librados: shutdown 2026-02-17T05:57:17.535 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:17.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.604+0000 7f1df9322640 10 monclient: get_monmap_and_config 2026-02-17T05:57:17.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.604+0000 7f1df9322640 10 monclient: build_initial_monmap 2026-02-17T05:57:17.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.604+0000 7f1df9322640 10 monclient: monmap: 2026-02-17T05:57:17.609 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:17.609 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:17.609 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:17.605146+0000 2026-02-17T05:57:17.609 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:17.605146+0000 2026-02-17T05:57:17.609 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:17.610 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:17.610 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:17.610 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:17.610 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:17.610 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:17.610 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:17.610 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:17.610 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:17.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.604+0000 7f1df9322640 1 Processor -- start 2026-02-17T05:57:17.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.604+0000 7f1df9322640 1 -- start start 2026-02-17T05:57:17.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.604+0000 7f1df9322640 10 monclient: init 2026-02-17T05:57:17.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:17.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 10 monclient: _add_conns ranks=[3,5,0,4,1,2] 2026-02-17T05:57:17.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1df414f020 0x7f1df406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 10 monclient(hunting): picked mon.noname-a con 0x7f1df414f020 addr v2:10.20.193.16:3300/0 2026-02-17T05:57:17.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f1df406e120 0x7f1df415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 10 monclient(hunting): picked mon.noname-c con 0x7f1df406e120 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:17.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1df414f400 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:17.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df40c5b10 con 0x7f1df414f400 2026-02-17T05:57:17.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:17.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1df406e620 con 0x7f1df414f020 2026-02-17T05:57:17.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:17.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f1df406e900 con 0x7f1df406e120 2026-02-17T05:57:17.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:17.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df9322640 10 monclient(hunting): authenticate will time out at 1090.174206s 2026-02-17T05:57:17.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df2ffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1df414f020 0x7f1df406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:17.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df27fc640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f1df406e120 0x7f1df415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:17.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df2ffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1df414f020 0x7f1df406dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:53598/0 (socket says 10.20.193.16:53598) 2026-02-17T05:57:17.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df2ffd640 1 -- 10.20.193.16:0/2720461953 learned_addr learned my addr 10.20.193.16:0/2720461953 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:17.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df2ffd640 10 monclient(hunting): get_auth_request con 0x7f1df414f020 auth_method 0 2026-02-17T05:57:17.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:17.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:17.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:17.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df27fc640 10 monclient(hunting): get_auth_request con 0x7f1df406e120 auth_method 0 2026-02-17T05:57:17.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:17.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:17.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.605+0000 7f1df27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:17.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.606+0000 7f1df2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:17.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.606+0000 7f1df2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:17.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.606+0000 7f1df2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:17.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.606+0000 7f1df27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:17.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.606+0000 7f1df27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:17.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.606+0000 7f1df27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:17.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.607+0000 7f1df2ffd640 10 monclient(hunting): handle_auth_done global_id 4566 payload 274 2026-02-17T05:57:17.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.607+0000 7f1df2ffd640 1 -- 10.20.193.16:0/2720461953 >> v2:10.20.193.148:3300/0 conn(0x7f1df406e120 msgr2=0x7f1df415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:17.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.607+0000 7f1df2ffd640 1 --2- 10.20.193.16:0/2720461953 >> v2:10.20.193.148:3300/0 conn(0x7f1df406e120 0x7f1df415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.607+0000 7f1df2ffd640 1 -- 10.20.193.16:0/2720461953 >> v1:10.20.193.16:6789/0 conn(0x7f1df414f400 legacy=0x7f1df40c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:17.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.607+0000 7f1df2ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:17.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.607+0000 7f1df2ffd640 1 monclient: found mon.noname-a 2026-02-17T05:57:17.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.607+0000 7f1df2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:17.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.607+0000 7f1df2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:17.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.607+0000 7f1df2ffd640 1 -- 10.20.193.16:0/2720461953 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df406ebe0 con 0x7f1df414f020 2026-02-17T05:57:17.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.608+0000 7f1df2ffd640 1 --2- 10.20.193.16:0/2720461953 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 0x7f1df406dbe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1df4156400 tx=0x7f1de802ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46e74270d603e21 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:17.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 1 -- 10.20.193.16:0/2720461953 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de803c070 con 0x7f1df414f020 2026-02-17T05:57:17.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:17.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:17.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 10 monclient: dump: 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:17.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 5 monclient: authenticate success, global_id 4566 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 1 -- 10.20.193.16:0/2720461953 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de802fbd0 con 0x7f1df414f020 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 1 -- 10.20.193.16:0/2720461953 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8037440 con 0x7f1df414f020 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df1ffb640 10 monclient: dump: 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:17.628 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 4 set_mon_vals no callback set 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 10 monclient: shutdown 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:17.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 1 -- 10.20.193.16:0/2720461953 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 msgr2=0x7f1df406dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 1 --2- 10.20.193.16:0/2720461953 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 0x7f1df406dbe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1df4156400 tx=0x7f1de802ef60 comp rx=0 tx=0).stop 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 1 -- 10.20.193.16:0/2720461953 shutdown_connections 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 1 --2- 10.20.193.16:0/2720461953 >> v2:10.20.193.148:3300/0 conn(0x7f1df406e120 0x7f1df415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 1 --2- 10.20.193.16:0/2720461953 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 0x7f1df406dbe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 1 -- 10.20.193.16:0/2720461953 >> 10.20.193.16:0/2720461953 conn(0x7f1df40cd770 msgr2=0x7f1df40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 1 -- 10.20.193.16:0/2720461953 shutdown_connections 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.610+0000 7f1df9322640 1 -- 10.20.193.16:0/2720461953 wait complete. 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient: build_initial_monmap 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient: monmap: 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:17.630 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 librados: starting msgr at 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 librados: starting objecter 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 Processor -- start 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 -- start start 2026-02-17T05:57:17.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 librados: setting wanted keys 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 librados: calling monclient init 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient: init 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1df406e120 0x7f1df41f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient(hunting): picked mon.noname-b con 0x7f1df406e120 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 0x7f1df41f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient(hunting): picked mon.noname-a con 0x7f1df414f020 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1df414f400 0x7f1df413f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient(hunting): picked mon.noname-c con 0x7f1df414f400 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1df40c6720 con 0x7f1df414f020 2026-02-17T05:57:17.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1df406f1a0 con 0x7f1df406e120 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1df40c68a0 con 0x7f1df414f400 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.611+0000 7f1df9322640 10 monclient(hunting): authenticate will time out at 1090.180839s 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df27fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 0x7f1df41f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df2ffd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1df406e120 0x7f1df41f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df27fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 0x7f1df41f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:53602/0 (socket says 10.20.193.16:53602) 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df27fc640 1 -- 10.20.193.16:0/4208726764 learned_addr learned my addr 10.20.193.16:0/4208726764 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df27fc640 10 monclient(hunting): get_auth_request con 0x7f1df414f020 auth_method 0 2026-02-17T05:57:17.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df2ffd640 10 monclient(hunting): get_auth_request con 0x7f1df406e120 auth_method 0 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df37fe640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1df414f400 0x7f1df413f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df37fe640 10 monclient(hunting): get_auth_request con 0x7f1df414f400 auth_method 0 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:17.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.612+0000 7f1df37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.613+0000 7f1df27fc640 10 monclient(hunting): handle_auth_done global_id 4569 payload 824 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.613+0000 7f1df27fc640 1 -- 10.20.193.16:0/4208726764 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1df414f400 msgr2=0x7f1df413f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.613+0000 7f1df27fc640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1df414f400 0x7f1df413f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.613+0000 7f1df27fc640 1 -- 10.20.193.16:0/4208726764 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1df406e120 msgr2=0x7f1df41f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.613+0000 7f1df27fc640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1df406e120 0x7f1df41f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.613+0000 7f1df27fc640 10 monclient: _finish_hunting 0 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.613+0000 7f1df27fc640 1 monclient: found mon.noname-a 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.613+0000 7f1df27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.613+0000 7f1df27fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.613+0000 7f1df27fc640 1 -- 10.20.193.16:0/4208726764 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df41f4ce0 con 0x7f1df414f020 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.614+0000 7f1df27fc640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 0x7f1df41f3de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00e890 tx=0x7f1ddc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.614+0000 7f1dd37fe640 10 client.?.objecter ms_handle_connect 0x7f1df414f020 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.614+0000 7f1dd37fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 1 -- 10.20.193.16:0/4208726764 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc017070 con 0x7f1df414f020 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 10 monclient: dump: 2026-02-17T05:57:17.636 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:17.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 1 -- 10.20.193.16:0/4208726764 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ddc00c940 con 0x7f1df414f020 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 1 -- 10.20.193.16:0/4208726764 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc00cd10 con 0x7f1df414f020 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1dd37fe640 10 monclient: dump: 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:17.638 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df17fa640 4 set_mon_vals no callback set 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 5 monclient: authenticate success, global_id 4569 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 10 monclient: _renew_subs 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 1 -- 10.20.193.16:0/4208726764 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df41f4440 con 0x7f1df414f020 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 10 client.4569.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 10 monclient: _renew_subs 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 1 -- 10.20.193.16:0/4208726764 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df41f4a00 con 0x7f1df414f020 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 1 librados: init done 2026-02-17T05:57:17.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:17.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:17.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:17.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.616+0000 7f1df9322640 1 -- 10.20.193.16:0/4208726764 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df406dbe0 con 0x7f1df414f020 2026-02-17T05:57:17.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.628+0000 7f1dd37fe640 1 -- 10.20.193.16:0/4208726764 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1ddc024060 con 0x7f1df414f020 2026-02-17T05:57:17.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.629+0000 7f1dd37fe640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1dc0079140 0x7f1dc007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.629+0000 7f1df2ffd640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1dc0079140 0x7f1dc007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:17.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.629+0000 7f1df2ffd640 10 monclient: get_auth_request con 0x7f1dc0079140 auth_method 0 2026-02-17T05:57:17.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.629+0000 7f1dd37fe640 10 client.4569.objecter ms_handle_connect 0x7f1dc0079140 2026-02-17T05:57:17.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.629+0000 7f1df2ffd640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1dc0079140 0x7f1dc007b600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1df40c9d60 tx=0x7f1de8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:17.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.631+0000 7f1dd37fe640 1 -- 10.20.193.16:0/4208726764 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1ddc05e2e0 con 0x7f1df414f020 2026-02-17T05:57:17.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.631+0000 7f1dd37fe640 10 client.4569.objecter ms_dispatch2 0x7f1df4000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:17.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.631+0000 7f1dd37fe640 3 client.4569.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:17.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.631+0000 7f1dd37fe640 3 client.4569.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:17.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.631+0000 7f1dd37fe640 20 client.4569.objecter dump_active .. 0 homeless 2026-02-17T05:57:17.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.732+0000 7f1dd37fe640 1 -- 10.20.193.16:0/4208726764 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1ddc0622a0 con 0x7f1df414f020 2026-02-17T05:57:17.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.732+0000 7f1dd37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:17.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.732+0000 7f1dd37fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:17.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.831+0000 7f1df9322640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:17.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.831+0000 7f1df9322640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:17.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.831+0000 7f1df9322640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:17.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.831+0000 7f1df9322640 1 -- 10.20.193.16:0/4208726764 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1df41562c0 con 0x7f1df414f020 2026-02-17T05:57:17.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.834+0000 7f1dd37fe640 1 -- 10.20.193.16:0/4208726764 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1ddc0665a0 con 0x7f1df414f020 2026-02-17T05:57:17.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.834+0000 7f1dd37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:17.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.834+0000 7f1dd37fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:17.835 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:17.835 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:17.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 10 librados: watch_flush enter 2026-02-17T05:57:17.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 10 librados: watch_flush exit 2026-02-17T05:57:17.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 20 client.4569.objecter shutdown clearing up homeless session... 2026-02-17T05:57:17.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 10 client.4569.objecter successfully canceled tick 2026-02-17T05:57:17.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 -- 10.20.193.16:0/4208726764 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1dc0079140 msgr2=0x7f1dc007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:17.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1dc0079140 0x7f1dc007b600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1df40c9d60 tx=0x7f1de8033000 comp rx=0 tx=0).stop 2026-02-17T05:57:17.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 10 monclient: shutdown 2026-02-17T05:57:17.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 -- 10.20.193.16:0/4208726764 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 msgr2=0x7f1df41f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 0x7f1df41f3de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00e890 tx=0x7f1ddc00ed60 comp rx=0 tx=0).stop 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 -- 10.20.193.16:0/4208726764 shutdown_connections 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1dc0079140 0x7f1dc007b600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1df414f400 0x7f1df413f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1df414f020 0x7f1df41f3de0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 --2- 10.20.193.16:0/4208726764 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1df406e120 0x7f1df41f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 -- 10.20.193.16:0/4208726764 >> 10.20.193.16:0/4208726764 conn(0x7f1df40cd770 msgr2=0x7f1df406fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 -- 10.20.193.16:0/4208726764 shutdown_connections 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 -- 10.20.193.16:0/4208726764 wait complete. 2026-02-17T05:57:17.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:17.835+0000 7f1df9322640 1 librados: shutdown 2026-02-17T05:57:20.845 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:20.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.915+0000 7f8db2af2640 10 monclient: get_monmap_and_config 2026-02-17T05:57:20.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.915+0000 7f8db2af2640 10 monclient: build_initial_monmap 2026-02-17T05:57:20.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.915+0000 7f8db2af2640 10 monclient: monmap: 2026-02-17T05:57:20.916 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:20.917 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:20.917 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:20.916389+0000 2026-02-17T05:57:20.917 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:20.916389+0000 2026-02-17T05:57:20.917 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:20.917 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:20.917 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:20.917 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:20.917 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:20.917 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:20.917 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:20.918 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:20.918 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:20.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.915+0000 7f8db2af2640 1 Processor -- start 2026-02-17T05:57:20.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.915+0000 7f8db2af2640 1 -- start start 2026-02-17T05:57:20.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.915+0000 7f8db2af2640 10 monclient: init 2026-02-17T05:57:20.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:20.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 10 monclient: _add_conns ranks=[3,4,0,1,2,5] 2026-02-17T05:57:20.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f8dac0700f0 0x7f8dac0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 10 monclient(hunting): picked mon.noname-a con 0x7f8dac0700f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:57:20.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f8dac0c7fe0 0x7f8dac0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 10 monclient(hunting): picked mon.noname-b con 0x7f8dac0c7fe0 addr v2:10.20.193.128:3300/0 2026-02-17T05:57:20.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8dac1536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:20.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dac0c8970 con 0x7f8dac1536f0 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8dac0c4820 con 0x7f8dac0700f0 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f8dac0c4b00 con 0x7f8dac0c7fe0 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db2af2640 10 monclient(hunting): authenticate will time out at 1093.485441s 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8db0867640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f8dac0700f0 0x7f8dac0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.916+0000 7f8da3fff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f8dac0c7fe0 0x7f8dac0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8db0867640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f8dac0700f0 0x7f8dac0704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:58114/0 (socket says 10.20.193.16:58114) 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8db0867640 1 -- 10.20.193.16:0/3200932240 learned_addr learned my addr 10.20.193.16:0/3200932240 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:20.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8db0867640 10 monclient(hunting): get_auth_request con 0x7f8dac0700f0 auth_method 0 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8db0867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8db0867640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8db0867640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8da3fff640 10 monclient(hunting): get_auth_request con 0x7f8dac0c7fe0 auth_method 0 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8da3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8da3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8da3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8db0867640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8db0867640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8db0867640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8da3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:20.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8da3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:20.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.917+0000 7f8da3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:20.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.918+0000 7f8db0867640 10 monclient(hunting): handle_auth_done global_id 4572 payload 274 2026-02-17T05:57:20.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.918+0000 7f8db0867640 1 -- 10.20.193.16:0/3200932240 >> v2:10.20.193.128:3300/0 conn(0x7f8dac0c7fe0 msgr2=0x7f8dac0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:20.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.918+0000 7f8db0867640 1 --2- 10.20.193.16:0/3200932240 >> v2:10.20.193.128:3300/0 conn(0x7f8dac0c7fe0 0x7f8dac0c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:20.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.918+0000 7f8db0867640 1 -- 10.20.193.16:0/3200932240 >> v1:10.20.193.16:6789/0 conn(0x7f8dac1536f0 legacy=0x7f8dac153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:20.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.918+0000 7f8db0867640 10 monclient: _finish_hunting 0 2026-02-17T05:57:20.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.918+0000 7f8db0867640 1 monclient: found mon.noname-a 2026-02-17T05:57:20.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.918+0000 7f8db0867640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:20.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.918+0000 7f8db0867640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:20.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.918+0000 7f8db0867640 1 -- 10.20.193.16:0/3200932240 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac0c4de0 con 0x7f8dac0700f0 2026-02-17T05:57:20.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.919+0000 7f8db0867640 1 --2- 10.20.193.16:0/3200932240 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0700f0 0x7f8dac0704d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0c5b80 tx=0x7f8d9c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a013cae6e68f4b84 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:20.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.921+0000 7f8da37fe640 1 -- 10.20.193.16:0/3200932240 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c03c070 con 0x7f8dac0700f0 2026-02-17T05:57:20.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.921+0000 7f8da37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:20.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:20.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 10 monclient: dump: 2026-02-17T05:57:20.935 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:20.935 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:20.935 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:20.935 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:20.935 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:20.935 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:20.935 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 1 -- 10.20.193.16:0/3200932240 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d9c02fa90 con 0x7f8dac0700f0 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 5 monclient: authenticate success, global_id 4572 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:20.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 1 -- 10.20.193.16:0/3200932240 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c02fe60 con 0x7f8dac0700f0 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 4 set_mon_vals no callback set 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8da37fe640 10 monclient: dump: 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:20.937 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 10 monclient: shutdown 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 1 -- 10.20.193.16:0/3200932240 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0700f0 msgr2=0x7f8dac0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 1 --2- 10.20.193.16:0/3200932240 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0700f0 0x7f8dac0704d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0c5b80 tx=0x7f8d9c02ee60 comp rx=0 tx=0).stop 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 1 -- 10.20.193.16:0/3200932240 shutdown_connections 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 1 --2- 10.20.193.16:0/3200932240 >> v2:10.20.193.128:3300/0 conn(0x7f8dac0c7fe0 0x7f8dac0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 1 --2- 10.20.193.16:0/3200932240 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0700f0 0x7f8dac0704d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 1 -- 10.20.193.16:0/3200932240 >> 10.20.193.16:0/3200932240 conn(0x7f8dac0cd770 msgr2=0x7f8dac0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:20.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 1 -- 10.20.193.16:0/3200932240 shutdown_connections 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 1 -- 10.20.193.16:0/3200932240 wait complete. 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 10 monclient: build_initial_monmap 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.922+0000 7f8db2af2640 10 monclient: monmap: 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:20.939 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 librados: starting msgr at 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 librados: starting objecter 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 Processor -- start 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 -- start start 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 librados: setting wanted keys 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 librados: calling monclient init 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient: init 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:57:20.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8dac0700f0 0x7f8dac1f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient(hunting): picked mon.noname-c con 0x7f8dac0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0c7fe0 0x7f8dac142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient(hunting): picked mon.noname-a con 0x7f8dac0c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8dac1536f0 0x7f8dac0cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient(hunting): picked mon.noname-b con 0x7f8dac1536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac0c5680 con 0x7f8dac0c7fe0 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac0c53a0 con 0x7f8dac1536f0 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac074c00 con 0x7f8dac0700f0 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:20.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8db2af2640 10 monclient(hunting): authenticate will time out at 1093.492685s 2026-02-17T05:57:20.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.923+0000 7f8da3fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0c7fe0 0x7f8dac142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:20.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8da3fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0c7fe0 0x7f8dac142a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:58126/0 (socket says 10.20.193.16:58126) 2026-02-17T05:57:20.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8da3fff640 1 -- 10.20.193.16:0/3188641388 learned_addr learned my addr 10.20.193.16:0/3188641388 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:20.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8da3fff640 10 monclient(hunting): get_auth_request con 0x7f8dac0c7fe0 auth_method 0 2026-02-17T05:57:20.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8da3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:20.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8da3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:20.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8da3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:20.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db1068640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8dac1536f0 0x7f8dac0cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:20.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db0867640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8dac0700f0 0x7f8dac1f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:20.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8da3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8da3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8da3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db1068640 10 monclient(hunting): get_auth_request con 0x7f8dac1536f0 auth_method 0 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db1068640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db1068640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db1068640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db0867640 10 monclient(hunting): get_auth_request con 0x7f8dac0700f0 auth_method 0 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db0867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db0867640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db0867640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:20.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db1068640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:20.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db1068640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:20.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db1068640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:20.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db0867640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:20.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db0867640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:20.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.924+0000 7f8db0867640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:20.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.925+0000 7f8da3fff640 10 monclient(hunting): handle_auth_done global_id 4575 payload 824 2026-02-17T05:57:20.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.925+0000 7f8da3fff640 1 -- 10.20.193.16:0/3188641388 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8dac0700f0 msgr2=0x7f8dac1f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:20.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.925+0000 7f8da3fff640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8dac0700f0 0x7f8dac1f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:20.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.925+0000 7f8da3fff640 1 -- 10.20.193.16:0/3188641388 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8dac1536f0 msgr2=0x7f8dac0cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:20.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.925+0000 7f8da3fff640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8dac1536f0 0x7f8dac0cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.925+0000 7f8da3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.925+0000 7f8da3fff640 1 monclient: found mon.noname-a 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.925+0000 7f8da3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.925+0000 7f8da3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.925+0000 7f8da3fff640 1 -- 10.20.193.16:0/3188641388 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac0cb160 con 0x7f8dac0c7fe0 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.926+0000 7f8da3fff640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0c7fe0 0x7f8dac142a60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400e890 tx=0x7f8d9400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.926+0000 7f8da17fa640 10 client.?.objecter ms_handle_connect 0x7f8dac0c7fe0 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.926+0000 7f8da17fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 1 -- 10.20.193.16:0/3188641388 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94017070 con 0x7f8dac0c7fe0 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:20.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 10 monclient: dump: 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:20.946 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 10 monclient: _finish_auth 0 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 10 monclient: _check_auth_tickets 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 1 -- 10.20.193.16:0/3188641388 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d9400c940 con 0x7f8dac0c7fe0 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 1 -- 10.20.193.16:0/3188641388 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400cd30 con 0x7f8dac0c7fe0 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da2ffd640 4 set_mon_vals no callback set 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8da17fa640 10 monclient: dump: 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:20.947 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 5 monclient: authenticate success, global_id 4575 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 10 monclient: _renew_subs 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:20.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 1 -- 10.20.193.16:0/3188641388 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dac1f3e70 con 0x7f8dac0c7fe0 2026-02-17T05:57:20.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 10 client.4575.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:20.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 10 monclient: _renew_subs 2026-02-17T05:57:20.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:20.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 1 -- 10.20.193.16:0/3188641388 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dac1f4430 con 0x7f8dac0c7fe0 2026-02-17T05:57:20.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 1 librados: init done 2026-02-17T05:57:20.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:20.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:20.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:20.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.928+0000 7f8db2af2640 1 -- 10.20.193.16:0/3188641388 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dac0704d0 con 0x7f8dac0c7fe0 2026-02-17T05:57:20.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.944+0000 7f8da17fa640 1 -- 10.20.193.16:0/3188641388 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f8d94024060 con 0x7f8dac0c7fe0 2026-02-17T05:57:20.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.944+0000 7f8da17fa640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d7c079140 0x7f8d7c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.944+0000 7f8db0867640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d7c079140 0x7f8d7c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:20.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.944+0000 7f8db0867640 10 monclient: get_auth_request con 0x7f8d7c079140 auth_method 0 2026-02-17T05:57:20.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.945+0000 7f8da17fa640 10 client.4575.objecter ms_handle_connect 0x7f8d7c079140 2026-02-17T05:57:20.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:20.945+0000 7f8db0867640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d7c079140 0x7f8d7c07b600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0c9b90 tx=0x7f8d9c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:21.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.047+0000 7f8da17fa640 1 -- 10.20.193.16:0/3188641388 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f8d9405e2a0 con 0x7f8dac0c7fe0 2026-02-17T05:57:21.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.047+0000 7f8da17fa640 10 client.4575.objecter ms_dispatch2 0x7f8dac000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:21.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.047+0000 7f8da17fa640 3 client.4575.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:21.049 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.047+0000 7f8da17fa640 3 client.4575.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:21.049 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.047+0000 7f8da17fa640 20 client.4575.objecter dump_active .. 0 homeless 2026-02-17T05:57:21.049 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.047+0000 7f8da17fa640 1 -- 10.20.193.16:0/3188641388 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8d94062260 con 0x7f8dac0c7fe0 2026-02-17T05:57:21.049 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.047+0000 7f8da17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:21.049 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.047+0000 7f8da17fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:21.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.146+0000 7f8db2af2640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:21.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.146+0000 7f8db2af2640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:21.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.146+0000 7f8db2af2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:21.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.146+0000 7f8db2af2640 1 -- 10.20.193.16:0/3188641388 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8dac06f500 con 0x7f8dac0c7fe0 2026-02-17T05:57:21.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.148+0000 7f8da17fa640 1 -- 10.20.193.16:0/3188641388 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8d94066560 con 0x7f8dac0c7fe0 2026-02-17T05:57:21.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.148+0000 7f8da17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:21.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.148+0000 7f8da17fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:21.149 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:21.149 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:21.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 10 librados: watch_flush enter 2026-02-17T05:57:21.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 10 librados: watch_flush exit 2026-02-17T05:57:21.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 20 client.4575.objecter shutdown clearing up homeless session... 2026-02-17T05:57:21.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 10 client.4575.objecter successfully canceled tick 2026-02-17T05:57:21.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 -- 10.20.193.16:0/3188641388 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d7c079140 msgr2=0x7f8d7c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:21.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d7c079140 0x7f8d7c07b600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0c9b90 tx=0x7f8d9c033000 comp rx=0 tx=0).stop 2026-02-17T05:57:21.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 10 monclient: shutdown 2026-02-17T05:57:21.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:21.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 -- 10.20.193.16:0/3188641388 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0c7fe0 msgr2=0x7f8dac142a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:21.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0c7fe0 0x7f8dac142a60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400e890 tx=0x7f8d9400ed60 comp rx=0 tx=0).stop 2026-02-17T05:57:21.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 -- 10.20.193.16:0/3188641388 shutdown_connections 2026-02-17T05:57:21.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8d7c079140 0x7f8d7c07b600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:21.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8dac1536f0 0x7f8dac0cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:21.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8dac0c7fe0 0x7f8dac142a60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:21.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 --2- 10.20.193.16:0/3188641388 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8dac0700f0 0x7f8dac1f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:21.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 -- 10.20.193.16:0/3188641388 >> 10.20.193.16:0/3188641388 conn(0x7f8dac0cd770 msgr2=0x7f8dac151180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:21.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 -- 10.20.193.16:0/3188641388 shutdown_connections 2026-02-17T05:57:21.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 -- 10.20.193.16:0/3188641388 wait complete. 2026-02-17T05:57:21.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:21.150+0000 7f8db2af2640 1 librados: shutdown 2026-02-17T05:57:23.795 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:03:02.622 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:23.796 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:03:02.624 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:23.803 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:03:02.631 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:23.822 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:03:02.648 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:23.824 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:03:02.613 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:23.826 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:03:02.652 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:23.827 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:03:02.616 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:23.834 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:03:02.660 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:23.835 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:03:02.624 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:24.160 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:24.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.229+0000 7f6a115cc640 10 monclient: get_monmap_and_config 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.229+0000 7f6a115cc640 10 monclient: build_initial_monmap 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.229+0000 7f6a115cc640 10 monclient: monmap: 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:24.230765+0000 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:24.230765+0000 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:24.231 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:24.232 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:24.232 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:24.232 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:24.232 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:24.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 1 Processor -- start 2026-02-17T05:57:24.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 1 -- start start 2026-02-17T05:57:24.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 10 monclient: init 2026-02-17T05:57:24.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:24.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 10 monclient: _add_conns ranks=[2,0,4,1,3,5] 2026-02-17T05:57:24.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6a0c0700f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:57:24.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6a0c0c86f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:24.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f6a0c153e00 0x7f6a0c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 10 monclient(hunting): picked mon.noname-b con 0x7f6a0c153e00 addr v2:10.20.193.128:3300/0 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0c0c4820 con 0x7f6a0c0c86f0 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0c0c4b00 con 0x7f6a0c0700f0 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0c0c4de0 con 0x7f6a0c153e00 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.230+0000 7f6a115cc640 10 monclient(hunting): authenticate will time out at 1096.799725s 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.231+0000 7f6a0a7fc640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f6a0c0c86f0 0x7f6a0c1536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:53902/0 (socket says 10.20.193.16:53902) 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.231+0000 7f6a0a7fc640 1 -- 10.20.193.16:0/2507125450 learned_addr learned my addr 10.20.193.16:0/2507125450 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.231+0000 7f6a0b7fe640 1 --2- 10.20.193.16:0/2507125450 >> v2:10.20.193.128:3300/0 conn(0x7f6a0c153e00 0x7f6a0c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.231+0000 7f6a0b7fe640 10 monclient(hunting): get_auth_request con 0x7f6a0c153e00 auth_method 0 2026-02-17T05:57:24.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.231+0000 7f6a0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.231+0000 7f6a0b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.231+0000 7f6a0b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.231+0000 7f6a0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.231+0000 7f6a0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.231+0000 7f6a0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 1 -- 10.20.193.16:0/2507125450 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1257657279 0 0) 0x7f6a0c0c4820 con 0x7f6a0c0c86f0 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 10 monclient(hunting): my global_id is 4578 2026-02-17T05:57:24.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 1 -- 10.20.193.16:0/2507125450 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69f8003620 con 0x7f6a0c0c86f0 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 1 -- 10.20.193.16:0/2507125450 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2941446924 0 0) 0x7f6a0c0c4b00 con 0x7f6a0c0700f0 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 10 monclient(hunting): my global_id is 4706 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.232+0000 7f6a09ffb640 1 -- 10.20.193.16:0/2507125450 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a0c0c4820 con 0x7f6a0c0700f0 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.233+0000 7f6a0b7fe640 10 monclient(hunting): handle_auth_done global_id 4636 payload 274 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.233+0000 7f6a0b7fe640 1 -- 10.20.193.16:0/2507125450 >> v1:10.20.193.148:6789/0 conn(0x7f6a0c0700f0 legacy=0x7f6a0c0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.233+0000 7f6a0b7fe640 1 -- 10.20.193.16:0/2507125450 >> v1:10.20.193.16:6789/0 conn(0x7f6a0c0c86f0 legacy=0x7f6a0c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.233+0000 7f6a0b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.233+0000 7f6a0b7fe640 1 monclient: found mon.noname-b 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.233+0000 7f6a0b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:24.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.233+0000 7f6a0b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:57:24.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.233+0000 7f6a0b7fe640 1 -- 10.20.193.16:0/2507125450 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c0c50c0 con 0x7f6a0c153e00 2026-02-17T05:57:24.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.234+0000 7f6a0b7fe640 1 --2- 10.20.193.16:0/2507125450 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6a0c153e00 0x7f6a0c0704d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009d90 tx=0x7f69fc02f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b38c206c9b3c3ba server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:24.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 1 -- 10.20.193.16:0/2507125450 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc03c070 con 0x7f6a0c153e00 2026-02-17T05:57:24.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:24.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:57:24.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 10 monclient: dump: 2026-02-17T05:57:24.250 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:24.250 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:24.250 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:24.250 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:24.250 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:24.251 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:24.251 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:24.251 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:24.251 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:24.251 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:24.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:24.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:24.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:24.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 1 -- 10.20.193.16:0/2507125450 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69fc037440 con 0x7f6a0c153e00 2026-02-17T05:57:24.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a115cc640 5 monclient: authenticate success, global_id 4636 2026-02-17T05:57:24.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a115cc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:24.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:24.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 1 -- 10.20.193.16:0/2507125450 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc037810 con 0x7f6a0c153e00 2026-02-17T05:57:24.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a115cc640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:24.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a115cc640 4 set_mon_vals no callback set 2026-02-17T05:57:24.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:24.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:24.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.236+0000 7f6a09ffb640 10 monclient: dump: 2026-02-17T05:57:24.253 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:24.253 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:24.254 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:24.254 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:24.254 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:24.254 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:24.254 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:24.254 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:24.255 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:24.255 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:24.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 10 monclient: shutdown 2026-02-17T05:57:24.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:24.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 -- 10.20.193.16:0/2507125450 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6a0c153e00 msgr2=0x7f6a0c0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:24.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 --2- 10.20.193.16:0/2507125450 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6a0c153e00 0x7f6a0c0704d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009d90 tx=0x7f69fc02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:57:24.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 -- 10.20.193.16:0/2507125450 shutdown_connections 2026-02-17T05:57:24.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 --2- 10.20.193.16:0/2507125450 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6a0c153e00 0x7f6a0c0704d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:24.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 -- 10.20.193.16:0/2507125450 >> 10.20.193.16:0/2507125450 conn(0x7f6a0c0cd770 msgr2=0x7f6a0c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:24.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 -- 10.20.193.16:0/2507125450 shutdown_connections 2026-02-17T05:57:24.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 -- 10.20.193.16:0/2507125450 wait complete. 2026-02-17T05:57:24.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 10 monclient: build_initial_monmap 2026-02-17T05:57:24.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 10 monclient: monmap: 2026-02-17T05:57:24.257 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:24.257 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:24.257 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:24.257 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:24.257 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:24.257 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:24.258 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:24.258 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:24.258 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:24.258 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:24.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 librados: starting msgr at 2026-02-17T05:57:24.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 librados: starting objecter 2026-02-17T05:57:24.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 Processor -- start 2026-02-17T05:57:24.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 -- start start 2026-02-17T05:57:24.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 librados: setting wanted keys 2026-02-17T05:57:24.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 1 librados: calling monclient init 2026-02-17T05:57:24.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.237+0000 7f6a115cc640 10 monclient: init 2026-02-17T05:57:24.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:24.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:57:24.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6a0c0700f0 0x7f6a0c145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:24.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 10 monclient(hunting): picked mon.noname-c con 0x7f6a0c0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:24.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6a0c0c86f0 0x7f6a0c1e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:24.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 10 monclient(hunting): picked mon.noname-b con 0x7f6a0c0c86f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:24.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6a0c153e00 0x7f6a0c1e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:24.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 10 monclient(hunting): picked mon.noname-a con 0x7f6a0c153e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:24.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:24.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c074c00 con 0x7f6a0c153e00 2026-02-17T05:57:24.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:24.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c0c5680 con 0x7f6a0c0c86f0 2026-02-17T05:57:24.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:24.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c074d80 con 0x7f6a0c0700f0 2026-02-17T05:57:24.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:24.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a115cc640 10 monclient(hunting): authenticate will time out at 1096.807267s 2026-02-17T05:57:24.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0b7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6a0c153e00 0x7f6a0c1e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:24.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0a7fc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6a0c0c86f0 0x7f6a0c1e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:24.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0affd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6a0c0700f0 0x7f6a0c145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:24.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0b7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6a0c153e00 0x7f6a0c1e5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:58140/0 (socket says 10.20.193.16:58140) 2026-02-17T05:57:24.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0b7fe640 1 -- 10.20.193.16:0/3977447152 learned_addr learned my addr 10.20.193.16:0/3977447152 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:24.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0a7fc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6a0c0c86f0 0x7f6a0c1e54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:42564/0 (socket says 10.20.193.16:42564) 2026-02-17T05:57:24.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0b7fe640 10 monclient(hunting): get_auth_request con 0x7f6a0c153e00 auth_method 0 2026-02-17T05:57:24.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:24.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:24.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:24.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0a7fc640 10 monclient(hunting): get_auth_request con 0x7f6a0c0c86f0 auth_method 0 2026-02-17T05:57:24.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:24.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:24.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:24.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0affd640 10 monclient(hunting): get_auth_request con 0x7f6a0c0700f0 auth_method 0 2026-02-17T05:57:24.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:24.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:24.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:24.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:24.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:24.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:24.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:24.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:24.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.238+0000 7f6a0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:24.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.239+0000 7f6a0affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:24.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.239+0000 7f6a0affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:24.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.239+0000 7f6a0affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:24.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.240+0000 7f6a0b7fe640 10 monclient(hunting): handle_auth_done global_id 4581 payload 824 2026-02-17T05:57:24.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.240+0000 7f6a0b7fe640 1 -- 10.20.193.16:0/3977447152 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6a0c0700f0 msgr2=0x7f6a0c145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:24.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.240+0000 7f6a0b7fe640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6a0c0700f0 0x7f6a0c145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:24.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.240+0000 7f6a0b7fe640 1 -- 10.20.193.16:0/3977447152 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6a0c0c86f0 msgr2=0x7f6a0c1e54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:24.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.240+0000 7f6a0b7fe640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6a0c0c86f0 0x7f6a0c1e54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:24.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.240+0000 7f6a0b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:24.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.240+0000 7f6a0b7fe640 1 monclient: found mon.noname-a 2026-02-17T05:57:24.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.240+0000 7f6a0b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:24.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.240+0000 7f6a0b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:24.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.240+0000 7f6a0b7fe640 1 -- 10.20.193.16:0/3977447152 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c1f46a0 con 0x7f6a0c153e00 2026-02-17T05:57:24.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.241+0000 7f6a0b7fe640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6a0c153e00 0x7f6a0c1e5a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009d90 tx=0x7f69fc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:24.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.241+0000 7f69eb7fe640 10 client.?.objecter ms_handle_connect 0x7f6a0c153e00 2026-02-17T05:57:24.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.241+0000 7f69eb7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:24.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 1 -- 10.20.193.16:0/3977447152 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc047020 con 0x7f6a0c153e00 2026-02-17T05:57:24.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:24.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:24.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 10 monclient: dump: 2026-02-17T05:57:24.268 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:24.268 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:24.268 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:24.269 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:24.269 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:24.269 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:24.269 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:24.269 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:24.269 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:24.269 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:24.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:24.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 1 -- 10.20.193.16:0/3977447152 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69fc03d040 con 0x7f6a0c153e00 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 1 -- 10.20.193.16:0/3977447152 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc03c050 con 0x7f6a0c153e00 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f69eb7fe640 10 monclient: dump: 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:24.270 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f6a097fa640 4 set_mon_vals no callback set 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.242+0000 7f6a115cc640 5 monclient: authenticate success, global_id 4581 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 10 monclient: _renew_subs 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 1 -- 10.20.193.16:0/3977447152 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c1e5010 con 0x7f6a0c153e00 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 10 client.4581.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 10 monclient: _renew_subs 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 1 -- 10.20.193.16:0/3977447152 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c1f3e00 con 0x7f6a0c153e00 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 1 librados: init done 2026-02-17T05:57:24.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:24.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:24.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:24.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.243+0000 7f6a115cc640 1 -- 10.20.193.16:0/3977447152 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c146540 con 0x7f6a0c153e00 2026-02-17T05:57:24.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.352+0000 7f69eb7fe640 1 -- 10.20.193.16:0/3977447152 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f69fc054070 con 0x7f6a0c153e00 2026-02-17T05:57:24.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.353+0000 7f69eb7fe640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f69d80791c0 0x7f69d807b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:24.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.353+0000 7f6a0affd640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f69d80791c0 0x7f69d807b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:24.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.353+0000 7f6a0affd640 10 monclient: get_auth_request con 0x7f69d80791c0 auth_method 0 2026-02-17T05:57:24.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.353+0000 7f6a0affd640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f69d80791c0 0x7f69d807b680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a00003fd0 tx=0x7f6a00002a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:24.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.353+0000 7f69eb7fe640 10 client.4581.objecter ms_handle_connect 0x7f69d80791c0 2026-02-17T05:57:24.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.361+0000 7f69eb7fe640 1 -- 10.20.193.16:0/3977447152 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f69fc084030 con 0x7f6a0c153e00 2026-02-17T05:57:24.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.361+0000 7f69eb7fe640 10 client.4581.objecter ms_dispatch2 0x7f6a0c000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:24.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.361+0000 7f69eb7fe640 3 client.4581.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:24.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.361+0000 7f69eb7fe640 3 client.4581.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:24.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.361+0000 7f69eb7fe640 20 client.4581.objecter dump_active .. 0 homeless 2026-02-17T05:57:24.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.361+0000 7f69eb7fe640 1 -- 10.20.193.16:0/3977447152 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f69fc051020 con 0x7f6a0c153e00 2026-02-17T05:57:24.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.361+0000 7f69eb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:24.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.361+0000 7f69eb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:24.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.463+0000 7f6a115cc640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:24.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.463+0000 7f6a115cc640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:24.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.463+0000 7f6a115cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:24.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.463+0000 7f6a115cc640 1 -- 10.20.193.16:0/3977447152 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6a0c1573f0 con 0x7f6a0c153e00 2026-02-17T05:57:24.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.466+0000 7f69eb7fe640 1 -- 10.20.193.16:0/3977447152 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f69fc087ff0 con 0x7f6a0c153e00 2026-02-17T05:57:24.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.466+0000 7f69eb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:24.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.466+0000 7f69eb7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:24.466 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:24.466 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:24.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 10 librados: watch_flush enter 2026-02-17T05:57:24.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 10 librados: watch_flush exit 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 20 client.4581.objecter shutdown clearing up homeless session... 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 10 client.4581.objecter successfully canceled tick 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 -- 10.20.193.16:0/3977447152 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f69d80791c0 msgr2=0x7f69d807b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f69d80791c0 0x7f69d807b680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a00003fd0 tx=0x7f6a00002a40 comp rx=0 tx=0).stop 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 10 monclient: shutdown 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 -- 10.20.193.16:0/3977447152 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6a0c153e00 msgr2=0x7f6a0c1e5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6a0c153e00 0x7f6a0c1e5a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009d90 tx=0x7f69fc004b40 comp rx=0 tx=0).stop 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 -- 10.20.193.16:0/3977447152 shutdown_connections 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f69d80791c0 0x7f69d807b680 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6a0c0c86f0 0x7f6a0c1e54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6a0c153e00 0x7f6a0c1e5a30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 --2- 10.20.193.16:0/3977447152 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6a0c0700f0 0x7f6a0c145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 -- 10.20.193.16:0/3977447152 >> 10.20.193.16:0/3977447152 conn(0x7f6a0c0cd770 msgr2=0x7f6a0c14e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 -- 10.20.193.16:0/3977447152 shutdown_connections 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 -- 10.20.193.16:0/3977447152 wait complete. 2026-02-17T05:57:24.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:24.467+0000 7f6a115cc640 1 librados: shutdown 2026-02-17T05:57:27.477 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:27.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.546+0000 7f583927e640 10 monclient: get_monmap_and_config 2026-02-17T05:57:27.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.546+0000 7f583927e640 10 monclient: build_initial_monmap 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.546+0000 7f583927e640 10 monclient: monmap: 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:27.547384+0000 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:27.547384+0000 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:27.547 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:27.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.546+0000 7f583927e640 1 Processor -- start 2026-02-17T05:57:27.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.546+0000 7f583927e640 1 -- start start 2026-02-17T05:57:27.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.546+0000 7f583927e640 10 monclient: init 2026-02-17T05:57:27.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:27.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 10 monclient: _add_conns ranks=[0,2,4,3,1,5] 2026-02-17T05:57:27.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5834155e00 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:27.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f58341561e0 addr v1:10.20.193.148:6789/0 2026-02-17T05:57:27.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f583415fa10 0x7f58340ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 10 monclient(hunting): picked mon.noname-b con 0x7f583415fa10 addr v2:10.20.193.128:3300/0 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58340c96d0 con 0x7f5834155e00 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5834074860 con 0x7f58341561e0 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f58340c9940 con 0x7f583415fa10 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f583927e640 10 monclient(hunting): authenticate will time out at 1100.116507s 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f5832d76640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f5834155e00 0x7f5834074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:53914/0 (socket says 10.20.193.16:53914) 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.547+0000 7f5832d76640 1 -- 10.20.193.16:0/4038156470 learned_addr learned my addr 10.20.193.16:0/4038156470 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.548+0000 7f5833577640 1 --2- 10.20.193.16:0/4038156470 >> v2:10.20.193.128:3300/0 conn(0x7f583415fa10 0x7f58340ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.548+0000 7f5833577640 10 monclient(hunting): get_auth_request con 0x7f583415fa10 auth_method 0 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.548+0000 7f5833577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.548+0000 7f5833577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.548+0000 7f5833577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.548+0000 7f5833577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.548+0000 7f5833577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.548+0000 7f5833577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5831d74640 1 -- 10.20.193.16:0/4038156470 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2017479239 0 0) 0x7f58340c96d0 con 0x7f5834155e00 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5831d74640 10 monclient(hunting): _negotiate 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5831d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5831d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5831d74640 10 monclient(hunting): my global_id is 4584 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5831d74640 1 -- 10.20.193.16:0/4038156470 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5820003620 con 0x7f5834155e00 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5833577640 10 monclient(hunting): handle_auth_done global_id 4642 payload 274 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5833577640 1 -- 10.20.193.16:0/4038156470 >> v1:10.20.193.148:6789/0 conn(0x7f58341561e0 legacy=0x7f583415f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5833577640 1 -- 10.20.193.16:0/4038156470 >> v1:10.20.193.16:6789/0 conn(0x7f5834155e00 legacy=0x7f5834074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5833577640 10 monclient: _finish_hunting 0 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5833577640 1 monclient: found mon.noname-b 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5833577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5833577640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:57:27.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.549+0000 7f5833577640 1 -- 10.20.193.16:0/4038156470 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58340c9c20 con 0x7f583415fa10 2026-02-17T05:57:27.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.550+0000 7f5833577640 1 --2- 10.20.193.16:0/4038156470 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f583415fa10 0x7f58340ca700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5824009d90 tx=0x7f582402f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7052b9fa4d16ad5f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 1 -- 10.20.193.16:0/4038156470 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582403c070 con 0x7f583415fa10 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 10 monclient: dump: 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 10 monclient: _finish_auth 0 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 10 monclient: _check_auth_tickets 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:27.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 1 -- 10.20.193.16:0/4038156470 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5824037440 con 0x7f583415fa10 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 1 -- 10.20.193.16:0/4038156470 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824037810 con 0x7f583415fa10 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f5831d74640 10 monclient: dump: 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 5 monclient: authenticate success, global_id 4642 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 4 set_mon_vals no callback set 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 10 monclient: shutdown 2026-02-17T05:57:27.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 1 -- 10.20.193.16:0/4038156470 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f583415fa10 msgr2=0x7f58340ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 1 --2- 10.20.193.16:0/4038156470 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f583415fa10 0x7f58340ca700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5824009d90 tx=0x7f582402f1e0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 1 -- 10.20.193.16:0/4038156470 shutdown_connections 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 1 --2- 10.20.193.16:0/4038156470 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f583415fa10 0x7f58340ca700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 1 -- 10.20.193.16:0/4038156470 >> 10.20.193.16:0/4038156470 conn(0x7f58340ce6c0 msgr2=0x7f58340cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 1 -- 10.20.193.16:0/4038156470 shutdown_connections 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.552+0000 7f583927e640 1 -- 10.20.193.16:0/4038156470 wait complete. 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.553+0000 7f583927e640 10 monclient: build_initial_monmap 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.553+0000 7f583927e640 10 monclient: monmap: 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:27.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.553+0000 7f583927e640 1 librados: starting msgr at 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.553+0000 7f583927e640 1 librados: starting objecter 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.553+0000 7f583927e640 1 Processor -- start 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.553+0000 7f583927e640 1 -- start start 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.553+0000 7f583927e640 1 librados: setting wanted keys 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.553+0000 7f583927e640 1 librados: calling monclient init 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.553+0000 7f583927e640 10 monclient: init 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5834155e00 0x7f58341f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 10 monclient(hunting): picked mon.noname-c con 0x7f5834155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f58341561e0 0x7f58340c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 10 monclient(hunting): picked mon.noname-a con 0x7f58341561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f583415fa10 0x7f58340ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 10 monclient(hunting): picked mon.noname-b con 0x7f583415fa10 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58340c9360 con 0x7f58341561e0 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f58340ca1e0 con 0x7f583415fa10 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5834074dc0 con 0x7f5834155e00 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:27.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f583927e640 10 monclient(hunting): authenticate will time out at 1100.123142s 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f58341561e0 0x7f58340c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5833577640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f583415fa10 0x7f58340ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f58341561e0 0x7f58340c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:58156/0 (socket says 10.20.193.16:58156) 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832575640 1 -- 10.20.193.16:0/642249596 learned_addr learned my addr 10.20.193.16:0/642249596 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832575640 10 monclient(hunting): get_auth_request con 0x7f58341561e0 auth_method 0 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5833577640 10 monclient(hunting): get_auth_request con 0x7f583415fa10 auth_method 0 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5833577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5833577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5833577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832d76640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5834155e00 0x7f58341f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832d76640 10 monclient(hunting): get_auth_request con 0x7f5834155e00 auth_method 0 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5833577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5833577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5833577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.554+0000 7f5832d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.556+0000 7f5833577640 10 monclient(hunting): handle_auth_done global_id 4645 payload 824 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.556+0000 7f5833577640 1 -- 10.20.193.16:0/642249596 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5834155e00 msgr2=0x7f58341f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.556+0000 7f5833577640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5834155e00 0x7f58341f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.556+0000 7f5833577640 1 -- 10.20.193.16:0/642249596 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f58341561e0 msgr2=0x7f58340c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.556+0000 7f5833577640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f58341561e0 0x7f58340c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.556+0000 7f5833577640 10 monclient: _finish_hunting 0 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.556+0000 7f5833577640 1 monclient: found mon.noname-b 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.556+0000 7f5833577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.556+0000 7f5833577640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.556+0000 7f5833577640 1 -- 10.20.193.16:0/642249596 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58341f4950 con 0x7f583415fa10 2026-02-17T05:57:27.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.557+0000 7f5833577640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f583415fa10 0x7f58340ccd20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5824036000 tx=0x7f5824004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.557+0000 7f58137fe640 10 client.?.objecter ms_handle_connect 0x7f583415fa10 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.557+0000 7f58137fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 1 -- 10.20.193.16:0/642249596 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824047020 con 0x7f583415fa10 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 10 monclient: dump: 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 1 -- 10.20.193.16:0/642249596 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582403d040 con 0x7f583415fa10 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f583927e640 5 monclient: authenticate success, global_id 4645 2026-02-17T05:57:27.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f583927e640 10 monclient: _renew_subs 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f583927e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f583927e640 1 -- 10.20.193.16:0/642249596 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58340c58a0 con 0x7f583415fa10 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 1 -- 10.20.193.16:0/642249596 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582403c050 con 0x7f583415fa10 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f58137fe640 10 monclient: dump: 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:27.564 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f5831573640 4 set_mon_vals no callback set 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f583927e640 10 client.4645.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f583927e640 10 monclient: _renew_subs 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f583927e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f583927e640 1 -- 10.20.193.16:0/642249596 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341f40b0 con 0x7f583415fa10 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.558+0000 7f583927e640 1 librados: init done 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.559+0000 7f583927e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.559+0000 7f583927e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.559+0000 7f583927e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:27.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.559+0000 7f583927e640 1 -- 10.20.193.16:0/642249596 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58340caec0 con 0x7f583415fa10 2026-02-17T05:57:27.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.665+0000 7f58137fe640 1 -- 10.20.193.16:0/642249596 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f5824041a10 con 0x7f583415fa10 2026-02-17T05:57:27.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.665+0000 7f58137fe640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5800079190 0x7f580007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.666+0000 7f5832d76640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5800079190 0x7f580007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.666+0000 7f5832d76640 10 monclient: get_auth_request con 0x7f5800079190 auth_method 0 2026-02-17T05:57:27.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.666+0000 7f5832d76640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5800079190 0x7f580007b650 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5828005980 tx=0x7f5828007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.666+0000 7f58137fe640 10 client.4645.objecter ms_handle_connect 0x7f5800079190 2026-02-17T05:57:27.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.668+0000 7f58137fe640 1 -- 10.20.193.16:0/642249596 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f58240832d0 con 0x7f583415fa10 2026-02-17T05:57:27.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.668+0000 7f58137fe640 10 client.4645.objecter ms_dispatch2 0x7f5834000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:27.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.668+0000 7f58137fe640 3 client.4645.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:27.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.668+0000 7f58137fe640 3 client.4645.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:27.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.668+0000 7f58137fe640 20 client.4645.objecter dump_active .. 0 homeless 2026-02-17T05:57:27.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.674+0000 7f58137fe640 1 -- 10.20.193.16:0/642249596 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5824051080 con 0x7f583415fa10 2026-02-17T05:57:27.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.674+0000 7f58137fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:27.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.674+0000 7f58137fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:27.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.774+0000 7f583927e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.774+0000 7f583927e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.774+0000 7f583927e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.774+0000 7f583927e640 1 -- 10.20.193.16:0/642249596 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f583406dbb0 con 0x7f583415fa10 2026-02-17T05:57:27.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.776+0000 7f58137fe640 1 -- 10.20.193.16:0/642249596 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f5824087290 con 0x7f583415fa10 2026-02-17T05:57:27.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.776+0000 7f58137fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:27.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.776+0000 7f58137fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:27.777 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:27.777 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:27.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 10 librados: watch_flush enter 2026-02-17T05:57:27.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 10 librados: watch_flush exit 2026-02-17T05:57:27.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 20 client.4645.objecter shutdown clearing up homeless session... 2026-02-17T05:57:27.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 10 client.4645.objecter successfully canceled tick 2026-02-17T05:57:27.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 -- 10.20.193.16:0/642249596 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5800079190 msgr2=0x7f580007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5800079190 0x7f580007b650 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5828005980 tx=0x7f5828007dd0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 10 monclient: shutdown 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 -- 10.20.193.16:0/642249596 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f583415fa10 msgr2=0x7f58340ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f583415fa10 0x7f58340ccd20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5824036000 tx=0x7f5824004b40 comp rx=0 tx=0).stop 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 -- 10.20.193.16:0/642249596 shutdown_connections 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5800079190 0x7f580007b650 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f583415fa10 0x7f58340ccd20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f58341561e0 0x7f58340c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 --2- 10.20.193.16:0/642249596 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5834155e00 0x7f58341f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 -- 10.20.193.16:0/642249596 >> 10.20.193.16:0/642249596 conn(0x7f58340ce6c0 msgr2=0x7f5834154050 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 -- 10.20.193.16:0/642249596 shutdown_connections 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 -- 10.20.193.16:0/642249596 wait complete. 2026-02-17T05:57:27.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:27.778+0000 7f583927e640 1 librados: shutdown 2026-02-17T05:57:30.787 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:30.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.854+0000 7f561c122640 10 monclient: get_monmap_and_config 2026-02-17T05:57:30.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.854+0000 7f561c122640 10 monclient: build_initial_monmap 2026-02-17T05:57:30.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.854+0000 7f561c122640 10 monclient: monmap: 2026-02-17T05:57:30.855 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:30.855 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:30.856 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:30.855303+0000 2026-02-17T05:57:30.856 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:30.855303+0000 2026-02-17T05:57:30.856 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:30.856 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:30.856 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:30.856 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:30.856 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:30.856 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:30.856 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:30.856 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:30.857 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:30.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.854+0000 7f561c122640 1 Processor -- start 2026-02-17T05:57:30.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.854+0000 7f561c122640 1 -- start start 2026-02-17T05:57:30.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.854+0000 7f561c122640 10 monclient: init 2026-02-17T05:57:30.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:30.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 10 monclient: _add_conns ranks=[0,5,3,2,4,1] 2026-02-17T05:57:30.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f56140700f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:30.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f56140c86f0 0x7f56140704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:30.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 10 monclient(hunting): picked mon.noname-c con 0x7f56140c86f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:30.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f56141536f0 0x7f5614153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:30.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 10 monclient(hunting): picked mon.noname-a con 0x7f56141536f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:57:30.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56140c4820 con 0x7f56140700f0 2026-02-17T05:57:30.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:30.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f56140c4b00 con 0x7f56141536f0 2026-02-17T05:57:30.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:30.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f56140c4de0 con 0x7f56140c86f0 2026-02-17T05:57:30.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:30.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561c122640 10 monclient(hunting): authenticate will time out at 1103.424240s 2026-02-17T05:57:30.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561a698640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f56141536f0 0x7f5614153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:30.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561a698640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f56141536f0 0x7f5614153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:41808/0 (socket says 10.20.193.16:41808) 2026-02-17T05:57:30.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561a698640 1 -- 10.20.193.16:0/1271874568 learned_addr learned my addr 10.20.193.16:0/1271874568 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:30.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561a698640 10 monclient(hunting): get_auth_request con 0x7f56141536f0 auth_method 0 2026-02-17T05:57:30.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561a698640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:30.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561a698640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:30.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f561a698640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:30.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.855+0000 7f5619696640 1 --2- 10.20.193.16:0/1271874568 >> v2:10.20.193.148:3300/0 conn(0x7f56140c86f0 0x7f56140704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f5619696640 10 monclient(hunting): get_auth_request con 0x7f56140c86f0 auth_method 0 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f5619696640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f5619696640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f5619696640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f561a698640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f561a698640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f561a698640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f5619696640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f5619696640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f5619696640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:30.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f561a698640 10 monclient(hunting): handle_auth_done global_id 4590 payload 274 2026-02-17T05:57:30.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f561a698640 1 -- 10.20.193.16:0/1271874568 >> v2:10.20.193.148:3300/0 conn(0x7f56140c86f0 msgr2=0x7f56140704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:30.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f561a698640 1 --2- 10.20.193.16:0/1271874568 >> v2:10.20.193.148:3300/0 conn(0x7f56140c86f0 0x7f56140704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:30.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f561a698640 1 -- 10.20.193.16:0/1271874568 >> v1:10.20.193.16:6789/0 conn(0x7f56140700f0 legacy=0x7f56140c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:30.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f561a698640 10 monclient: _finish_hunting 0 2026-02-17T05:57:30.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.856+0000 7f561a698640 1 monclient: found mon.noname-a 2026-02-17T05:57:30.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.857+0000 7f561a698640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:30.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.857+0000 7f561a698640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:30.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.857+0000 7f561a698640 1 -- 10.20.193.16:0/1271874568 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56140c50c0 con 0x7f56141536f0 2026-02-17T05:57:30.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.857+0000 7f561a698640 1 --2- 10.20.193.16:0/1271874568 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56141536f0 0x7f5614153b20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56140c5b80 tx=0x7f561002f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cf0fd540f85382 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:30.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.859+0000 7f5618e95640 1 -- 10.20.193.16:0/1271874568 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561002fc50 con 0x7f56141536f0 2026-02-17T05:57:30.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:30.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:30.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 10 monclient: dump: 2026-02-17T05:57:30.872 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:30.872 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:30.872 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:30.872 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:30.872 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:30.872 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:30.873 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:30.873 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:30.873 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:30.873 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:30.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 10 monclient: _finish_auth 0 2026-02-17T05:57:30.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 10 monclient: _check_auth_tickets 2026-02-17T05:57:30.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:30.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 1 -- 10.20.193.16:0/1271874568 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561002fdf0 con 0x7f56141536f0 2026-02-17T05:57:30.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 5 monclient: authenticate success, global_id 4590 2026-02-17T05:57:30.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 1 -- 10.20.193.16:0/1271874568 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610037610 con 0x7f56141536f0 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f5618e95640 10 monclient: dump: 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:30.874 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:30.875 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:30.875 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:30.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:30.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 4 set_mon_vals no callback set 2026-02-17T05:57:30.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 10 monclient: shutdown 2026-02-17T05:57:30.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:30.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 1 -- 10.20.193.16:0/1271874568 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56141536f0 msgr2=0x7f5614153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:30.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 1 --2- 10.20.193.16:0/1271874568 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56141536f0 0x7f5614153b20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56140c5b80 tx=0x7f561002f230 comp rx=0 tx=0).stop 2026-02-17T05:57:30.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 1 -- 10.20.193.16:0/1271874568 shutdown_connections 2026-02-17T05:57:30.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 1 --2- 10.20.193.16:0/1271874568 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56141536f0 0x7f5614153b20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 1 --2- 10.20.193.16:0/1271874568 >> v2:10.20.193.148:3300/0 conn(0x7f56140c86f0 0x7f56140704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 1 -- 10.20.193.16:0/1271874568 >> 10.20.193.16:0/1271874568 conn(0x7f56140cd770 msgr2=0x7f56140cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 1 -- 10.20.193.16:0/1271874568 shutdown_connections 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 1 -- 10.20.193.16:0/1271874568 wait complete. 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 10 monclient: build_initial_monmap 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.860+0000 7f561c122640 10 monclient: monmap: 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:30.876 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 librados: starting msgr at 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 librados: starting objecter 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 Processor -- start 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 -- start start 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 librados: setting wanted keys 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 librados: calling monclient init 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient: init 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:30.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f56140700f0 0x7f561406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient(hunting): picked mon.noname-b con 0x7f56140700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56140c86f0 0x7f5614073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient(hunting): picked mon.noname-a con 0x7f56140c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f56141536f0 0x7f561406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient(hunting): picked mon.noname-c con 0x7f56141536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5614154060 con 0x7f56140c86f0 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f56140c5680 con 0x7f56140700f0 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:30.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5614074c00 con 0x7f56141536f0 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561c122640 10 monclient(hunting): authenticate will time out at 1103.430616s 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f5619696640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56140c86f0 0x7f5614073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f5619696640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56140c86f0 0x7f5614073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:41814/0 (socket says 10.20.193.16:41814) 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f5619696640 1 -- 10.20.193.16:0/1027555321 learned_addr learned my addr 10.20.193.16:0/1027555321 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f5619696640 10 monclient(hunting): get_auth_request con 0x7f56140c86f0 auth_method 0 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f5619696640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f5619696640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f5619696640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.861+0000 7f561a698640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f56141536f0 0x7f561406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f561a698640 10 monclient(hunting): get_auth_request con 0x7f56141536f0 auth_method 0 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f561a698640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:30.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f561a698640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f561a698640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619e97640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f56140700f0 0x7f561406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619696640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619696640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619696640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619e97640 10 monclient(hunting): get_auth_request con 0x7f56140700f0 auth_method 0 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619e97640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619e97640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619e97640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f561a698640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:30.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f561a698640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f561a698640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619e97640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619e97640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.862+0000 7f5619e97640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.863+0000 7f5619696640 10 monclient(hunting): handle_auth_done global_id 4593 payload 824 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.863+0000 7f5619696640 1 -- 10.20.193.16:0/1027555321 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f56141536f0 msgr2=0x7f561406d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.863+0000 7f5619696640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f56141536f0 0x7f561406d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.863+0000 7f5619696640 1 -- 10.20.193.16:0/1027555321 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f56140700f0 msgr2=0x7f561406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.863+0000 7f5619696640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f56140700f0 0x7f561406e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.863+0000 7f5619696640 10 monclient: _finish_hunting 0 2026-02-17T05:57:30.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.863+0000 7f5619696640 1 monclient: found mon.noname-a 2026-02-17T05:57:30.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.863+0000 7f5619696640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:30.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.863+0000 7f5619696640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:30.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.863+0000 7f5619696640 1 -- 10.20.193.16:0/1027555321 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561406e700 con 0x7f56140c86f0 2026-02-17T05:57:30.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.864+0000 7f5619696640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56140c86f0 0x7f5614073e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f560800ebe0 tx=0x7f560800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:30.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.864+0000 7f56027fc640 10 client.?.objecter ms_handle_connect 0x7f56140c86f0 2026-02-17T05:57:30.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.864+0000 7f56027fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:30.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 1 -- 10.20.193.16:0/1027555321 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608017070 con 0x7f56140c86f0 2026-02-17T05:57:30.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:30.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:30.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 10 monclient: dump: 2026-02-17T05:57:30.883 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:30.883 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:30.883 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:30.883 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:30.883 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:30.883 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:30.883 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:30.883 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:30.883 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:30.883 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:30.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 10 monclient: _finish_auth 0 2026-02-17T05:57:30.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 10 monclient: _check_auth_tickets 2026-02-17T05:57:30.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:30.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 1 -- 10.20.193.16:0/1027555321 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f560800cc40 con 0x7f56140c86f0 2026-02-17T05:57:30.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f561c122640 5 monclient: authenticate success, global_id 4593 2026-02-17T05:57:30.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f561c122640 10 monclient: _renew_subs 2026-02-17T05:57:30.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f561c122640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:30.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f561c122640 1 -- 10.20.193.16:0/1027555321 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561406e9e0 con 0x7f56140c86f0 2026-02-17T05:57:30.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:30.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 1 -- 10.20.193.16:0/1027555321 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608013470 con 0x7f56140c86f0 2026-02-17T05:57:30.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:30.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:30.885 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f56027fc640 10 monclient: dump: 2026-02-17T05:57:30.885 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:30.885 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:30.885 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:30.885 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:30.885 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:30.885 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f5603fff640 4 set_mon_vals no callback set 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f561c122640 10 client.4593.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f561c122640 10 monclient: _renew_subs 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f561c122640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f561c122640 1 -- 10.20.193.16:0/1027555321 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f561406efa0 con 0x7f56140c86f0 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.865+0000 7f561c122640 1 librados: init done 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.866+0000 7f561c122640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:30.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.866+0000 7f561c122640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:30.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.866+0000 7f561c122640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:30.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.866+0000 7f561c122640 1 -- 10.20.193.16:0/1027555321 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56140704d0 con 0x7f56140c86f0 2026-02-17T05:57:30.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.877+0000 7f56027fc640 1 -- 10.20.193.16:0/1027555321 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f5608013610 con 0x7f56140c86f0 2026-02-17T05:57:30.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.878+0000 7f56027fc640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55e4079140 0x7f55e407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:30.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.878+0000 7f5619e97640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55e4079140 0x7f55e407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:30.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.878+0000 7f5619e97640 10 monclient: get_auth_request con 0x7f55e4079140 auth_method 0 2026-02-17T05:57:30.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.878+0000 7f5619e97640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55e4079140 0x7f55e407b600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f56040046e0 tx=0x7f56040091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:30.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.878+0000 7f56027fc640 10 client.4593.objecter ms_handle_connect 0x7f55e4079140 2026-02-17T05:57:30.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.878+0000 7f56027fc640 1 -- 10.20.193.16:0/1027555321 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f560805e770 con 0x7f56140c86f0 2026-02-17T05:57:30.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.878+0000 7f56027fc640 10 client.4593.objecter ms_dispatch2 0x7f5614000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:30.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.878+0000 7f56027fc640 3 client.4593.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:30.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.878+0000 7f56027fc640 3 client.4593.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:30.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.879+0000 7f56027fc640 20 client.4593.objecter dump_active .. 0 homeless 2026-02-17T05:57:30.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.980+0000 7f56027fc640 1 -- 10.20.193.16:0/1027555321 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5608062730 con 0x7f56140c86f0 2026-02-17T05:57:30.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.980+0000 7f56027fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:30.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:30.980+0000 7f56027fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:31.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.081+0000 7f561c122640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:31.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.081+0000 7f561c122640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:31.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.081+0000 7f561c122640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:31.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.081+0000 7f561c122640 1 -- 10.20.193.16:0/1027555321 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f56141573f0 con 0x7f56140c86f0 2026-02-17T05:57:31.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.084+0000 7f56027fc640 1 -- 10.20.193.16:0/1027555321 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f5608066a30 con 0x7f56140c86f0 2026-02-17T05:57:31.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.084+0000 7f56027fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:31.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.084+0000 7f56027fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:31.086 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:31.086 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:31.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 10 librados: watch_flush enter 2026-02-17T05:57:31.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 10 librados: watch_flush exit 2026-02-17T05:57:31.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 20 client.4593.objecter shutdown clearing up homeless session... 2026-02-17T05:57:31.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 10 client.4593.objecter successfully canceled tick 2026-02-17T05:57:31.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 -- 10.20.193.16:0/1027555321 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55e4079140 msgr2=0x7f55e407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:31.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55e4079140 0x7f55e407b600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f56040046e0 tx=0x7f56040091c0 comp rx=0 tx=0).stop 2026-02-17T05:57:31.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 10 monclient: shutdown 2026-02-17T05:57:31.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:31.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 -- 10.20.193.16:0/1027555321 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56140c86f0 msgr2=0x7f5614073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:31.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56140c86f0 0x7f5614073e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f560800ebe0 tx=0x7f560800c520 comp rx=0 tx=0).stop 2026-02-17T05:57:31.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 -- 10.20.193.16:0/1027555321 shutdown_connections 2026-02-17T05:57:31.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55e4079140 0x7f55e407b600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:31.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f56141536f0 0x7f561406d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:31.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f56140c86f0 0x7f5614073e00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:31.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 --2- 10.20.193.16:0/1027555321 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f56140700f0 0x7f561406e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:31.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 -- 10.20.193.16:0/1027555321 >> 10.20.193.16:0/1027555321 conn(0x7f56140cd770 msgr2=0x7f561414fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:31.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 -- 10.20.193.16:0/1027555321 shutdown_connections 2026-02-17T05:57:31.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 -- 10.20.193.16:0/1027555321 wait complete. 2026-02-17T05:57:31.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:31.086+0000 7f561c122640 1 librados: shutdown 2026-02-17T05:57:34.096 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:34.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.166+0000 7fbd222ca640 10 monclient: get_monmap_and_config 2026-02-17T05:57:34.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.166+0000 7fbd222ca640 10 monclient: build_initial_monmap 2026-02-17T05:57:34.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.166+0000 7fbd222ca640 10 monclient: monmap: 2026-02-17T05:57:34.167 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:34.167 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:34.167 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:34.167115+0000 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:34.167115+0000 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:34.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.166+0000 7fbd222ca640 1 Processor -- start 2026-02-17T05:57:34.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.166+0000 7fbd222ca640 1 -- start start 2026-02-17T05:57:34.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.166+0000 7fbd222ca640 10 monclient: init 2026-02-17T05:57:34.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:34.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 10 monclient: _add_conns ranks=[4,3,0,5,1,2] 2026-02-17T05:57:34.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fbd1c0700f0 0x7fbd1c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:34.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 10 monclient(hunting): picked mon.noname-b con 0x7fbd1c0700f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:57:34.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fbd1c0c7fe0 0x7fbd1c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:34.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 10 monclient(hunting): picked mon.noname-a con 0x7fbd1c0c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T05:57:34.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbd1c1536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:34.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd1c0c8970 con 0x7fbd1c1536f0 2026-02-17T05:57:34.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:34.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbd1c0c4820 con 0x7fbd1c0c7fe0 2026-02-17T05:57:34.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:34.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fbd1c0c4b00 con 0x7fbd1c0700f0 2026-02-17T05:57:34.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:34.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd222ca640 10 monclient(hunting): authenticate will time out at 1106.736085s 2026-02-17T05:57:34.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd1b7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fbd1c0c7fe0 0x7fbd1c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:34.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd1b7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fbd1c0c7fe0 0x7fbd1c0c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:41818/0 (socket says 10.20.193.16:41818) 2026-02-17T05:57:34.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd1b7fe640 1 -- 10.20.193.16:0/49067724 learned_addr learned my addr 10.20.193.16:0/49067724 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd1b7fe640 10 monclient(hunting): get_auth_request con 0x7fbd1c0c7fe0 auth_method 0 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd1b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.167+0000 7fbd1b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.168+0000 7fbd1b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.168+0000 7fbd1b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.168+0000 7fbd1b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.169+0000 7fbd1b7fe640 10 monclient(hunting): handle_auth_done global_id 4596 payload 274 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.169+0000 7fbd1b7fe640 1 -- 10.20.193.16:0/49067724 >> v2:10.20.193.128:3300/0 conn(0x7fbd1c0700f0 msgr2=0x7fbd1c0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.169+0000 7fbd1b7fe640 1 --2- 10.20.193.16:0/49067724 >> v2:10.20.193.128:3300/0 conn(0x7fbd1c0700f0 0x7fbd1c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.169+0000 7fbd1b7fe640 1 -- 10.20.193.16:0/49067724 >> v1:10.20.193.16:6789/0 conn(0x7fbd1c1536f0 legacy=0x7fbd1c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:34.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.169+0000 7fbd1b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:34.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.169+0000 7fbd1b7fe640 1 monclient: found mon.noname-a 2026-02-17T05:57:34.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.169+0000 7fbd1b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:34.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.169+0000 7fbd1b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:34.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.169+0000 7fbd1b7fe640 1 -- 10.20.193.16:0/49067724 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c0c4de0 con 0x7fbd1c0c7fe0 2026-02-17T05:57:34.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.170+0000 7fbd1b7fe640 1 --2- 10.20.193.16:0/49067724 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0c7fe0 0x7fbd1c0c8430 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c070b90 tx=0x7fbd0c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb479d12c817f461 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:34.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 1 -- 10.20.193.16:0/49067724 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c03c070 con 0x7fbd1c0c7fe0 2026-02-17T05:57:34.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:34.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:34.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 10 monclient: dump: 2026-02-17T05:57:34.178 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:34.179 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:34.179 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:34.179 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:34.179 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:34.179 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:34.179 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:34.179 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:34.179 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 10 monclient: _finish_auth 0 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 10 monclient: _check_auth_tickets 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 1 -- 10.20.193.16:0/49067724 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd0c02fa90 con 0x7fbd1c0c7fe0 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 5 monclient: authenticate success, global_id 4596 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 1 -- 10.20.193.16:0/49067724 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c02fe60 con 0x7fbd1c0c7fe0 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 4 set_mon_vals no callback set 2026-02-17T05:57:34.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd1affd640 10 monclient: dump: 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 10 monclient: shutdown 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:34.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 1 -- 10.20.193.16:0/49067724 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0c7fe0 msgr2=0x7fbd1c0c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 1 --2- 10.20.193.16:0/49067724 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0c7fe0 0x7fbd1c0c8430 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c070b90 tx=0x7fbd0c02ee60 comp rx=0 tx=0).stop 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 1 -- 10.20.193.16:0/49067724 shutdown_connections 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 1 --2- 10.20.193.16:0/49067724 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0c7fe0 0x7fbd1c0c8430 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 1 --2- 10.20.193.16:0/49067724 >> v2:10.20.193.128:3300/0 conn(0x7fbd1c0700f0 0x7fbd1c0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 1 -- 10.20.193.16:0/49067724 >> 10.20.193.16:0/49067724 conn(0x7fbd1c0cd770 msgr2=0x7fbd1c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 1 -- 10.20.193.16:0/49067724 shutdown_connections 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.172+0000 7fbd222ca640 1 -- 10.20.193.16:0/49067724 wait complete. 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.173+0000 7fbd222ca640 10 monclient: build_initial_monmap 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.173+0000 7fbd222ca640 10 monclient: monmap: 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:34.182 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.173+0000 7fbd222ca640 1 librados: starting msgr at 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.173+0000 7fbd222ca640 1 librados: starting objecter 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.173+0000 7fbd222ca640 1 Processor -- start 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.173+0000 7fbd222ca640 1 -- start start 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.173+0000 7fbd222ca640 1 librados: setting wanted keys 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.173+0000 7fbd222ca640 1 librados: calling monclient init 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.173+0000 7fbd222ca640 10 monclient: init 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0700f0 0x7fbd1c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:34.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 10 monclient(hunting): picked mon.noname-a con 0x7fbd1c0700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbd1c0c7fe0 0x7fbd1c156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 10 monclient(hunting): picked mon.noname-c con 0x7fbd1c0c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbd1c1536f0 0x7fbd1c1f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 10 monclient(hunting): picked mon.noname-b con 0x7fbd1c1536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c0c5680 con 0x7fbd1c0700f0 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c0c53a0 con 0x7fbd1c1536f0 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c074c00 con 0x7fbd1c0c7fe0 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd222ca640 10 monclient(hunting): authenticate will time out at 1106.743126s 2026-02-17T05:57:34.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd1bfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0700f0 0x7fbd1c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd1bfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0700f0 0x7fbd1c1542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:41832/0 (socket says 10.20.193.16:41832) 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd1bfff640 1 -- 10.20.193.16:0/1244526432 learned_addr learned my addr 10.20.193.16:0/1244526432 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd1bfff640 10 monclient(hunting): get_auth_request con 0x7fbd1c0700f0 auth_method 0 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd1bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd1bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.174+0000 7fbd1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.176+0000 7fbd1bfff640 10 monclient(hunting): handle_auth_done global_id 4599 payload 824 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.176+0000 7fbd1bfff640 1 -- 10.20.193.16:0/1244526432 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbd1c0c7fe0 msgr2=0x7fbd1c156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:34.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.176+0000 7fbd1bfff640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbd1c0c7fe0 0x7fbd1c156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:34.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.176+0000 7fbd1bfff640 1 -- 10.20.193.16:0/1244526432 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbd1c1536f0 msgr2=0x7fbd1c1f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:34.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.176+0000 7fbd1bfff640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbd1c1536f0 0x7fbd1c1f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:34.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.176+0000 7fbd1bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:57:34.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.176+0000 7fbd1bfff640 1 monclient: found mon.noname-a 2026-02-17T05:57:34.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.176+0000 7fbd1bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:34.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.176+0000 7fbd1bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:34.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.176+0000 7fbd1bfff640 1 -- 10.20.193.16:0/1244526432 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c1570f0 con 0x7fbd1c0700f0 2026-02-17T05:57:34.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.177+0000 7fbd1bfff640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0700f0 0x7fbd1c1542c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800b530 tx=0x7fbd0800ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:34.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.177+0000 7fbd18ff9640 10 client.?.objecter ms_handle_connect 0x7fbd1c0700f0 2026-02-17T05:57:34.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.177+0000 7fbd18ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:34.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 1 -- 10.20.193.16:0/1244526432 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08013020 con 0x7fbd1c0700f0 2026-02-17T05:57:34.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:34.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:34.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 10 monclient: dump: 2026-02-17T05:57:34.189 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:34.189 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 10 monclient: _finish_auth 0 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:57:34.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 1 -- 10.20.193.16:0/1244526432 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd08004480 con 0x7fbd1c0700f0 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 5 monclient: authenticate success, global_id 4599 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 10 monclient: _renew_subs 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 1 -- 10.20.193.16:0/1244526432 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c1f3160 con 0x7fbd1c0700f0 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 1 -- 10.20.193.16:0/1244526432 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08002920 con 0x7fbd1c0700f0 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd18ff9640 10 monclient: dump: 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:34.191 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd1a7fc640 4 set_mon_vals no callback set 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 10 client.4599.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 10 monclient: _renew_subs 2026-02-17T05:57:34.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:34.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 1 -- 10.20.193.16:0/1244526432 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c1f3720 con 0x7fbd1c0700f0 2026-02-17T05:57:34.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 1 librados: init done 2026-02-17T05:57:34.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:34.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:34.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:34.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.179+0000 7fbd222ca640 1 -- 10.20.193.16:0/1244526432 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1c06f500 con 0x7fbd1c0700f0 2026-02-17T05:57:34.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.289+0000 7fbd18ff9640 1 -- 10.20.193.16:0/1244526432 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fbd08002ac0 con 0x7fbd1c0700f0 2026-02-17T05:57:34.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.289+0000 7fbd18ff9640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbcec079140 0x7fbcec07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:34.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.289+0000 7fbd1b7fe640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbcec079140 0x7fbcec07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:34.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.289+0000 7fbd1b7fe640 10 monclient: get_auth_request con 0x7fbcec079140 auth_method 0 2026-02-17T05:57:34.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.289+0000 7fbd1b7fe640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbcec079140 0x7fbcec07b600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0c9b90 tx=0x7fbd0c031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:34.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.289+0000 7fbd18ff9640 10 client.4599.objecter ms_handle_connect 0x7fbcec079140 2026-02-17T05:57:34.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.297+0000 7fbd18ff9640 1 -- 10.20.193.16:0/1244526432 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fbd0805e200 con 0x7fbd1c0700f0 2026-02-17T05:57:34.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.297+0000 7fbd18ff9640 10 client.4599.objecter ms_dispatch2 0x7fbd1c000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:34.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.297+0000 7fbd18ff9640 3 client.4599.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:34.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.297+0000 7fbd18ff9640 3 client.4599.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:34.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.297+0000 7fbd18ff9640 20 client.4599.objecter dump_active .. 0 homeless 2026-02-17T05:57:34.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.297+0000 7fbd18ff9640 1 -- 10.20.193.16:0/1244526432 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbd1c06f500 con 0x7fbd1c0700f0 2026-02-17T05:57:34.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.297+0000 7fbd18ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:34.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.297+0000 7fbd18ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:34.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.394+0000 7fbd222ca640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:34.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.394+0000 7fbd222ca640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:34.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.394+0000 7fbd222ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:34.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.394+0000 7fbd222ca640 1 -- 10.20.193.16:0/1244526432 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbd1c050a10 con 0x7fbd1c0700f0 2026-02-17T05:57:34.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.398+0000 7fbd18ff9640 1 -- 10.20.193.16:0/1244526432 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fbd1c050a10 con 0x7fbd1c0700f0 2026-02-17T05:57:34.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.398+0000 7fbd18ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:34.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.398+0000 7fbd18ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:34.398 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:34.398 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:34.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 10 librados: watch_flush enter 2026-02-17T05:57:34.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 10 librados: watch_flush exit 2026-02-17T05:57:34.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 20 client.4599.objecter shutdown clearing up homeless session... 2026-02-17T05:57:34.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 10 client.4599.objecter successfully canceled tick 2026-02-17T05:57:34.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 -- 10.20.193.16:0/1244526432 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbcec079140 msgr2=0x7fbcec07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:34.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbcec079140 0x7fbcec07b600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0c9b90 tx=0x7fbd0c031000 comp rx=0 tx=0).stop 2026-02-17T05:57:34.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 10 monclient: shutdown 2026-02-17T05:57:34.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:34.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 -- 10.20.193.16:0/1244526432 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0700f0 msgr2=0x7fbd1c1542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:34.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0700f0 0x7fbd1c1542c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800b530 tx=0x7fbd0800ba00 comp rx=0 tx=0).stop 2026-02-17T05:57:34.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 -- 10.20.193.16:0/1244526432 shutdown_connections 2026-02-17T05:57:34.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbcec079140 0x7fbcec07b600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:34.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbd1c1536f0 0x7fbd1c1f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:34.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbd1c0c7fe0 0x7fbd1c156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:34.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 --2- 10.20.193.16:0/1244526432 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd1c0700f0 0x7fbd1c1542c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:34.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 -- 10.20.193.16:0/1244526432 >> 10.20.193.16:0/1244526432 conn(0x7fbd1c0cd770 msgr2=0x7fbd1c151ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:34.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 -- 10.20.193.16:0/1244526432 shutdown_connections 2026-02-17T05:57:34.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 -- 10.20.193.16:0/1244526432 wait complete. 2026-02-17T05:57:34.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:34.399+0000 7fbd222ca640 1 librados: shutdown 2026-02-17T05:57:37.408 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:37.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.480+0000 7fc0eced3640 10 monclient: get_monmap_and_config 2026-02-17T05:57:37.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.480+0000 7fc0eced3640 10 monclient: build_initial_monmap 2026-02-17T05:57:37.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.480+0000 7fc0eced3640 10 monclient: monmap: 2026-02-17T05:57:37.484 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:37.484 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:37.484 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:37.481416+0000 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:37.481416+0000 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:37.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.480+0000 7fc0eced3640 1 Processor -- start 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.480+0000 7fc0eced3640 1 -- start start 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.480+0000 7fc0eced3640 10 monclient: init 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 10 monclient: _add_conns ranks=[3,4,5,0,1,2] 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fc0e80700f0 0x7fc0e80704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 10 monclient(hunting): picked mon.noname-a con 0x7fc0e80700f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fc0e80c7fe0 0x7fc0e80c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 10 monclient(hunting): picked mon.noname-b con 0x7fc0e80c7fe0 addr v2:10.20.193.128:3300/0 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fc0e81536f0 0x7fc0e8153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 10 monclient(hunting): picked mon.noname-c con 0x7fc0e81536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:37.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e80c8970 con 0x7fc0e80700f0 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e80c4820 con 0x7fc0e80c7fe0 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e80c4b00 con 0x7fc0e81536f0 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0eced3640 10 monclient(hunting): authenticate will time out at 1110.050445s 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0e6575640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fc0e80700f0 0x7fc0e80704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0e5d74640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fc0e80c7fe0 0x7fc0e80c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.481+0000 7fc0e6d76640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fc0e81536f0 0x7fc0e8153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6575640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fc0e80700f0 0x7fc0e80704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:41846/0 (socket says 10.20.193.16:41846) 2026-02-17T05:57:37.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6575640 1 -- 10.20.193.16:0/791113539 learned_addr learned my addr 10.20.193.16:0/791113539 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6575640 10 monclient(hunting): get_auth_request con 0x7fc0e80700f0 auth_method 0 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e5d74640 10 monclient(hunting): get_auth_request con 0x7fc0e80c7fe0 auth_method 0 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6d76640 10 monclient(hunting): get_auth_request con 0x7fc0e81536f0 auth_method 0 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:37.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.482+0000 7fc0e6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6d76640 10 monclient(hunting): handle_auth_done global_id 4724 payload 274 2026-02-17T05:57:37.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6d76640 1 -- 10.20.193.16:0/791113539 >> v2:10.20.193.128:3300/0 conn(0x7fc0e80c7fe0 msgr2=0x7fc0e80c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:37.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6d76640 1 --2- 10.20.193.16:0/791113539 >> v2:10.20.193.128:3300/0 conn(0x7fc0e80c7fe0 0x7fc0e80c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6d76640 1 -- 10.20.193.16:0/791113539 >> v2:10.20.193.16:3300/0 conn(0x7fc0e80700f0 msgr2=0x7fc0e80704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:37.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6d76640 1 --2- 10.20.193.16:0/791113539 >> v2:10.20.193.16:3300/0 conn(0x7fc0e80700f0 0x7fc0e80704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6d76640 10 monclient: _finish_hunting 0 2026-02-17T05:57:37.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6d76640 1 monclient: found mon.noname-c 2026-02-17T05:57:37.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:37.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6d76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:37.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6d76640 1 -- 10.20.193.16:0/791113539 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e80c4de0 con 0x7fc0e81536f0 2026-02-17T05:57:37.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e5d74640 1 --2- 10.20.193.16:0/791113539 >> v2:10.20.193.128:3300/0 conn(0x7fc0e80c7fe0 0x7fc0e80c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:57:37.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.483+0000 7fc0e6575640 1 --2- 10.20.193.16:0/791113539 >> v2:10.20.193.16:3300/0 conn(0x7fc0e80700f0 0x7fc0e80704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:57:37.497 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.484+0000 7fc0e6d76640 1 --2- 10.20.193.16:0/791113539 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0e81536f0 0x7fc0e8153b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80c5b80 tx=0x7fc0d802f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e1e58603bff4df1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:37.498 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 1 -- 10.20.193.16:0/791113539 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d803c070 con 0x7fc0e81536f0 2026-02-17T05:57:37.498 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:37.498 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:37.498 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 10 monclient: dump: 2026-02-17T05:57:37.498 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:37.498 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:37.498 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:37.498 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:37.498 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:37.498 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 10 monclient: _finish_auth 0 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 10 monclient: _check_auth_tickets 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 1 -- 10.20.193.16:0/791113539 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d802fdb0 con 0x7fc0e81536f0 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 1 -- 10.20.193.16:0/791113539 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8037660 con 0x7fc0e81536f0 2026-02-17T05:57:37.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 5 monclient: authenticate success, global_id 4724 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 4 set_mon_vals no callback set 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0e5573640 10 monclient: dump: 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:37.500 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 10 monclient: shutdown 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 1 -- 10.20.193.16:0/791113539 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0e81536f0 msgr2=0x7fc0e8153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 1 --2- 10.20.193.16:0/791113539 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0e81536f0 0x7fc0e8153b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80c5b80 tx=0x7fc0d802f130 comp rx=0 tx=0).stop 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 1 -- 10.20.193.16:0/791113539 shutdown_connections 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 1 --2- 10.20.193.16:0/791113539 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0e81536f0 0x7fc0e8153b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 1 --2- 10.20.193.16:0/791113539 >> v2:10.20.193.128:3300/0 conn(0x7fc0e80c7fe0 0x7fc0e80c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 1 --2- 10.20.193.16:0/791113539 >> v2:10.20.193.16:3300/0 conn(0x7fc0e80700f0 0x7fc0e80704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 1 -- 10.20.193.16:0/791113539 >> 10.20.193.16:0/791113539 conn(0x7fc0e80cd770 msgr2=0x7fc0e80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 1 -- 10.20.193.16:0/791113539 shutdown_connections 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.486+0000 7fc0eced3640 1 -- 10.20.193.16:0/791113539 wait complete. 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient: build_initial_monmap 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient: monmap: 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:37.502 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 librados: starting msgr at 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 librados: starting objecter 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 Processor -- start 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 -- start start 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 librados: setting wanted keys 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 librados: calling monclient init 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient: init 2026-02-17T05:57:37.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0e80700f0 0x7fc0e8148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient(hunting): picked mon.noname-c con 0x7fc0e80700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc0e80c7fe0 0x7fc0e8073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient(hunting): picked mon.noname-a con 0x7fc0e80c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0e81536f0 0x7fc0e81f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient(hunting): picked mon.noname-b con 0x7fc0e81536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e80c5680 con 0x7fc0e80c7fe0 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e80c53a0 con 0x7fc0e81536f0 2026-02-17T05:57:37.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8154040 con 0x7fc0e80700f0 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.487+0000 7fc0eced3640 10 monclient(hunting): authenticate will time out at 1110.056962s 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e5d74640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc0e80c7fe0 0x7fc0e8073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6575640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0e80700f0 0x7fc0e8148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6d76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0e81536f0 0x7fc0e81f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e5d74640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc0e80c7fe0 0x7fc0e8073f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:41850/0 (socket says 10.20.193.16:41850) 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e5d74640 1 -- 10.20.193.16:0/1516292678 learned_addr learned my addr 10.20.193.16:0/1516292678 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e5d74640 10 monclient(hunting): get_auth_request con 0x7fc0e80c7fe0 auth_method 0 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:37.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6575640 10 monclient(hunting): get_auth_request con 0x7fc0e80700f0 auth_method 0 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6d76640 10 monclient(hunting): get_auth_request con 0x7fc0e81536f0 auth_method 0 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:37.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.488+0000 7fc0e6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.490+0000 7fc0e6d76640 10 monclient(hunting): handle_auth_done global_id 4654 payload 824 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.490+0000 7fc0e6d76640 1 -- 10.20.193.16:0/1516292678 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0e80700f0 msgr2=0x7fc0e8148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.490+0000 7fc0e6d76640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0e80700f0 0x7fc0e8148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.490+0000 7fc0e6d76640 1 -- 10.20.193.16:0/1516292678 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc0e80c7fe0 msgr2=0x7fc0e8073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:37.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.490+0000 7fc0e6d76640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc0e80c7fe0 0x7fc0e8073f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.490+0000 7fc0e6d76640 10 monclient: _finish_hunting 0 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.490+0000 7fc0e6d76640 1 monclient: found mon.noname-b 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.490+0000 7fc0e6d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.490+0000 7fc0e6d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.490+0000 7fc0e6d76640 1 -- 10.20.193.16:0/1516292678 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e8072750 con 0x7fc0e81536f0 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.491+0000 7fc0e6d76640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0e81536f0 0x7fc0e81f5fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8036000 tx=0x7fc0d8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.491+0000 7fc0ceffd640 10 client.?.objecter ms_handle_connect 0x7fc0e81536f0 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.491+0000 7fc0ceffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.492+0000 7fc0ceffd640 1 -- 10.20.193.16:0/1516292678 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8047020 con 0x7fc0e81536f0 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.492+0000 7fc0ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:37.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 10 monclient: dump: 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:37.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 10 monclient: _finish_auth 0 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 10 monclient: _check_auth_tickets 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 5 monclient: authenticate success, global_id 4654 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 1 -- 10.20.193.16:0/1516292678 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d8042470 con 0x7fc0e81536f0 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 10 monclient: _renew_subs 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 1 -- 10.20.193.16:0/1516292678 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e81f4c60 con 0x7fc0e81536f0 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 1 -- 10.20.193.16:0/1516292678 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d803c040 con 0x7fc0e81536f0 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0ceffd640 10 monclient: dump: 2026-02-17T05:57:37.510 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0e4d72640 4 set_mon_vals no callback set 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 10 client.4654.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 10 monclient: _renew_subs 2026-02-17T05:57:37.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:37.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 1 -- 10.20.193.16:0/1516292678 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e81f5220 con 0x7fc0e81536f0 2026-02-17T05:57:37.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 1 librados: init done 2026-02-17T05:57:37.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:37.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:37.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:37.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.493+0000 7fc0eced3640 1 -- 10.20.193.16:0/1516292678 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e8074c00 con 0x7fc0e81536f0 2026-02-17T05:57:37.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.504+0000 7fc0ceffd640 1 -- 10.20.193.16:0/1516292678 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fc0d8042860 con 0x7fc0e81536f0 2026-02-17T05:57:37.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.505+0000 7fc0ceffd640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc0b40791c0 0x7fc0b407b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:37.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.505+0000 7fc0e6575640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc0b40791c0 0x7fc0b407b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:37.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.505+0000 7fc0e6575640 10 monclient: get_auth_request con 0x7fc0b40791c0 auth_method 0 2026-02-17T05:57:37.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.505+0000 7fc0ceffd640 10 client.4654.objecter ms_handle_connect 0x7fc0b40791c0 2026-02-17T05:57:37.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.505+0000 7fc0e6575640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc0b40791c0 0x7fc0b407b680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc006da0 tx=0x7fc0dc009340 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:37.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.506+0000 7fc0ceffd640 1 -- 10.20.193.16:0/1516292678 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fc0d8083bc0 con 0x7fc0e81536f0 2026-02-17T05:57:37.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.506+0000 7fc0ceffd640 10 client.4654.objecter ms_dispatch2 0x7fc0e8000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:37.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.506+0000 7fc0ceffd640 3 client.4654.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:37.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.506+0000 7fc0ceffd640 3 client.4654.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:37.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.506+0000 7fc0ceffd640 20 client.4654.objecter dump_active .. 0 homeless 2026-02-17T05:57:37.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.608+0000 7fc0ceffd640 1 -- 10.20.193.16:0/1516292678 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc0d8051030 con 0x7fc0e81536f0 2026-02-17T05:57:37.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.609+0000 7fc0ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:37.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.609+0000 7fc0ceffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:37.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.709+0000 7fc0eced3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:37.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.709+0000 7fc0eced3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:37.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.709+0000 7fc0eced3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:37.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.709+0000 7fc0eced3640 1 -- 10.20.193.16:0/1516292678 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc0e80c8750 con 0x7fc0e81536f0 2026-02-17T05:57:37.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.711+0000 7fc0ceffd640 1 -- 10.20.193.16:0/1516292678 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc0e8072750 con 0x7fc0e81536f0 2026-02-17T05:57:37.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.711+0000 7fc0ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:37.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.711+0000 7fc0ceffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:37.712 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:37.712 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:37.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.712+0000 7fc0eced3640 10 librados: watch_flush enter 2026-02-17T05:57:37.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.712+0000 7fc0eced3640 10 librados: watch_flush exit 2026-02-17T05:57:37.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.712+0000 7fc0eced3640 20 client.4654.objecter shutdown clearing up homeless session... 2026-02-17T05:57:37.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.712+0000 7fc0eced3640 10 client.4654.objecter successfully canceled tick 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.712+0000 7fc0eced3640 1 -- 10.20.193.16:0/1516292678 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc0b40791c0 msgr2=0x7fc0b407b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.712+0000 7fc0eced3640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc0b40791c0 0x7fc0b407b680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc006da0 tx=0x7fc0dc009340 comp rx=0 tx=0).stop 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.712+0000 7fc0eced3640 10 monclient: shutdown 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.712+0000 7fc0eced3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.712+0000 7fc0eced3640 1 -- 10.20.193.16:0/1516292678 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0e81536f0 msgr2=0x7fc0e81f5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.712+0000 7fc0eced3640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0e81536f0 0x7fc0e81f5fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8036000 tx=0x7fc0d8004b40 comp rx=0 tx=0).stop 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.713+0000 7fc0eced3640 1 -- 10.20.193.16:0/1516292678 shutdown_connections 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.713+0000 7fc0eced3640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc0b40791c0 0x7fc0b407b680 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.713+0000 7fc0eced3640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc0e81536f0 0x7fc0e81f5fe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.713+0000 7fc0eced3640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc0e80c7fe0 0x7fc0e8073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.713+0000 7fc0eced3640 1 --2- 10.20.193.16:0/1516292678 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc0e80700f0 0x7fc0e8148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:37.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.713+0000 7fc0eced3640 1 -- 10.20.193.16:0/1516292678 >> 10.20.193.16:0/1516292678 conn(0x7fc0e80cd770 msgr2=0x7fc0e814e760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:37.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.713+0000 7fc0eced3640 1 -- 10.20.193.16:0/1516292678 shutdown_connections 2026-02-17T05:57:37.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.713+0000 7fc0eced3640 1 -- 10.20.193.16:0/1516292678 wait complete. 2026-02-17T05:57:37.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:37.713+0000 7fc0eced3640 1 librados: shutdown 2026-02-17T05:57:40.722 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:40.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.791+0000 7fde6dabf640 10 monclient: get_monmap_and_config 2026-02-17T05:57:40.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.791+0000 7fde6dabf640 10 monclient: build_initial_monmap 2026-02-17T05:57:40.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.791+0000 7fde6dabf640 10 monclient: monmap: 2026-02-17T05:57:40.792 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:40.792 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:40.792 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:40.792167+0000 2026-02-17T05:57:40.792 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:40.792167+0000 2026-02-17T05:57:40.792 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:40.792 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:40.792 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:40.793 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:40.793 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:40.793 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:40.793 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:40.793 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:40.793 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:40.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.791+0000 7fde6dabf640 1 Processor -- start 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.791+0000 7fde6dabf640 1 -- start start 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.791+0000 7fde6dabf640 10 monclient: init 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde6dabf640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde6dabf640 10 monclient: _add_conns ranks=[1,2,0,5,4,3] 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde6dabf640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fde680700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde6dabf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fde680c86f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde6dabf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fde68153e00 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde6dabf640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde680c4820 con 0x7fde68153e00 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde6dabf640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde680c4b00 con 0x7fde680700f0 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde6dabf640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde680c4de0 con 0x7fde680c86f0 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde6dabf640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde6dabf640 10 monclient(hunting): authenticate will time out at 1113.361237s 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde67fff640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fde68153e00 0x7fde68164350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:34296/0 (socket says 10.20.193.16:34296) 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.792+0000 7fde67fff640 1 -- 10.20.193.16:0/2958856837 learned_addr learned my addr 10.20.193.16:0/2958856837 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2144847626 0 0) 0x7fde680c4820 con 0x7fde68153e00 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): _negotiate 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): my global_id is 4608 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde4c003620 con 0x7fde68153e00 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 449612701 0 0) 0x7fde680c4b00 con 0x7fde680700f0 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): _negotiate 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:40.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): my global_id is 4657 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde680c4820 con 0x7fde680700f0 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1276855698 0 0) 0x7fde680c4de0 con 0x7fde680c86f0 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): _negotiate 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 10 monclient(hunting): my global_id is 4730 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.793+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde680c4b00 con 0x7fde680c86f0 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 <== mon.1 v1:10.20.193.128:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3852224710 0 0) 0x7fde680c4820 con 0x7fde680700f0 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 >> v1:10.20.193.148:6789/0 conn(0x7fde680c86f0 legacy=0x7fde681536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 >> v1:10.20.193.16:6789/0 conn(0x7fde68153e00 legacy=0x7fde68164350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient: _finish_hunting 0 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 monclient: found mon.noname-b-legacy 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.128:6789/0 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 --> v1:10.20.193.128:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde680c50c0 con 0x7fde680700f0 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient: _finish_auth 0 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient: _check_auth_tickets 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 <== mon.1 v1:10.20.193.128:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7fde54002ee0 con 0x7fde680700f0 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde6dabf640 5 monclient: authenticate success, global_id 4657 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient: dump: 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:40.800 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] but i'm connected to v1:10.20.193.128:6789/0, reconnecting 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 >> v1:10.20.193.128:6789/0 conn(0x7fde680700f0 legacy=0x7fde680c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde4c0053d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient(hunting): picked mon.a con 0x7fde4c004ff0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fde4c003a70 0x7fde4c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient(hunting): picked mon.b con 0x7fde4c003a70 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fde4c009ab0 0x7fde4c00bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient(hunting): picked mon.c con 0x7fde4c009ab0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c00c570 con 0x7fde4c004ff0 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c00c720 con 0x7fde4c003a70 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c00c920 con 0x7fde4c009ab0 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:40.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde667fc640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.795+0000 7fde6dabf640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde67fff640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fde4c009ab0 0x7fde4c00bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde677fe640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde4c0053d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde677fe640 10 monclient(hunting): get_auth_request con 0x7fde4c004ff0 auth_method 0 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde677fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde677fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde67fff640 10 monclient(hunting): get_auth_request con 0x7fde4c009ab0 auth_method 0 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde67fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde67fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde677fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.796+0000 7fde66ffd640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fde4c003a70 0x7fde4c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde67fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde66ffd640 10 monclient(hunting): get_auth_request con 0x7fde4c003a70 auth_method 0 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde66ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde66ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde66ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 10 monclient(hunting): handle_auth_done global_id 4657 payload 293 2026-02-17T05:57:40.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 1 -- 10.20.193.16:0/2958856837 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fde4c009ab0 msgr2=0x7fde4c00bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fde4c009ab0 0x7fde4c00bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 1 -- 10.20.193.16:0/2958856837 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fde4c003a70 msgr2=0x7fde4c003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fde4c003a70 0x7fde4c003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 1 monclient: found mon.a 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde67fff640 1 -- 10.20.193.16:0/2958856837 reap_dead start 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 1 -- 10.20.193.16:0/2958856837 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde680c4820 con 0x7fde4c004ff0 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.797+0000 7fde677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:40.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.798+0000 7fde677fe640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde4c0053d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fde54007fb0 tx=0x7fde5402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51ff18593d5d7a8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:40.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5403c070 con 0x7fde4c004ff0 2026-02-17T05:57:40.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde667fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:40.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde667fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:40.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde667fc640 10 monclient: dump: 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde5402fe00 con 0x7fde4c004ff0 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde667fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde667fc640 1 -- 10.20.193.16:0/2958856837 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54007bd0 con 0x7fde4c004ff0 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde667fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde667fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde667fc640 10 monclient: dump: 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:40.806 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 4 set_mon_vals no callback set 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 10 monclient: shutdown 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 1 -- 10.20.193.16:0/2958856837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 msgr2=0x7fde4c0053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde4c0053d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fde54007fb0 tx=0x7fde5402f3a0 comp rx=0 tx=0).stop 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 1 -- 10.20.193.16:0/2958856837 shutdown_connections 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 1 --2- 10.20.193.16:0/2958856837 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde4c0053d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 1 -- 10.20.193.16:0/2958856837 >> 10.20.193.16:0/2958856837 conn(0x7fde680cd770 msgr2=0x7fde680cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 1 -- 10.20.193.16:0/2958856837 shutdown_connections 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.800+0000 7fde6dabf640 1 -- 10.20.193.16:0/2958856837 wait complete. 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.801+0000 7fde6dabf640 10 monclient: build_initial_monmap 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.801+0000 7fde6dabf640 10 monclient: monmap: 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.801+0000 7fde6dabf640 1 librados: starting msgr at 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.801+0000 7fde6dabf640 1 librados: starting objecter 2026-02-17T05:57:40.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.801+0000 7fde6dabf640 1 Processor -- start 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.801+0000 7fde6dabf640 1 -- start start 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.801+0000 7fde6dabf640 1 librados: setting wanted keys 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.801+0000 7fde6dabf640 1 librados: calling monclient init 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.801+0000 7fde6dabf640 10 monclient: init 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde680743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 10 monclient(hunting): picked mon.noname-a con 0x7fde4c004ff0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fde681518d0 0x7fde681f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 10 monclient(hunting): picked mon.noname-b con 0x7fde681518d0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fde681f5200 0x7fde681f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 10 monclient(hunting): picked mon.noname-c con 0x7fde681f5200 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde68074c00 con 0x7fde4c004ff0 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fde680c5680 con 0x7fde681518d0 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde68074d80 con 0x7fde681f5200 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde6dabf640 10 monclient(hunting): authenticate will time out at 1113.371206s 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde677fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde680743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde677fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde680743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:42128/0 (socket says 10.20.193.16:42128) 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde677fe640 1 -- 10.20.193.16:0/747962927 learned_addr learned my addr 10.20.193.16:0/747962927 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde677fe640 10 monclient(hunting): get_auth_request con 0x7fde4c004ff0 auth_method 0 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde677fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:40.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde677fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde66ffd640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fde681518d0 0x7fde681f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde67fff640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fde681f5200 0x7fde681f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde66ffd640 10 monclient(hunting): get_auth_request con 0x7fde681518d0 auth_method 0 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde66ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde66ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde66ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde67fff640 10 monclient(hunting): get_auth_request con 0x7fde681f5200 auth_method 0 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde67fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.802+0000 7fde67fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.803+0000 7fde66ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.803+0000 7fde66ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.803+0000 7fde66ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.803+0000 7fde67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.803+0000 7fde67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.803+0000 7fde67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.804+0000 7fde677fe640 10 monclient(hunting): handle_auth_done global_id 4611 payload 824 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.804+0000 7fde677fe640 1 -- 10.20.193.16:0/747962927 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fde681f5200 msgr2=0x7fde681f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.804+0000 7fde677fe640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fde681f5200 0x7fde681f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.804+0000 7fde677fe640 1 -- 10.20.193.16:0/747962927 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fde681518d0 msgr2=0x7fde681f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.804+0000 7fde677fe640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fde681518d0 0x7fde681f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.804+0000 7fde677fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:40.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.804+0000 7fde677fe640 1 monclient: found mon.noname-a 2026-02-17T05:57:40.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.804+0000 7fde677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:40.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.804+0000 7fde677fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:40.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.804+0000 7fde677fe640 1 -- 10.20.193.16:0/747962927 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde681f5700 con 0x7fde4c004ff0 2026-02-17T05:57:40.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.805+0000 7fde677fe640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde680743f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fde54007fb0 tx=0x7fde680c4cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:40.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.805+0000 7fde43fff640 10 client.?.objecter ms_handle_connect 0x7fde4c004ff0 2026-02-17T05:57:40.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.805+0000 7fde43fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 1 -- 10.20.193.16:0/747962927 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5403c070 con 0x7fde4c004ff0 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 10 monclient: dump: 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 10 monclient: _finish_auth 0 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 1 -- 10.20.193.16:0/747962927 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde5403d040 con 0x7fde4c004ff0 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde6dabf640 5 monclient: authenticate success, global_id 4611 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde6dabf640 10 monclient: _renew_subs 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde6dabf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:40.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde6dabf640 1 -- 10.20.193.16:0/747962927 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde681f59e0 con 0x7fde4c004ff0 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 1 -- 10.20.193.16:0/747962927 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54003350 con 0x7fde4c004ff0 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde43fff640 10 monclient: dump: 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde65ffb640 4 set_mon_vals no callback set 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde6dabf640 10 client.4611.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde6dabf640 10 monclient: _renew_subs 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde6dabf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde6dabf640 1 -- 10.20.193.16:0/747962927 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde681f3040 con 0x7fde4c004ff0 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.806+0000 7fde6dabf640 1 librados: init done 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.807+0000 7fde6dabf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.807+0000 7fde6dabf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.807+0000 7fde6dabf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:40.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.807+0000 7fde6dabf640 1 -- 10.20.193.16:0/747962927 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde680704d0 con 0x7fde4c004ff0 2026-02-17T05:57:40.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.819+0000 7fde43fff640 1 -- 10.20.193.16:0/747962927 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fde54003ae0 con 0x7fde4c004ff0 2026-02-17T05:57:40.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.819+0000 7fde43fff640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fde38079190 0x7fde3807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:40.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.819+0000 7fde66ffd640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fde38079190 0x7fde3807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:40.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.820+0000 7fde66ffd640 10 monclient: get_auth_request con 0x7fde38079190 auth_method 0 2026-02-17T05:57:40.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.820+0000 7fde43fff640 10 client.4611.objecter ms_handle_connect 0x7fde38079190 2026-02-17T05:57:40.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.820+0000 7fde66ffd640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fde38079190 0x7fde3807b650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde580028f0 tx=0x7fde58004d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:40.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.927+0000 7fde43fff640 1 -- 10.20.193.16:0/747962927 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fde5402fe00 con 0x7fde4c004ff0 2026-02-17T05:57:40.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.927+0000 7fde43fff640 10 client.4611.objecter ms_dispatch2 0x7fde68000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:40.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.927+0000 7fde43fff640 3 client.4611.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:40.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.927+0000 7fde43fff640 3 client.4611.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:40.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.927+0000 7fde43fff640 20 client.4611.objecter dump_active .. 0 homeless 2026-02-17T05:57:40.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.927+0000 7fde43fff640 1 -- 10.20.193.16:0/747962927 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fde540871c0 con 0x7fde4c004ff0 2026-02-17T05:57:40.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.927+0000 7fde43fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:40.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:40.927+0000 7fde43fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:41.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.026+0000 7fde6dabf640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:41.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.026+0000 7fde6dabf640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:41.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.026+0000 7fde6dabf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:41.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.026+0000 7fde6dabf640 1 -- 10.20.193.16:0/747962927 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fde681573f0 con 0x7fde4c004ff0 2026-02-17T05:57:41.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.028+0000 7fde43fff640 1 -- 10.20.193.16:0/747962927 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fde5408b4c0 con 0x7fde4c004ff0 2026-02-17T05:57:41.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.028+0000 7fde43fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:41.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.028+0000 7fde43fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:41.029 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:41.029 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:41.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 10 librados: watch_flush enter 2026-02-17T05:57:41.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 10 librados: watch_flush exit 2026-02-17T05:57:41.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 20 client.4611.objecter shutdown clearing up homeless session... 2026-02-17T05:57:41.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 10 client.4611.objecter successfully canceled tick 2026-02-17T05:57:41.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 -- 10.20.193.16:0/747962927 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fde38079190 msgr2=0x7fde3807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:41.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fde38079190 0x7fde3807b650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde580028f0 tx=0x7fde58004d40 comp rx=0 tx=0).stop 2026-02-17T05:57:41.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 10 monclient: shutdown 2026-02-17T05:57:41.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:41.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 -- 10.20.193.16:0/747962927 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 msgr2=0x7fde680743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:41.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde680743f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fde54007fb0 tx=0x7fde680c4cd0 comp rx=0 tx=0).stop 2026-02-17T05:57:41.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 -- 10.20.193.16:0/747962927 shutdown_connections 2026-02-17T05:57:41.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fde38079190 0x7fde3807b650 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:41.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fde681518d0 0x7fde681f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:41.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fde681f5200 0x7fde681f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:41.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 --2- 10.20.193.16:0/747962927 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fde4c004ff0 0x7fde680743f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:41.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 -- 10.20.193.16:0/747962927 >> 10.20.193.16:0/747962927 conn(0x7fde680cd770 msgr2=0x7fde680c9490 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:41.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 -- 10.20.193.16:0/747962927 shutdown_connections 2026-02-17T05:57:41.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 -- 10.20.193.16:0/747962927 wait complete. 2026-02-17T05:57:41.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:41.029+0000 7fde6dabf640 1 librados: shutdown 2026-02-17T05:57:43.845 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:03:22.672 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:43.848 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:03:22.676 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:43.858 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:03:22.685 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:43.875 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:03:22.701 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:43.877 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:03:22.703 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:43.884 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:03:22.673 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:43.885 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:03:22.711 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:43.887 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:03:22.676 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:43.895 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:03:22.684 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:57:44.040 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:44.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.107+0000 7f6245aa0640 10 monclient: get_monmap_and_config 2026-02-17T05:57:44.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.107+0000 7f6245aa0640 10 monclient: build_initial_monmap 2026-02-17T05:57:44.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.107+0000 7f6245aa0640 10 monclient: monmap: 2026-02-17T05:57:44.108 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:44.108 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:44.108 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:44.108178+0000 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:44.108178+0000 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.107+0000 7f6245aa0640 1 Processor -- start 2026-02-17T05:57:44.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.107+0000 7f6245aa0640 1 -- start start 2026-02-17T05:57:44.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.107+0000 7f6245aa0640 10 monclient: init 2026-02-17T05:57:44.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:44.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 10 monclient: _add_conns ranks=[1,3,5,2,0,4] 2026-02-17T05:57:44.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f62400700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:57:44.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f62400c86f0 0x7f62400704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:44.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 10 monclient(hunting): picked mon.noname-a con 0x7f62400c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:57:44.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f62401536f0 0x7f6240153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:44.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 10 monclient(hunting): picked mon.noname-c con 0x7f62401536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:44.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62400c4820 con 0x7f62400700f0 2026-02-17T05:57:44.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f62400c4b00 con 0x7f62400c86f0 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f62400c4de0 con 0x7f62401536f0 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f6245aa0640 10 monclient(hunting): authenticate will time out at 1116.677133s 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623effd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f62400c86f0 0x7f62400704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623f7fe640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f62400700f0 0x7f62400c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:56438/0 (socket says 10.20.193.16:56438) 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623f7fe640 1 -- 10.20.193.16:0/3960232269 learned_addr learned my addr 10.20.193.16:0/3960232269 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623ffff640 1 --2- 10.20.193.16:0/3960232269 >> v2:10.20.193.148:3300/0 conn(0x7f62401536f0 0x7f6240153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623effd640 10 monclient(hunting): get_auth_request con 0x7f62400c86f0 auth_method 0 2026-02-17T05:57:44.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623ffff640 10 monclient(hunting): get_auth_request con 0x7f62401536f0 auth_method 0 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.108+0000 7f623ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.109+0000 7f623effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.109+0000 7f623effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.109+0000 7f623effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.109+0000 7f623ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.109+0000 7f623ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:44.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.109+0000 7f623ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:44.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623e7fc640 1 -- 10.20.193.16:0/3960232269 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4133652581 0 0) 0x7f62400c4820 con 0x7f62400700f0 2026-02-17T05:57:44.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623e7fc640 10 monclient(hunting): _negotiate 2026-02-17T05:57:44.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:44.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:44.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623e7fc640 10 monclient(hunting): my global_id is 4663 2026-02-17T05:57:44.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623e7fc640 1 -- 10.20.193.16:0/3960232269 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6224003620 con 0x7f62400700f0 2026-02-17T05:57:44.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623effd640 10 monclient(hunting): handle_auth_done global_id 4614 payload 274 2026-02-17T05:57:44.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623effd640 1 -- 10.20.193.16:0/3960232269 >> v2:10.20.193.148:3300/0 conn(0x7f62401536f0 msgr2=0x7f6240153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:44.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623effd640 1 --2- 10.20.193.16:0/3960232269 >> v2:10.20.193.148:3300/0 conn(0x7f62401536f0 0x7f6240153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623effd640 1 -- 10.20.193.16:0/3960232269 >> v1:10.20.193.128:6789/0 conn(0x7f62400700f0 legacy=0x7f62400c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623effd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623effd640 1 monclient: found mon.noname-a 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623effd640 1 -- 10.20.193.16:0/3960232269 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62400c50c0 con 0x7f62400c86f0 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.110+0000 7f623effd640 1 --2- 10.20.193.16:0/3960232269 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 0x7f62400704d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f62400c9540 tx=0x7f623002ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea2f653bf2ba524 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 1 -- 10.20.193.16:0/3960232269 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623003c070 con 0x7f62400c86f0 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 10 monclient: dump: 2026-02-17T05:57:44.121 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:44.122 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:44.122 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:44.122 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:44.122 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:44.122 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:44.122 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:44.122 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:44.122 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:44.122 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:44.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 1 -- 10.20.193.16:0/3960232269 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623002fa60 con 0x7f62400c86f0 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f6245aa0640 5 monclient: authenticate success, global_id 4614 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 1 -- 10.20.193.16:0/3960232269 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623002fe30 con 0x7f62400c86f0 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f623e7fc640 10 monclient: dump: 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:44.123 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:44.124 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:44.124 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:44.124 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:44.124 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:44.124 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:44.124 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:44.124 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:44.124 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:44.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f6245aa0640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:44.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.112+0000 7f6245aa0640 4 set_mon_vals no callback set 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 10 monclient: shutdown 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 1 -- 10.20.193.16:0/3960232269 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 msgr2=0x7f62400704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 1 --2- 10.20.193.16:0/3960232269 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 0x7f62400704d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f62400c9540 tx=0x7f623002ed50 comp rx=0 tx=0).stop 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 1 -- 10.20.193.16:0/3960232269 shutdown_connections 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 1 --2- 10.20.193.16:0/3960232269 >> v2:10.20.193.148:3300/0 conn(0x7f62401536f0 0x7f6240153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 1 --2- 10.20.193.16:0/3960232269 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 0x7f62400704d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 1 -- 10.20.193.16:0/3960232269 >> 10.20.193.16:0/3960232269 conn(0x7f62400cd770 msgr2=0x7f62400cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 1 -- 10.20.193.16:0/3960232269 shutdown_connections 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 1 -- 10.20.193.16:0/3960232269 wait complete. 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 10 monclient: build_initial_monmap 2026-02-17T05:57:44.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.113+0000 7f6245aa0640 10 monclient: monmap: 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 librados: starting msgr at 2026-02-17T05:57:44.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 librados: starting objecter 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 Processor -- start 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 -- start start 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 librados: setting wanted keys 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 librados: calling monclient init 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient: init 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f62400700f0 0x7f624015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient(hunting): picked mon.noname-c con 0x7f62400700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 0x7f6240152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient(hunting): picked mon.noname-a con 0x7f62400c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:44.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f62401536f0 0x7f624006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient(hunting): picked mon.noname-b con 0x7f62401536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6240154060 con 0x7f62400c86f0 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f62400c5680 con 0x7f62401536f0 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6240074c00 con 0x7f62400700f0 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f6245aa0640 10 monclient(hunting): authenticate will time out at 1116.683570s 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f623effd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 0x7f6240152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:44.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f623effd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 0x7f6240152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:42138/0 (socket says 10.20.193.16:42138) 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f623effd640 1 -- 10.20.193.16:0/3210976275 learned_addr learned my addr 10.20.193.16:0/3210976275 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f623effd640 10 monclient(hunting): get_auth_request con 0x7f62400c86f0 auth_method 0 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f623f7fe640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f62400700f0 0x7f624015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f623effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f623effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f623effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.114+0000 7f623ffff640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f62401536f0 0x7f624006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:44.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623f7fe640 10 monclient(hunting): get_auth_request con 0x7f62400700f0 auth_method 0 2026-02-17T05:57:44.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:44.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:44.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:44.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623ffff640 10 monclient(hunting): get_auth_request con 0x7f62401536f0 auth_method 0 2026-02-17T05:57:44.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:44.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:44.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:44.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:44.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:44.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.115+0000 7f623ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.116+0000 7f623effd640 10 monclient(hunting): handle_auth_done global_id 4617 payload 824 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.116+0000 7f623effd640 1 -- 10.20.193.16:0/3210976275 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f62400700f0 msgr2=0x7f624015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.116+0000 7f623effd640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f62400700f0 0x7f624015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.116+0000 7f623effd640 1 -- 10.20.193.16:0/3210976275 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f62401536f0 msgr2=0x7f624006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.116+0000 7f623effd640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f62401536f0 0x7f624006d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.116+0000 7f623effd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.116+0000 7f623effd640 1 monclient: found mon.noname-a 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.116+0000 7f623effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:44.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.116+0000 7f623effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:44.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.116+0000 7f623effd640 1 -- 10.20.193.16:0/3210976275 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624015b7d0 con 0x7f62400c86f0 2026-02-17T05:57:44.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.117+0000 7f623effd640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 0x7f6240152440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6230009130 tx=0x7f62300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:44.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.117+0000 7f621bfff640 10 client.?.objecter ms_handle_connect 0x7f62400c86f0 2026-02-17T05:57:44.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.117+0000 7f621bfff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:44.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 1 -- 10.20.193.16:0/3210976275 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230046020 con 0x7f62400c86f0 2026-02-17T05:57:44.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 10 monclient: dump: 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:44.136 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 10 monclient: _finish_auth 0 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 1 -- 10.20.193.16:0/3210976275 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6230042630 con 0x7f62400c86f0 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 1 -- 10.20.193.16:0/3210976275 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623003c040 con 0x7f62400c86f0 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 5 monclient: authenticate success, global_id 4617 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 10 monclient: _renew_subs 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f623dffb640 4 set_mon_vals no callback set 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 1 -- 10.20.193.16:0/3210976275 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62401f79d0 con 0x7f62400c86f0 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:44.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f621bfff640 10 monclient: dump: 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 10 client.4617.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:44.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 10 monclient: _renew_subs 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 1 -- 10.20.193.16:0/3210976275 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62401f3cd0 con 0x7f62400c86f0 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 1 librados: init done 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.119+0000 7f6245aa0640 1 -- 10.20.193.16:0/3210976275 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62400704d0 con 0x7f62400c86f0 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.130+0000 7f621bfff640 1 -- 10.20.193.16:0/3210976275 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f6230053070 con 0x7f62400c86f0 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.131+0000 7f621bfff640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6210079190 0x7f621007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.131+0000 7f623f7fe640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6210079190 0x7f621007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.131+0000 7f623f7fe640 10 monclient: get_auth_request con 0x7f6210079190 auth_method 0 2026-02-17T05:57:44.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.131+0000 7f623f7fe640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6210079190 0x7f621007b650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f622c007b60 tx=0x7f622c007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:44.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.131+0000 7f621bfff640 10 client.4617.objecter ms_handle_connect 0x7f6210079190 2026-02-17T05:57:44.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.133+0000 7f621bfff640 1 -- 10.20.193.16:0/3210976275 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f623002f920 con 0x7f62400c86f0 2026-02-17T05:57:44.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.133+0000 7f621bfff640 10 client.4617.objecter ms_dispatch2 0x7f6240000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:44.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.133+0000 7f621bfff640 3 client.4617.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:44.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.133+0000 7f621bfff640 3 client.4617.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:44.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.133+0000 7f621bfff640 20 client.4617.objecter dump_active .. 0 homeless 2026-02-17T05:57:44.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.240+0000 7f621bfff640 1 -- 10.20.193.16:0/3210976275 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6230050020 con 0x7f62400c86f0 2026-02-17T05:57:44.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.240+0000 7f621bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:44.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.240+0000 7f621bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:44.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.340+0000 7f6245aa0640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:44.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.340+0000 7f6245aa0640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:44.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.340+0000 7f6245aa0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:44.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.340+0000 7f6245aa0640 1 -- 10.20.193.16:0/3210976275 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f62401573f0 con 0x7f62400c86f0 2026-02-17T05:57:44.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.343+0000 7f621bfff640 1 -- 10.20.193.16:0/3210976275 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f62300877d0 con 0x7f62400c86f0 2026-02-17T05:57:44.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.343+0000 7f621bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:44.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.343+0000 7f621bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:44.344 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:44.344 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:44.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 10 librados: watch_flush enter 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 10 librados: watch_flush exit 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 20 client.4617.objecter shutdown clearing up homeless session... 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 10 client.4617.objecter successfully canceled tick 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 -- 10.20.193.16:0/3210976275 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6210079190 msgr2=0x7f621007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6210079190 0x7f621007b650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f622c007b60 tx=0x7f622c007ab0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 10 monclient: shutdown 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 -- 10.20.193.16:0/3210976275 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 msgr2=0x7f6240152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 0x7f6240152440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6230009130 tx=0x7f62300047c0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 -- 10.20.193.16:0/3210976275 shutdown_connections 2026-02-17T05:57:44.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6210079190 0x7f621007b650 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f62401536f0 0x7f624006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f62400c86f0 0x7f6240152440 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 --2- 10.20.193.16:0/3210976275 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f62400700f0 0x7f624015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:44.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 -- 10.20.193.16:0/3210976275 >> 10.20.193.16:0/3210976275 conn(0x7f62400cd770 msgr2=0x7f62400cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:44.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 -- 10.20.193.16:0/3210976275 shutdown_connections 2026-02-17T05:57:44.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 -- 10.20.193.16:0/3210976275 wait complete. 2026-02-17T05:57:44.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:44.344+0000 7f6245aa0640 1 librados: shutdown 2026-02-17T05:57:47.353 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:47.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.422+0000 7fd2017cc640 10 monclient: get_monmap_and_config 2026-02-17T05:57:47.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.422+0000 7fd2017cc640 10 monclient: build_initial_monmap 2026-02-17T05:57:47.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.422+0000 7fd2017cc640 10 monclient: monmap: 2026-02-17T05:57:47.423 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:47.423436+0000 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:47.423436+0000 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:47.424 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:47.425 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.422+0000 7fd2017cc640 1 Processor -- start 2026-02-17T05:57:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.422+0000 7fd2017cc640 1 -- start start 2026-02-17T05:57:47.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.422+0000 7fd2017cc640 10 monclient: init 2026-02-17T05:57:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 10 monclient: _add_conns ranks=[4,0,5,3,2,1] 2026-02-17T05:57:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd1fc0700f0 0x7fd1fc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 10 monclient(hunting): picked mon.noname-b con 0x7fd1fc0700f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:57:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd1fc0c7fe0 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd1fc1536f0 0x7fd1fc153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 10 monclient(hunting): picked mon.noname-c con 0x7fd1fc1536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1fc0c4820 con 0x7fd1fc0c7fe0 2026-02-17T05:57:47.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fd1fc0c4b00 con 0x7fd1fc0700f0 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd1fc0c4de0 con 0x7fd1fc1536f0 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd2017cc640 10 monclient(hunting): authenticate will time out at 1119.992391s 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd1fa7fc640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fd1fc0c7fe0 0x7fd1fc0c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:34300/0 (socket says 10.20.193.16:34300) 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd1fa7fc640 1 -- 10.20.193.16:0/1860088062 learned_addr learned my addr 10.20.193.16:0/1860088062 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd1fb7fe640 1 --2- 10.20.193.16:0/1860088062 >> v2:10.20.193.148:3300/0 conn(0x7fd1fc1536f0 0x7fd1fc153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.423+0000 7fd1faffd640 1 --2- 10.20.193.16:0/1860088062 >> v2:10.20.193.128:3300/0 conn(0x7fd1fc0700f0 0x7fd1fc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1fb7fe640 10 monclient(hunting): get_auth_request con 0x7fd1fc1536f0 auth_method 0 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:47.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1faffd640 10 monclient(hunting): get_auth_request con 0x7fd1fc0700f0 auth_method 0 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.424+0000 7fd1faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1f9ffb640 1 -- 10.20.193.16:0/1860088062 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2417772017 0 0) 0x7fd1fc0c4820 con 0x7fd1fc0c7fe0 2026-02-17T05:57:47.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1f9ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1f9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1f9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1f9ffb640 10 monclient(hunting): my global_id is 4620 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1f9ffb640 1 -- 10.20.193.16:0/1860088062 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1e8003620 con 0x7fd1fc0c7fe0 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1fb7fe640 10 monclient(hunting): handle_auth_done global_id 4742 payload 274 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1fb7fe640 1 -- 10.20.193.16:0/1860088062 >> v2:10.20.193.128:3300/0 conn(0x7fd1fc0700f0 msgr2=0x7fd1fc0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1fb7fe640 1 --2- 10.20.193.16:0/1860088062 >> v2:10.20.193.128:3300/0 conn(0x7fd1fc0700f0 0x7fd1fc0704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1fb7fe640 1 -- 10.20.193.16:0/1860088062 >> v1:10.20.193.16:6789/0 conn(0x7fd1fc0c7fe0 legacy=0x7fd1fc0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1fb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1fb7fe640 1 monclient: found mon.noname-c 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:47.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1fb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:47.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.425+0000 7fd1fb7fe640 1 -- 10.20.193.16:0/1860088062 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc0c50c0 con 0x7fd1fc1536f0 2026-02-17T05:57:47.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.426+0000 7fd1fb7fe640 1 --2- 10.20.193.16:0/1860088062 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd1fc1536f0 0x7fd1fc153b20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0c9540 tx=0x7fd1ec02f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a1a0db3e08c750d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 1 -- 10.20.193.16:0/1860088062 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec03c070 con 0x7fd1fc1536f0 2026-02-17T05:57:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:47.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 10 monclient: dump: 2026-02-17T05:57:47.438 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:47.438 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:47.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 1 -- 10.20.193.16:0/1860088062 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec02fe40 con 0x7fd1fc1536f0 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 5 monclient: authenticate success, global_id 4742 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 4 set_mon_vals no callback set 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 1 -- 10.20.193.16:0/1860088062 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec037640 con 0x7fd1fc1536f0 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd1f9ffb640 10 monclient: dump: 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:47.440 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 10 monclient: shutdown 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 1 -- 10.20.193.16:0/1860088062 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd1fc1536f0 msgr2=0x7fd1fc153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 1 --2- 10.20.193.16:0/1860088062 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd1fc1536f0 0x7fd1fc153b20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0c9540 tx=0x7fd1ec02f130 comp rx=0 tx=0).stop 2026-02-17T05:57:47.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 1 -- 10.20.193.16:0/1860088062 shutdown_connections 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 1 --2- 10.20.193.16:0/1860088062 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd1fc1536f0 0x7fd1fc153b20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 1 --2- 10.20.193.16:0/1860088062 >> v2:10.20.193.128:3300/0 conn(0x7fd1fc0700f0 0x7fd1fc0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 1 -- 10.20.193.16:0/1860088062 >> 10.20.193.16:0/1860088062 conn(0x7fd1fc0cd770 msgr2=0x7fd1fc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 1 -- 10.20.193.16:0/1860088062 shutdown_connections 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.428+0000 7fd2017cc640 1 -- 10.20.193.16:0/1860088062 wait complete. 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient: build_initial_monmap 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient: monmap: 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:47.442 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 librados: starting msgr at 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 librados: starting objecter 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 Processor -- start 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 -- start start 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 librados: setting wanted keys 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 librados: calling monclient init 2026-02-17T05:57:47.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient: init 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd1fc0700f0 0x7fd1fc1f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient(hunting): picked mon.noname-c con 0x7fd1fc0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd1fc0c7fe0 0x7fd1fc1f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient(hunting): picked mon.noname-b con 0x7fd1fc0c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd1fc1536f0 0x7fd1fc144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient(hunting): picked mon.noname-a con 0x7fd1fc1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc154060 con 0x7fd1fc1536f0 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:47.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc0c5680 con 0x7fd1fc0c7fe0 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc074c00 con 0x7fd1fc0700f0 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.429+0000 7fd2017cc640 10 monclient(hunting): authenticate will time out at 1119.998898s 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fb7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd1fc1536f0 0x7fd1fc144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1faffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd1fc0700f0 0x7fd1fc1f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fb7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd1fc1536f0 0x7fd1fc144c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:42148/0 (socket says 10.20.193.16:42148) 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fb7fe640 1 -- 10.20.193.16:0/283100574 learned_addr learned my addr 10.20.193.16:0/283100574 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fb7fe640 10 monclient(hunting): get_auth_request con 0x7fd1fc1536f0 auth_method 0 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fa7fc640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd1fc0c7fe0 0x7fd1fc1f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1faffd640 10 monclient(hunting): get_auth_request con 0x7fd1fc0700f0 auth_method 0 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fa7fc640 10 monclient(hunting): get_auth_request con 0x7fd1fc0c7fe0 auth_method 0 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.430+0000 7fd1fa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.431+0000 7fd1fb7fe640 10 monclient(hunting): handle_auth_done global_id 4623 payload 824 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.431+0000 7fd1fb7fe640 1 -- 10.20.193.16:0/283100574 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd1fc0700f0 msgr2=0x7fd1fc1f19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.431+0000 7fd1fb7fe640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd1fc0700f0 0x7fd1fc1f19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.431+0000 7fd1fb7fe640 1 -- 10.20.193.16:0/283100574 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd1fc0c7fe0 msgr2=0x7fd1fc1f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.431+0000 7fd1fb7fe640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd1fc0c7fe0 0x7fd1fc1f1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.431+0000 7fd1fb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:47.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.431+0000 7fd1fb7fe640 1 monclient: found mon.noname-a 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.431+0000 7fd1fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.431+0000 7fd1fb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.431+0000 7fd1fb7fe640 1 -- 10.20.193.16:0/283100574 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc14c9c0 con 0x7fd1fc1536f0 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.432+0000 7fd1fb7fe640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd1fc1536f0 0x7fd1fc144c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009e90 tx=0x7fd1ec004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.432+0000 7fd1db7fe640 10 client.?.objecter ms_handle_connect 0x7fd1fc1536f0 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.432+0000 7fd1db7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 1 -- 10.20.193.16:0/283100574 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec047070 con 0x7fd1fc1536f0 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 10 monclient: dump: 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:47.448 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:47.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 1 -- 10.20.193.16:0/283100574 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec003a90 con 0x7fd1fc1536f0 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd2017cc640 5 monclient: authenticate success, global_id 4623 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 1 -- 10.20.193.16:0/283100574 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec03c040 con 0x7fd1fc1536f0 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1db7fe640 10 monclient: dump: 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:47.450 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd1f97fa640 4 set_mon_vals no callback set 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd2017cc640 10 monclient: _renew_subs 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd2017cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd2017cc640 1 -- 10.20.193.16:0/283100574 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc14cca0 con 0x7fd1fc1536f0 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd2017cc640 10 client.4623.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd2017cc640 10 monclient: _renew_subs 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd2017cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:47.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd2017cc640 1 -- 10.20.193.16:0/283100574 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc14d260 con 0x7fd1fc1536f0 2026-02-17T05:57:47.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.434+0000 7fd2017cc640 1 librados: init done 2026-02-17T05:57:47.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.435+0000 7fd2017cc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:47.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.435+0000 7fd2017cc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:47.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.435+0000 7fd2017cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:47.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.435+0000 7fd2017cc640 1 -- 10.20.193.16:0/283100574 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1fc0704d0 con 0x7fd1fc1536f0 2026-02-17T05:57:47.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.447+0000 7fd1db7fe640 1 -- 10.20.193.16:0/283100574 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fd1ec004030 con 0x7fd1fc1536f0 2026-02-17T05:57:47.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.447+0000 7fd1db7fe640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1c8079190 0x7fd1c807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.447+0000 7fd1faffd640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1c8079190 0x7fd1c807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.447+0000 7fd1faffd640 10 monclient: get_auth_request con 0x7fd1c8079190 auth_method 0 2026-02-17T05:57:47.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.448+0000 7fd1db7fe640 10 client.4623.objecter ms_handle_connect 0x7fd1c8079190 2026-02-17T05:57:47.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.448+0000 7fd1faffd640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1c8079190 0x7fd1c807b650 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009ba0 tx=0x7fd1f0009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:47.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.547+0000 7fd1db7fe640 1 -- 10.20.193.16:0/283100574 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fd1ec083b30 con 0x7fd1fc1536f0 2026-02-17T05:57:47.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.547+0000 7fd1db7fe640 10 client.4623.objecter ms_dispatch2 0x7fd1fc000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:47.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.547+0000 7fd1db7fe640 3 client.4623.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:47.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.547+0000 7fd1db7fe640 3 client.4623.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:47.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.547+0000 7fd1db7fe640 20 client.4623.objecter dump_active .. 0 homeless 2026-02-17T05:57:47.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.547+0000 7fd1db7fe640 1 -- 10.20.193.16:0/283100574 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd1ec051020 con 0x7fd1fc1536f0 2026-02-17T05:57:47.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.547+0000 7fd1db7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:47.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.547+0000 7fd1db7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:47.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.649+0000 7fd2017cc640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:47.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.649+0000 7fd2017cc640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:47.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.649+0000 7fd2017cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:47.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.649+0000 7fd2017cc640 1 -- 10.20.193.16:0/283100574 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd1fc1573f0 con 0x7fd1fc1536f0 2026-02-17T05:57:47.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.652+0000 7fd1db7fe640 1 -- 10.20.193.16:0/283100574 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd1ec087af0 con 0x7fd1fc1536f0 2026-02-17T05:57:47.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.652+0000 7fd1db7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:47.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.652+0000 7fd1db7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:47.653 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:47.653 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:47.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 10 librados: watch_flush enter 2026-02-17T05:57:47.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 10 librados: watch_flush exit 2026-02-17T05:57:47.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 20 client.4623.objecter shutdown clearing up homeless session... 2026-02-17T05:57:47.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 10 client.4623.objecter successfully canceled tick 2026-02-17T05:57:47.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 -- 10.20.193.16:0/283100574 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1c8079190 msgr2=0x7fd1c807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:47.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1c8079190 0x7fd1c807b650 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009ba0 tx=0x7fd1f0009290 comp rx=0 tx=0).stop 2026-02-17T05:57:47.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 10 monclient: shutdown 2026-02-17T05:57:47.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:47.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 -- 10.20.193.16:0/283100574 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd1fc1536f0 msgr2=0x7fd1fc144c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:47.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd1fc1536f0 0x7fd1fc144c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009e90 tx=0x7fd1ec004b40 comp rx=0 tx=0).stop 2026-02-17T05:57:47.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 -- 10.20.193.16:0/283100574 shutdown_connections 2026-02-17T05:57:47.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1c8079190 0x7fd1c807b650 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd1fc0c7fe0 0x7fd1fc1f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd1fc1536f0 0x7fd1fc144c90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 --2- 10.20.193.16:0/283100574 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd1fc0700f0 0x7fd1fc1f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 -- 10.20.193.16:0/283100574 >> 10.20.193.16:0/283100574 conn(0x7fd1fc0cd770 msgr2=0x7fd1fc0c8410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:47.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 -- 10.20.193.16:0/283100574 shutdown_connections 2026-02-17T05:57:47.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 -- 10.20.193.16:0/283100574 wait complete. 2026-02-17T05:57:47.656 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:47.653+0000 7fd2017cc640 1 librados: shutdown 2026-02-17T05:57:50.663 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:50.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.729+0000 7fdaf0da9640 10 monclient: get_monmap_and_config 2026-02-17T05:57:50.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.729+0000 7fdaf0da9640 10 monclient: build_initial_monmap 2026-02-17T05:57:50.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.729+0000 7fdaf0da9640 10 monclient: monmap: 2026-02-17T05:57:50.731 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:50.730173+0000 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:50.730173+0000 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:50.732 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.729+0000 7fdaf0da9640 1 Processor -- start 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.729+0000 7fdaf0da9640 1 -- start start 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.729+0000 7fdaf0da9640 10 monclient: init 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 10 monclient: _add_conns ranks=[0,1,5,4,2,3] 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdaec0700f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fdaec0c86f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fdaec153e00 0x7fdaec0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 10 monclient(hunting): picked mon.noname-c con 0x7fdaec153e00 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaec0c4820 con 0x7fdaec0700f0 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaec0c4b00 con 0x7fdaec0c86f0 2026-02-17T05:57:50.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fdaec0c4de0 con 0x7fdaec153e00 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaf0da9640 10 monclient(hunting): authenticate will time out at 1123.299115s 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaea575640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fdaec0700f0 0x7fdaec0c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:48032/0 (socket says 10.20.193.16:48032) 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaea575640 1 -- 10.20.193.16:0/1804213689 learned_addr learned my addr 10.20.193.16:0/1804213689 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaead76640 1 --2- 10.20.193.16:0/1804213689 >> v2:10.20.193.148:3300/0 conn(0x7fdaec153e00 0x7fdaec0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaead76640 10 monclient(hunting): get_auth_request con 0x7fdaec153e00 auth_method 0 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaead76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaead76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.730+0000 7fdaead76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:50.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:50.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:50.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdae9573640 1 -- 10.20.193.16:0/1804213689 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231857736 0 0) 0x7fdaec0c4820 con 0x7fdaec0700f0 2026-02-17T05:57:50.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdae9573640 10 monclient(hunting): _negotiate 2026-02-17T05:57:50.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdae9573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:50.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdae9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdae9573640 10 monclient(hunting): my global_id is 4626 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdae9573640 1 -- 10.20.193.16:0/1804213689 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdad8003620 con 0x7fdaec0700f0 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 10 monclient(hunting): handle_auth_done global_id 4748 payload 274 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 1 -- 10.20.193.16:0/1804213689 >> v1:10.20.193.128:6789/0 conn(0x7fdaec0c86f0 legacy=0x7fdaec1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 1 -- 10.20.193.16:0/1804213689 >> v1:10.20.193.16:6789/0 conn(0x7fdaec0700f0 legacy=0x7fdaec0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 10 monclient: _finish_hunting 0 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 1 monclient: found mon.noname-c 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.731+0000 7fdaead76640 1 -- 10.20.193.16:0/1804213689 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec0c50c0 con 0x7fdaec153e00 2026-02-17T05:57:50.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.732+0000 7fdaead76640 1 --2- 10.20.193.16:0/1804213689 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdaec153e00 0x7fdaec0704d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009d90 tx=0x7fdadc02f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8542993319a7b5fa server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 1 -- 10.20.193.16:0/1804213689 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc03c070 con 0x7fdaec153e00 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 10 monclient: dump: 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:50.744 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 10 monclient: _finish_auth 0 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 10 monclient: _check_auth_tickets 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 1 -- 10.20.193.16:0/1804213689 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdadc037440 con 0x7fdaec153e00 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdaf0da9640 5 monclient: authenticate success, global_id 4748 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdaf0da9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 1 -- 10.20.193.16:0/1804213689 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc037810 con 0x7fdaec153e00 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdaf0da9640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:50.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdaf0da9640 4 set_mon_vals no callback set 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.734+0000 7fdae9573640 10 monclient: dump: 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:50.746 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 10 monclient: shutdown 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 1 -- 10.20.193.16:0/1804213689 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdaec153e00 msgr2=0x7fdaec0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 1 --2- 10.20.193.16:0/1804213689 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdaec153e00 0x7fdaec0704d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009d90 tx=0x7fdadc02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 1 -- 10.20.193.16:0/1804213689 shutdown_connections 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 1 --2- 10.20.193.16:0/1804213689 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdaec153e00 0x7fdaec0704d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 1 -- 10.20.193.16:0/1804213689 >> 10.20.193.16:0/1804213689 conn(0x7fdaec0cd770 msgr2=0x7fdaec0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 1 -- 10.20.193.16:0/1804213689 shutdown_connections 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 1 -- 10.20.193.16:0/1804213689 wait complete. 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 10 monclient: build_initial_monmap 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 10 monclient: monmap: 2026-02-17T05:57:50.747 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 1 librados: starting msgr at 2026-02-17T05:57:50.748 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 1 librados: starting objecter 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.735+0000 7fdaf0da9640 1 Processor -- start 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 1 -- start start 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 1 librados: setting wanted keys 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 1 librados: calling monclient init 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient: init 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdaec0700f0 0x7fdaec1f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient(hunting): picked mon.noname-b con 0x7fdaec0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdaec0c86f0 0x7fdaec06e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:50.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient(hunting): picked mon.noname-c con 0x7fdaec0c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdaec153e00 0x7fdaec1f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient(hunting): picked mon.noname-a con 0x7fdaec153e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec074c00 con 0x7fdaec153e00 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec0c5680 con 0x7fdaec0700f0 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec074d80 con 0x7fdaec0c86f0 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaf0da9640 10 monclient(hunting): authenticate will time out at 1123.305435s 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaead76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdaec153e00 0x7fdaec1f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:50.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaead76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdaec153e00 0x7fdaec1f41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:35394/0 (socket says 10.20.193.16:35394) 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaead76640 1 -- 10.20.193.16:0/3213273816 learned_addr learned my addr 10.20.193.16:0/3213273816 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaead76640 10 monclient(hunting): get_auth_request con 0x7fdaec153e00 auth_method 0 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaead76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaead76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaead76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdae9d74640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdaec0c86f0 0x7fdaec06e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdaea575640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdaec0700f0 0x7fdaec1f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdae9d74640 10 monclient(hunting): get_auth_request con 0x7fdaec0c86f0 auth_method 0 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdae9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdae9d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.736+0000 7fdae9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:50.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdaea575640 10 monclient(hunting): get_auth_request con 0x7fdaec0700f0 auth_method 0 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdaea575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdaea575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdaea575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdaead76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdaead76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdaead76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdae9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdae9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdae9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdaea575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdaea575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.737+0000 7fdaea575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:50.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.738+0000 7fdae9d74640 10 monclient(hunting): handle_auth_done global_id 4751 payload 824 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.738+0000 7fdae9d74640 1 -- 10.20.193.16:0/3213273816 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdaec0700f0 msgr2=0x7fdaec1f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.738+0000 7fdae9d74640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdaec0700f0 0x7fdaec1f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.738+0000 7fdae9d74640 1 -- 10.20.193.16:0/3213273816 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdaec153e00 msgr2=0x7fdaec1f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.738+0000 7fdae9d74640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdaec153e00 0x7fdaec1f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.738+0000 7fdae9d74640 10 monclient: _finish_hunting 0 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.738+0000 7fdae9d74640 1 monclient: found mon.noname-c 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.738+0000 7fdae9d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.738+0000 7fdae9d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.738+0000 7fdae9d74640 1 -- 10.20.193.16:0/3213273816 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec06e7c0 con 0x7fdaec0c86f0 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.739+0000 7fdae9d74640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdaec0c86f0 0x7fdaec06e280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c450 tx=0x7fdad400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.739+0000 7fdad2ffd640 10 client.?.objecter ms_handle_connect 0x7fdaec0c86f0 2026-02-17T05:57:50.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.739+0000 7fdad2ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdad2ffd640 1 -- 10.20.193.16:0/3213273816 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad40052d0 con 0x7fdaec0c86f0 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdad2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdad2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdad2ffd640 10 monclient: dump: 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:50.754 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdad2ffd640 10 monclient: _finish_auth 0 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdad2ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdad2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdaf0da9640 5 monclient: authenticate success, global_id 4751 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdaf0da9640 10 monclient: _renew_subs 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdaf0da9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdaf0da9640 1 -- 10.20.193.16:0/3213273816 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaec1f7cd0 con 0x7fdaec0c86f0 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdaf0da9640 10 client.4751.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdaf0da9640 10 monclient: _renew_subs 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdaf0da9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdaf0da9640 1 -- 10.20.193.16:0/3213273816 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaec1f7430 con 0x7fdaec0c86f0 2026-02-17T05:57:50.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.740+0000 7fdaf0da9640 1 librados: init done 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdaf0da9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdaf0da9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdaf0da9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdaf0da9640 1 -- 10.20.193.16:0/3213273816 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaec0704d0 con 0x7fdaec0c86f0 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdad2ffd640 1 -- 10.20.193.16:0/3213273816 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad4002930 con 0x7fdaec0c86f0 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdad2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdad2ffd640 1 -- 10.20.193.16:0/3213273816 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad40036f0 con 0x7fdaec0c86f0 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdad2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdad2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdae8d72640 4 set_mon_vals no callback set 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.741+0000 7fdad2ffd640 10 monclient: dump: 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:50.756 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:50.757 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:50.757 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:50.757 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:50.757 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:50.757 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:50.757 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:50.757 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:50.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.847+0000 7fdad2ffd640 1 -- 10.20.193.16:0/3213273816 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fdad4003950 con 0x7fdaec0c86f0 2026-02-17T05:57:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.848+0000 7fdad2ffd640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdab8079140 0x7fdab807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.848+0000 7fdaea575640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdab8079140 0x7fdab807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.848+0000 7fdaea575640 10 monclient: get_auth_request con 0x7fdab8079140 auth_method 0 2026-02-17T05:57:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.848+0000 7fdaea575640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdab8079140 0x7fdab807b600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdae0005980 tx=0x7fdae0007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:50.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.848+0000 7fdad2ffd640 10 client.4751.objecter ms_handle_connect 0x7fdab8079140 2026-02-17T05:57:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.849+0000 7fdad2ffd640 1 -- 10.20.193.16:0/3213273816 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fdad405e1c0 con 0x7fdaec0c86f0 2026-02-17T05:57:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.849+0000 7fdad2ffd640 10 client.4751.objecter ms_dispatch2 0x7fdaec000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.849+0000 7fdad2ffd640 3 client.4751.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.849+0000 7fdad2ffd640 3 client.4751.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:50.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.849+0000 7fdad2ffd640 20 client.4751.objecter dump_active .. 0 homeless 2026-02-17T05:57:50.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.855+0000 7fdad2ffd640 1 -- 10.20.193.16:0/3213273816 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdad4062180 con 0x7fdaec0c86f0 2026-02-17T05:57:50.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.855+0000 7fdad2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:50.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.855+0000 7fdad2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:50.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.954+0000 7fdaf0da9640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:50.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.954+0000 7fdaf0da9640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:50.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.954+0000 7fdaf0da9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:57:50.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.954+0000 7fdaf0da9640 1 -- 10.20.193.16:0/3213273816 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdaec140a30 con 0x7fdaec0c86f0 2026-02-17T05:57:50.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.956+0000 7fdad2ffd640 1 -- 10.20.193.16:0/3213273816 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fdad4066480 con 0x7fdaec0c86f0 2026-02-17T05:57:50.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.956+0000 7fdad2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:50.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.956+0000 7fdad2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:50.957 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:50.957 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:50.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 10 librados: watch_flush enter 2026-02-17T05:57:50.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 10 librados: watch_flush exit 2026-02-17T05:57:50.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 20 client.4751.objecter shutdown clearing up homeless session... 2026-02-17T05:57:50.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 10 client.4751.objecter successfully canceled tick 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 -- 10.20.193.16:0/3213273816 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdab8079140 msgr2=0x7fdab807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdab8079140 0x7fdab807b600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdae0005980 tx=0x7fdae0007dd0 comp rx=0 tx=0).stop 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 10 monclient: shutdown 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 -- 10.20.193.16:0/3213273816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdaec0c86f0 msgr2=0x7fdaec06e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdaec0c86f0 0x7fdaec06e280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c450 tx=0x7fdad400c920 comp rx=0 tx=0).stop 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 -- 10.20.193.16:0/3213273816 shutdown_connections 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdab8079140 0x7fdab807b600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdaec153e00 0x7fdaec1f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdaec0c86f0 0x7fdaec06e280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:50.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 --2- 10.20.193.16:0/3213273816 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdaec0700f0 0x7fdaec1f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:50.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 -- 10.20.193.16:0/3213273816 >> 10.20.193.16:0/3213273816 conn(0x7fdaec0cd770 msgr2=0x7fdaec14e560 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:50.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 -- 10.20.193.16:0/3213273816 shutdown_connections 2026-02-17T05:57:50.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 -- 10.20.193.16:0/3213273816 wait complete. 2026-02-17T05:57:50.961 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:50.958+0000 7fdaf0da9640 1 librados: shutdown 2026-02-17T05:57:53.967 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:54.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.035+0000 7f99e587c640 10 monclient: get_monmap_and_config 2026-02-17T05:57:54.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.035+0000 7f99e587c640 10 monclient: build_initial_monmap 2026-02-17T05:57:54.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.035+0000 7f99e587c640 10 monclient: monmap: 2026-02-17T05:57:54.036 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:54.036 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:54.036 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:54.036108+0000 2026-02-17T05:57:54.036 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:54.036108+0000 2026-02-17T05:57:54.036 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.035+0000 7f99e587c640 1 Processor -- start 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.035+0000 7f99e587c640 1 -- start start 2026-02-17T05:57:54.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.035+0000 7f99e587c640 10 monclient: init 2026-02-17T05:57:54.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:54.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 10 monclient: _add_conns ranks=[5,3,1,0,4,2] 2026-02-17T05:57:54.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f99e00700f0 0x7f99e00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:54.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 10 monclient(hunting): picked mon.noname-c con 0x7f99e00700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:57:54.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f99e00c7fe0 0x7f99e00c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 10 monclient(hunting): picked mon.noname-a con 0x7f99e00c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f99e01536f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e00c8970 con 0x7f99e01536f0 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f99e00c4820 con 0x7f99e00c7fe0 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f99e00c4b00 con 0x7f99e00700f0 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99e587c640 10 monclient(hunting): authenticate will time out at 1126.605181s 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99deffd640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f99e00700f0 0x7f99e00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99de7fc640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f99e00c7fe0 0x7f99e00c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:54.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99df7fe640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f99e01536f0 0x7f99e0153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:33384/0 (socket says 10.20.193.16:33384) 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99df7fe640 1 -- 10.20.193.16:0/1219112109 learned_addr learned my addr 10.20.193.16:0/1219112109 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99de7fc640 10 monclient(hunting): get_auth_request con 0x7f99e00c7fe0 auth_method 0 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99de7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99de7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99de7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99deffd640 10 monclient(hunting): get_auth_request con 0x7f99e00700f0 auth_method 0 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99deffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.036+0000 7f99deffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.037+0000 7f99deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.037+0000 7f99deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:54.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.037+0000 7f99deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:54.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.037+0000 7f99de7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:54.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.037+0000 7f99de7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:54.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.037+0000 7f99de7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:54.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.038+0000 7f99deffd640 10 monclient(hunting): handle_auth_done global_id 4754 payload 274 2026-02-17T05:57:54.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.038+0000 7f99deffd640 1 -- 10.20.193.16:0/1219112109 >> v2:10.20.193.16:3300/0 conn(0x7f99e00c7fe0 msgr2=0x7f99e00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:54.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.038+0000 7f99deffd640 1 --2- 10.20.193.16:0/1219112109 >> v2:10.20.193.16:3300/0 conn(0x7f99e00c7fe0 0x7f99e00c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:54.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.038+0000 7f99deffd640 1 -- 10.20.193.16:0/1219112109 >> v1:10.20.193.128:6789/0 conn(0x7f99e01536f0 legacy=0x7f99e0153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:54.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.038+0000 7f99deffd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:54.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.038+0000 7f99deffd640 1 monclient: found mon.noname-c 2026-02-17T05:57:54.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.038+0000 7f99deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:54.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.038+0000 7f99deffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:57:54.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.038+0000 7f99deffd640 1 -- 10.20.193.16:0/1219112109 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e00c4de0 con 0x7f99e00700f0 2026-02-17T05:57:54.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.039+0000 7f99deffd640 1 --2- 10.20.193.16:0/1219112109 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f99e00700f0 0x7f99e00704d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99e0070b90 tx=0x7f99d402ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b6b66b862b7e90 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:54.050 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 1 -- 10.20.193.16:0/1219112109 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d403c070 con 0x7f99e00700f0 2026-02-17T05:57:54.050 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:54.050 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 10 monclient: dump: 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:54.051 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99e587c640 5 monclient: authenticate success, global_id 4754 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99e587c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 1 -- 10.20.193.16:0/1219112109 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d402fa60 con 0x7f99e00700f0 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 1 -- 10.20.193.16:0/1219112109 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d402fe30 con 0x7f99e00700f0 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99ddffb640 10 monclient: dump: 2026-02-17T05:57:54.052 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99e587c640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.040+0000 7f99e587c640 4 set_mon_vals no callback set 2026-02-17T05:57:54.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 10 monclient: shutdown 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 1 -- 10.20.193.16:0/1219112109 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f99e00700f0 msgr2=0x7f99e00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 1 --2- 10.20.193.16:0/1219112109 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f99e00700f0 0x7f99e00704d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99e0070b90 tx=0x7f99d402ed50 comp rx=0 tx=0).stop 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 1 -- 10.20.193.16:0/1219112109 shutdown_connections 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 1 --2- 10.20.193.16:0/1219112109 >> v2:10.20.193.16:3300/0 conn(0x7f99e00c7fe0 0x7f99e00c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 1 --2- 10.20.193.16:0/1219112109 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f99e00700f0 0x7f99e00704d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 1 -- 10.20.193.16:0/1219112109 >> 10.20.193.16:0/1219112109 conn(0x7f99e00cd770 msgr2=0x7f99e00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 1 -- 10.20.193.16:0/1219112109 shutdown_connections 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 1 -- 10.20.193.16:0/1219112109 wait complete. 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 10 monclient: build_initial_monmap 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 10 monclient: monmap: 2026-02-17T05:57:54.054 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 1 librados: starting msgr at 2026-02-17T05:57:54.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.041+0000 7f99e587c640 1 librados: starting objecter 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 1 Processor -- start 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 1 -- start start 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 1 librados: setting wanted keys 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 1 librados: calling monclient init 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient: init 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f99e00700f0 0x7f99e01e8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient(hunting): picked mon.noname-a con 0x7f99e00700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f99e00c7fe0 0x7f99e00cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient(hunting): picked mon.noname-b con 0x7f99e00c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f99e01536f0 0x7f99e00cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:54.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient(hunting): picked mon.noname-c con 0x7f99e01536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99e00c5680 con 0x7f99e00700f0 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f99e00c53a0 con 0x7f99e00c7fe0 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0074c00 con 0x7f99e01536f0 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99e587c640 10 monclient(hunting): authenticate will time out at 1126.611510s 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99deffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f99e00700f0 0x7f99e01e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99df7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f99e01536f0 0x7f99e00cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:54.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99deffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f99e00700f0 0x7f99e01e8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:35404/0 (socket says 10.20.193.16:35404) 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99deffd640 1 -- 10.20.193.16:0/2485081287 learned_addr learned my addr 10.20.193.16:0/2485081287 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99deffd640 10 monclient(hunting): get_auth_request con 0x7f99e00700f0 auth_method 0 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99deffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99deffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.042+0000 7f99de7fc640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f99e00c7fe0 0x7f99e00cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99df7fe640 10 monclient(hunting): get_auth_request con 0x7f99e01536f0 auth_method 0 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99df7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99df7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:54.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99de7fc640 10 monclient(hunting): get_auth_request con 0x7f99e00c7fe0 auth_method 0 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99de7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99de7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99de7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99de7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99de7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.043+0000 7f99de7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.044+0000 7f99deffd640 10 monclient(hunting): handle_auth_done global_id 4635 payload 824 2026-02-17T05:57:54.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.044+0000 7f99deffd640 1 -- 10.20.193.16:0/2485081287 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f99e01536f0 msgr2=0x7f99e00cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.044+0000 7f99deffd640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f99e01536f0 0x7f99e00cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.044+0000 7f99deffd640 1 -- 10.20.193.16:0/2485081287 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f99e00c7fe0 msgr2=0x7f99e00cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.044+0000 7f99deffd640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f99e00c7fe0 0x7f99e00cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.044+0000 7f99deffd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.044+0000 7f99deffd640 1 monclient: found mon.noname-a 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.044+0000 7f99deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.044+0000 7f99deffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.044+0000 7f99deffd640 1 -- 10.20.193.16:0/2485081287 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e0072720 con 0x7f99e00700f0 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.045+0000 7f99deffd640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f99e00700f0 0x7f99e01e8320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f99d4002410 tx=0x7f99d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.045+0000 7f99b7fff640 10 client.?.objecter ms_handle_connect 0x7f99e00700f0 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.045+0000 7f99b7fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 1 -- 10.20.193.16:0/2485081287 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d403c040 con 0x7f99e00700f0 2026-02-17T05:57:54.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 10 monclient: dump: 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:54.061 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 10 monclient: _finish_auth 0 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 1 -- 10.20.193.16:0/2485081287 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d4042450 con 0x7f99e00700f0 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 1 -- 10.20.193.16:0/2485081287 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4042800 con 0x7f99e00700f0 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99b7fff640 10 monclient: dump: 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:54.062 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99dd7fa640 4 set_mon_vals no callback set 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99e587c640 5 monclient: authenticate success, global_id 4635 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99e587c640 10 monclient: _renew_subs 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99e587c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:54.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99e587c640 1 -- 10.20.193.16:0/2485081287 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e01e8cf0 con 0x7f99e00700f0 2026-02-17T05:57:54.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99e587c640 10 client.4635.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:54.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99e587c640 10 monclient: _renew_subs 2026-02-17T05:57:54.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99e587c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:54.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99e587c640 1 -- 10.20.193.16:0/2485081287 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e01e92b0 con 0x7f99e00700f0 2026-02-17T05:57:54.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.047+0000 7f99e587c640 1 librados: init done 2026-02-17T05:57:54.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.048+0000 7f99e587c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:54.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.048+0000 7f99e587c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:54.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.048+0000 7f99e587c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:54.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.048+0000 7f99e587c640 1 -- 10.20.193.16:0/2485081287 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e00704d0 con 0x7f99e00700f0 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.059+0000 7f99b7fff640 1 -- 10.20.193.16:0/2485081287 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f99d40429a0 con 0x7f99e00700f0 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.060+0000 7f99b7fff640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f99b0079140 0x7f99b007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.060+0000 7f99de7fc640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f99b0079140 0x7f99b007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.060+0000 7f99de7fc640 10 monclient: get_auth_request con 0x7f99b0079140 auth_method 0 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.060+0000 7f99b7fff640 10 client.4635.objecter ms_handle_connect 0x7f99b0079140 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.060+0000 7f99de7fc640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f99b0079140 0x7f99b007b600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f99e01ea0e0 tx=0x7f99c8009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.062+0000 7f99b7fff640 1 -- 10.20.193.16:0/2485081287 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f99d4083980 con 0x7f99e00700f0 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.062+0000 7f99b7fff640 10 client.4635.objecter ms_dispatch2 0x7f99e0000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.062+0000 7f99b7fff640 3 client.4635.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.062+0000 7f99b7fff640 3 client.4635.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:54.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.062+0000 7f99b7fff640 20 client.4635.objecter dump_active .. 0 homeless 2026-02-17T05:57:54.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.165+0000 7f99b7fff640 1 -- 10.20.193.16:0/2485081287 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f99d4087940 con 0x7f99e00700f0 2026-02-17T05:57:54.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.165+0000 7f99b7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:54.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.165+0000 7f99b7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:54.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.264+0000 7f99e587c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:54.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.264+0000 7f99e587c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:54.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.264+0000 7f99e587c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:57:54.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.264+0000 7f99e587c640 1 -- 10.20.193.16:0/2485081287 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f99e01573f0 con 0x7f99e00700f0 2026-02-17T05:57:54.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.268+0000 7f99b7fff640 1 -- 10.20.193.16:0/2485081287 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f99d408bc40 con 0x7f99e00700f0 2026-02-17T05:57:54.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.268+0000 7f99b7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:54.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.268+0000 7f99b7fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:54.269 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:54.269 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:54.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 10 librados: watch_flush enter 2026-02-17T05:57:54.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 10 librados: watch_flush exit 2026-02-17T05:57:54.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 20 client.4635.objecter shutdown clearing up homeless session... 2026-02-17T05:57:54.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 10 client.4635.objecter successfully canceled tick 2026-02-17T05:57:54.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 -- 10.20.193.16:0/2485081287 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f99b0079140 msgr2=0x7f99b007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:54.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f99b0079140 0x7f99b007b600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f99e01ea0e0 tx=0x7f99c8009290 comp rx=0 tx=0).stop 2026-02-17T05:57:54.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 10 monclient: shutdown 2026-02-17T05:57:54.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 -- 10.20.193.16:0/2485081287 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f99e00700f0 msgr2=0x7f99e01e8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f99e00700f0 0x7f99e01e8320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f99d4002410 tx=0x7f99d40047c0 comp rx=0 tx=0).stop 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 -- 10.20.193.16:0/2485081287 shutdown_connections 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f99b0079140 0x7f99b007b600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f99e00c7fe0 0x7f99e00cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f99e01536f0 0x7f99e00cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 --2- 10.20.193.16:0/2485081287 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f99e00700f0 0x7f99e01e8320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 -- 10.20.193.16:0/2485081287 >> 10.20.193.16:0/2485081287 conn(0x7f99e00cd770 msgr2=0x7f99e00cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 -- 10.20.193.16:0/2485081287 shutdown_connections 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 -- 10.20.193.16:0/2485081287 wait complete. 2026-02-17T05:57:54.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:54.269+0000 7f99e587c640 1 librados: shutdown 2026-02-17T05:57:57.279 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:57:57.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.346+0000 7f4f97190640 10 monclient: get_monmap_and_config 2026-02-17T05:57:57.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.346+0000 7f4f97190640 10 monclient: build_initial_monmap 2026-02-17T05:57:57.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.346+0000 7f4f97190640 10 monclient: monmap: 2026-02-17T05:57:57.347 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:57.347 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:57:57.347698+0000 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:57:57.347698+0000 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:57:57.348 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:57.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 1 Processor -- start 2026-02-17T05:57:57.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 1 -- start start 2026-02-17T05:57:57.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 10 monclient: init 2026-02-17T05:57:57.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:57.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 10 monclient: _add_conns ranks=[2,1,4,0,3,5] 2026-02-17T05:57:57.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4f900700f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:57:57.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f4f900c86f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:57:57.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f4f90153e00 0x7f4f900704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:57.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 10 monclient(hunting): picked mon.noname-b con 0x7f4f90153e00 addr v2:10.20.193.128:3300/0 2026-02-17T05:57:57.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f900c4820 con 0x7f4f900c86f0 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f900c4b00 con 0x7f4f900700f0 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f4f900c4de0 con 0x7f4f90153e00 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.347+0000 7f4f97190640 10 monclient(hunting): authenticate will time out at 1129.916989s 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.348+0000 7f4f95706640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f4f90153e00 0x7f4f900704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.348+0000 7f4f94f05640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f4f900700f0 0x7f4f900c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:41902/0 (socket says 10.20.193.16:41902) 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.348+0000 7f4f94f05640 1 -- 10.20.193.16:0/3554449412 learned_addr learned my addr 10.20.193.16:0/3554449412 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.348+0000 7f4f95706640 10 monclient(hunting): get_auth_request con 0x7f4f90153e00 auth_method 0 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.348+0000 7f4f95706640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:57.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.348+0000 7f4f95706640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.348+0000 7f4f95706640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.349+0000 7f4f95706640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.349+0000 7f4f95706640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.349+0000 7f4f95706640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f877fe640 1 -- 10.20.193.16:0/3554449412 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3519492175 0 0) 0x7f4f900c4b00 con 0x7f4f900700f0 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f877fe640 10 monclient(hunting): _negotiate 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f877fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f877fe640 10 monclient(hunting): my global_id is 4760 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f877fe640 1 -- 10.20.193.16:0/3554449412 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f7c003620 con 0x7f4f900700f0 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f95706640 10 monclient(hunting): handle_auth_done global_id 4687 payload 274 2026-02-17T05:57:57.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f95706640 1 -- 10.20.193.16:0/3554449412 >> v1:10.20.193.148:6789/0 conn(0x7f4f900700f0 legacy=0x7f4f900c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:57.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f95706640 1 -- 10.20.193.16:0/3554449412 >> v1:10.20.193.128:6789/0 conn(0x7f4f900c86f0 legacy=0x7f4f901536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:57.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f95706640 10 monclient: _finish_hunting 0 2026-02-17T05:57:57.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f95706640 1 monclient: found mon.noname-b 2026-02-17T05:57:57.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f95706640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:57.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f95706640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:57:57.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.350+0000 7f4f95706640 1 -- 10.20.193.16:0/3554449412 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f900c50c0 con 0x7f4f90153e00 2026-02-17T05:57:57.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.351+0000 7f4f95706640 1 --2- 10.20.193.16:0/3554449412 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f90153e00 0x7f4f900704d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009d90 tx=0x7f4f8c02f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5374fa44ff2057 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:57.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 1 -- 10.20.193.16:0/3554449412 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c03c070 con 0x7f4f90153e00 2026-02-17T05:57:57.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:57.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:57:57.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 10 monclient: dump: 2026-02-17T05:57:57.362 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:57.362 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:57.362 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:57.362 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:57.362 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:57.362 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 1 -- 10.20.193.16:0/3554449412 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f8c037440 con 0x7f4f90153e00 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 5 monclient: authenticate success, global_id 4687 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:57.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 1 -- 10.20.193.16:0/3554449412 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c037810 con 0x7f4f90153e00 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 4 set_mon_vals no callback set 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f877fe640 10 monclient: dump: 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:57.364 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 10 monclient: shutdown 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 1 -- 10.20.193.16:0/3554449412 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f90153e00 msgr2=0x7f4f900704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 1 --2- 10.20.193.16:0/3554449412 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f90153e00 0x7f4f900704d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009d90 tx=0x7f4f8c02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 1 -- 10.20.193.16:0/3554449412 shutdown_connections 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 1 --2- 10.20.193.16:0/3554449412 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f90153e00 0x7f4f900704d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 1 -- 10.20.193.16:0/3554449412 >> 10.20.193.16:0/3554449412 conn(0x7f4f900cd770 msgr2=0x7f4f900cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 1 -- 10.20.193.16:0/3554449412 shutdown_connections 2026-02-17T05:57:57.365 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.353+0000 7f4f97190640 1 -- 10.20.193.16:0/3554449412 wait complete. 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient: build_initial_monmap 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient: monmap: 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:57:57.366 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 librados: starting msgr at 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 librados: starting objecter 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 Processor -- start 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 -- start start 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 librados: setting wanted keys 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 librados: calling monclient init 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient: init 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f900700f0 0x7f4f901eac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient(hunting): picked mon.noname-a con 0x7f4f900700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:57:57.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f900c86f0 0x7f4f901f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient(hunting): picked mon.noname-c con 0x7f4f900c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f90153e00 0x7f4f901f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient(hunting): picked mon.noname-b con 0x7f4f90153e00 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90074c00 con 0x7f4f900700f0 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f4f900c5680 con 0x7f4f90153e00 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90074d80 con 0x7f4f900c86f0 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.354+0000 7f4f97190640 10 monclient(hunting): authenticate will time out at 1129.923980s 2026-02-17T05:57:57.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f94f05640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f900700f0 0x7f4f901eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f87fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f900c86f0 0x7f4f901f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f95706640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f90153e00 0x7f4f901f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f87fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f900c86f0 0x7f4f901f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:42222/0 (socket says 10.20.193.16:42222) 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f87fff640 1 -- 10.20.193.16:0/2390025914 learned_addr learned my addr 10.20.193.16:0/2390025914 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f87fff640 10 monclient(hunting): get_auth_request con 0x7f4f900c86f0 auth_method 0 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f87fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f87fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f94f05640 10 monclient(hunting): get_auth_request con 0x7f4f900700f0 auth_method 0 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f94f05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f94f05640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:57.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f94f05640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f95706640 10 monclient(hunting): get_auth_request con 0x7f4f90153e00 auth_method 0 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f95706640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f95706640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f95706640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f95706640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f95706640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f95706640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f94f05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f94f05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:57.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.355+0000 7f4f94f05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.356+0000 7f4f95706640 10 monclient(hunting): handle_auth_done global_id 4690 payload 824 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.356+0000 7f4f95706640 1 -- 10.20.193.16:0/2390025914 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f900c86f0 msgr2=0x7f4f901f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.356+0000 7f4f95706640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f900c86f0 0x7f4f901f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.356+0000 7f4f95706640 1 -- 10.20.193.16:0/2390025914 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f900700f0 msgr2=0x7f4f901eac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.356+0000 7f4f95706640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f900700f0 0x7f4f901eac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.356+0000 7f4f95706640 10 monclient: _finish_hunting 0 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.356+0000 7f4f95706640 1 monclient: found mon.noname-b 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.356+0000 7f4f95706640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.356+0000 7f4f95706640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.356+0000 7f4f95706640 1 -- 10.20.193.16:0/2390025914 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f90073a60 con 0x7f4f90153e00 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.357+0000 7f4f95706640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f90153e00 0x7f4f901f5cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c036000 tx=0x7f4f8c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:57.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.358+0000 7f4f857fa640 10 client.?.objecter ms_handle_connect 0x7f4f90153e00 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.358+0000 7f4f857fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 1 -- 10.20.193.16:0/2390025914 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c047020 con 0x7f4f90153e00 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 10 monclient: dump: 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:57.372 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 10 monclient: _finish_auth 0 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 10 monclient: _check_auth_tickets 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 1 -- 10.20.193.16:0/2390025914 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f8c03d040 con 0x7f4f90153e00 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 5 monclient: authenticate success, global_id 4690 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 10 monclient: _renew_subs 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 1 -- 10.20.193.16:0/2390025914 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f90073d40 con 0x7f4f90153e00 2026-02-17T05:57:57.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 1 -- 10.20.193.16:0/2390025914 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c03c050 con 0x7f4f90153e00 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f86ffd640 4 set_mon_vals no callback set 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f857fa640 10 monclient: dump: 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:57:57.374 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 10 client.4690.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 10 monclient: _renew_subs 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 1 -- 10.20.193.16:0/2390025914 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f901f4f00 con 0x7f4f90153e00 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 1 librados: init done 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:57.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.359+0000 7f4f97190640 1 -- 10.20.193.16:0/2390025914 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f900c5d00 con 0x7f4f90153e00 2026-02-17T05:57:57.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.465+0000 7f4f857fa640 1 -- 10.20.193.16:0/2390025914 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f4f8c041a30 con 0x7f4f90153e00 2026-02-17T05:57:57.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.465+0000 7f4f857fa640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f60079140 0x7f4f6007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:57.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.465+0000 7f4f94f05640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f60079140 0x7f4f6007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:57.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.465+0000 7f4f94f05640 10 monclient: get_auth_request con 0x7f4f60079140 auth_method 0 2026-02-17T05:57:57.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.466+0000 7f4f94f05640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f60079140 0x7f4f6007b600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4f80007b60 tx=0x7f4f80007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:57.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.466+0000 7f4f857fa640 10 client.4690.objecter ms_handle_connect 0x7f4f60079140 2026-02-17T05:57:57.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.467+0000 7f4f857fa640 1 -- 10.20.193.16:0/2390025914 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f4f8c041d50 con 0x7f4f90153e00 2026-02-17T05:57:57.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.467+0000 7f4f857fa640 10 client.4690.objecter ms_dispatch2 0x7f4f90000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:57:57.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.467+0000 7f4f857fa640 3 client.4690.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:57:57.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.467+0000 7f4f857fa640 3 client.4690.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:57:57.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.467+0000 7f4f857fa640 20 client.4690.objecter dump_active .. 0 homeless 2026-02-17T05:57:57.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.473+0000 7f4f857fa640 1 -- 10.20.193.16:0/2390025914 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4f900c5d00 con 0x7f4f90153e00 2026-02-17T05:57:57.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.473+0000 7f4f857fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:57.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.473+0000 7f4f857fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:57.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.577+0000 7f4f97190640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:57:57.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.577+0000 7f4f97190640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:57.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.577+0000 7f4f97190640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:57:57.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.577+0000 7f4f97190640 1 -- 10.20.193.16:0/2390025914 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4f90150d10 con 0x7f4f90153e00 2026-02-17T05:57:57.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.580+0000 7f4f857fa640 1 -- 10.20.193.16:0/2390025914 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f4f8c087230 con 0x7f4f90153e00 2026-02-17T05:57:57.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.580+0000 7f4f857fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:57:57.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.580+0000 7f4f857fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:57.581 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:57:57.581 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:57:57.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 10 librados: watch_flush enter 2026-02-17T05:57:57.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 10 librados: watch_flush exit 2026-02-17T05:57:57.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 20 client.4690.objecter shutdown clearing up homeless session... 2026-02-17T05:57:57.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 10 client.4690.objecter successfully canceled tick 2026-02-17T05:57:57.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 -- 10.20.193.16:0/2390025914 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f60079140 msgr2=0x7f4f6007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:57.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f60079140 0x7f4f6007b600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4f80007b60 tx=0x7f4f80007ab0 comp rx=0 tx=0).stop 2026-02-17T05:57:57.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 10 monclient: shutdown 2026-02-17T05:57:57.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:57.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 -- 10.20.193.16:0/2390025914 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f90153e00 msgr2=0x7f4f901f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:57.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f90153e00 0x7f4f901f5cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c036000 tx=0x7f4f8c004b40 comp rx=0 tx=0).stop 2026-02-17T05:57:57.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 -- 10.20.193.16:0/2390025914 shutdown_connections 2026-02-17T05:57:57.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4f60079140 0x7f4f6007b600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:57.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4f90153e00 0x7f4f901f5cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:57.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4f900c86f0 0x7f4f901f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:57.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 --2- 10.20.193.16:0/2390025914 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4f900700f0 0x7f4f901eac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:57.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 -- 10.20.193.16:0/2390025914 >> 10.20.193.16:0/2390025914 conn(0x7f4f900cd770 msgr2=0x7f4f9014dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:57.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 -- 10.20.193.16:0/2390025914 shutdown_connections 2026-02-17T05:57:57.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 -- 10.20.193.16:0/2390025914 wait complete. 2026-02-17T05:57:57.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:57:57.581+0000 7f4f97190640 1 librados: shutdown 2026-02-17T05:58:00.590 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:00.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.659+0000 7f807d3ce640 10 monclient: get_monmap_and_config 2026-02-17T05:58:00.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.659+0000 7f807d3ce640 10 monclient: build_initial_monmap 2026-02-17T05:58:00.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.659+0000 7f807d3ce640 10 monclient: monmap: 2026-02-17T05:58:00.660 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:00.660 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:00.660295+0000 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:00.660295+0000 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:00.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.659+0000 7f807d3ce640 1 Processor -- start 2026-02-17T05:58:00.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.659+0000 7f807d3ce640 1 -- start start 2026-02-17T05:58:00.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.659+0000 7f807d3ce640 10 monclient: init 2026-02-17T05:58:00.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:00.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 10 monclient: _add_conns ranks=[4,0,5,3,2,1] 2026-02-17T05:58:00.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f80780700f0 0x7f80780704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 10 monclient(hunting): picked mon.noname-b con 0x7f80780700f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f80780c7fe0 addr v1:10.20.193.16:6789/0 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f80781536f0 0x7f8078153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 10 monclient(hunting): picked mon.noname-c con 0x7f80781536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80780c4820 con 0x7f80780c7fe0 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f80780c4b00 con 0x7f80780700f0 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f80780c4de0 con 0x7f80781536f0 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f807d3ce640 10 monclient(hunting): authenticate will time out at 1133.229266s 2026-02-17T05:58:00.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f80767fc640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f80780c7fe0 0x7f80780c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:51038/0 (socket says 10.20.193.16:51038) 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f80767fc640 1 -- 10.20.193.16:0/1414807633 learned_addr learned my addr 10.20.193.16:0/1414807633 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f8076ffd640 1 --2- 10.20.193.16:0/1414807633 >> v2:10.20.193.128:3300/0 conn(0x7f80780700f0 0x7f80780704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f80777fe640 1 --2- 10.20.193.16:0/1414807633 >> v2:10.20.193.148:3300/0 conn(0x7f80781536f0 0x7f8078153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f8076ffd640 10 monclient(hunting): get_auth_request con 0x7f80780700f0 auth_method 0 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f8076ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f8076ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.660+0000 7f8076ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f80777fe640 10 monclient(hunting): get_auth_request con 0x7f80781536f0 auth_method 0 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f80777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f80777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f80777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:00.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f8076ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f8076ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f8076ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f80777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f80777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f80777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f8075ffb640 1 -- 10.20.193.16:0/1414807633 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2301717277 0 0) 0x7f80780c4820 con 0x7f80780c7fe0 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f8075ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f8075ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f8075ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f8075ffb640 10 monclient(hunting): my global_id is 4641 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.661+0000 7f8075ffb640 1 -- 10.20.193.16:0/1414807633 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8064003620 con 0x7f80780c7fe0 2026-02-17T05:58:00.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.662+0000 7f8076ffd640 10 monclient(hunting): handle_auth_done global_id 4693 payload 274 2026-02-17T05:58:00.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.662+0000 7f8076ffd640 1 -- 10.20.193.16:0/1414807633 >> v2:10.20.193.148:3300/0 conn(0x7f80781536f0 msgr2=0x7f8078153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:00.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.662+0000 7f8076ffd640 1 --2- 10.20.193.16:0/1414807633 >> v2:10.20.193.148:3300/0 conn(0x7f80781536f0 0x7f8078153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:00.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.662+0000 7f8076ffd640 1 -- 10.20.193.16:0/1414807633 >> v1:10.20.193.16:6789/0 conn(0x7f80780c7fe0 legacy=0x7f80780c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:00.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.662+0000 7f8076ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:58:00.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.662+0000 7f8076ffd640 1 monclient: found mon.noname-b 2026-02-17T05:58:00.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.662+0000 7f8076ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:00.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.662+0000 7f8076ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:58:00.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.662+0000 7f8076ffd640 1 -- 10.20.193.16:0/1414807633 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80780c50c0 con 0x7f80780700f0 2026-02-17T05:58:00.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.663+0000 7f8076ffd640 1 --2- 10.20.193.16:0/1414807633 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f80780700f0 0x7f80780704d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f80780c9540 tx=0x7f806c02ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e30d5dd9795ea9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:00.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 1 -- 10.20.193.16:0/1414807633 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c03c070 con 0x7f80780700f0 2026-02-17T05:58:00.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:00.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:00.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 10 monclient: dump: 2026-02-17T05:58:00.675 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:00.675 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 10 monclient: _finish_auth 0 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:00.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 1 -- 10.20.193.16:0/1414807633 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f806c02fa60 con 0x7f80780700f0 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 1 -- 10.20.193.16:0/1414807633 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c02fe30 con 0x7f80780700f0 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f807d3ce640 5 monclient: authenticate success, global_id 4693 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f807d3ce640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f807d3ce640 4 set_mon_vals no callback set 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.664+0000 7f8075ffb640 10 monclient: dump: 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:00.677 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 10 monclient: shutdown 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 -- 10.20.193.16:0/1414807633 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f80780700f0 msgr2=0x7f80780704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 --2- 10.20.193.16:0/1414807633 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f80780700f0 0x7f80780704d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f80780c9540 tx=0x7f806c02ed50 comp rx=0 tx=0).stop 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 -- 10.20.193.16:0/1414807633 shutdown_connections 2026-02-17T05:58:00.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 --2- 10.20.193.16:0/1414807633 >> v2:10.20.193.148:3300/0 conn(0x7f80781536f0 0x7f8078153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 --2- 10.20.193.16:0/1414807633 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f80780700f0 0x7f80780704d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 -- 10.20.193.16:0/1414807633 >> 10.20.193.16:0/1414807633 conn(0x7f80780cd770 msgr2=0x7f80780cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 -- 10.20.193.16:0/1414807633 shutdown_connections 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 -- 10.20.193.16:0/1414807633 wait complete. 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 10 monclient: build_initial_monmap 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 10 monclient: monmap: 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:00.679 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 librados: starting msgr at 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 librados: starting objecter 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.665+0000 7f807d3ce640 1 Processor -- start 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 1 -- start start 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 1 librados: setting wanted keys 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 1 librados: calling monclient init 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient: init 2026-02-17T05:58:00.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f80780700f0 0x7f80781e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient(hunting): picked mon.noname-a con 0x7f80780700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f80780c7fe0 0x7f80781f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient(hunting): picked mon.noname-c con 0x7f80780c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f80781536f0 0x7f80780cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient(hunting): picked mon.noname-b con 0x7f80781536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8078154060 con 0x7f80780700f0 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f80780c5680 con 0x7f80781536f0 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:00.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8078074c00 con 0x7f80780c7fe0 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f807d3ce640 10 monclient(hunting): authenticate will time out at 1133.235415s 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f8076ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f80780700f0 0x7f80781e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f80767fc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f80780c7fe0 0x7f80781f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f80777fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f80781536f0 0x7f80780cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f8076ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f80780700f0 0x7f80781e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44924/0 (socket says 10.20.193.16:44924) 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f8076ffd640 1 -- 10.20.193.16:0/4231995972 learned_addr learned my addr 10.20.193.16:0/4231995972 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f8076ffd640 10 monclient(hunting): get_auth_request con 0x7f80780700f0 auth_method 0 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f8076ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f8076ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:00.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f8076ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f80777fe640 10 monclient(hunting): get_auth_request con 0x7f80781536f0 auth_method 0 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f80777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f80777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f80777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f80767fc640 10 monclient(hunting): get_auth_request con 0x7f80780c7fe0 auth_method 0 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f80767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f80767fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.666+0000 7f80767fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.667+0000 7f80777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.667+0000 7f80777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.667+0000 7f80777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.667+0000 7f80767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:00.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.667+0000 7f80767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.667+0000 7f80767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.667+0000 7f8076ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.667+0000 7f8076ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.667+0000 7f8076ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.668+0000 7f80767fc640 10 monclient(hunting): handle_auth_done global_id 4769 payload 824 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.668+0000 7f80767fc640 1 -- 10.20.193.16:0/4231995972 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f80781536f0 msgr2=0x7f80780cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.668+0000 7f80767fc640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f80781536f0 0x7f80780cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.668+0000 7f80767fc640 1 -- 10.20.193.16:0/4231995972 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f80780700f0 msgr2=0x7f80781e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.668+0000 7f80767fc640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f80780700f0 0x7f80781e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.668+0000 7f80767fc640 10 monclient: _finish_hunting 0 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.668+0000 7f80767fc640 1 monclient: found mon.noname-c 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.668+0000 7f80767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:00.684 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.668+0000 7f80767fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.668+0000 7f80767fc640 1 -- 10.20.193.16:0/4231995972 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80781f73b0 con 0x7f80780c7fe0 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.669+0000 7f80767fc640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f80780c7fe0 0x7f80781f6a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f806000c480 tx=0x7f806000c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.669+0000 7f80577fe640 10 client.?.objecter ms_handle_connect 0x7f80780c7fe0 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.669+0000 7f80577fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 1 -- 10.20.193.16:0/4231995972 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060017070 con 0x7f80780c7fe0 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 10 monclient: dump: 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:00.685 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 5 monclient: authenticate success, global_id 4769 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 1 -- 10.20.193.16:0/4231995972 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8060005de0 con 0x7f80780c7fe0 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 1 -- 10.20.193.16:0/4231995972 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060007270 con 0x7f80780c7fe0 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 10 monclient: _renew_subs 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:00.686 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 1 -- 10.20.193.16:0/4231995972 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80781f7690 con 0x7f80780c7fe0 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80757fa640 4 set_mon_vals no callback set 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f80577fe640 10 monclient: dump: 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:00.687 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:00.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 10 client.4769.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:00.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 10 monclient: _renew_subs 2026-02-17T05:58:00.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:00.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 1 -- 10.20.193.16:0/4231995972 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80781f39a0 con 0x7f80780c7fe0 2026-02-17T05:58:00.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 1 librados: init done 2026-02-17T05:58:00.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:00.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:00.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:00.688 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.671+0000 7f807d3ce640 1 -- 10.20.193.16:0/4231995972 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80780704d0 con 0x7f80780c7fe0 2026-02-17T05:58:00.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.682+0000 7f80577fe640 1 -- 10.20.193.16:0/4231995972 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f8060007410 con 0x7f80780c7fe0 2026-02-17T05:58:00.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.683+0000 7f80577fe640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8044079190 0x7f804407b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:00.689 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.683+0000 7f8076ffd640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8044079190 0x7f804407b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:00.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.683+0000 7f8076ffd640 10 monclient: get_auth_request con 0x7f8044079190 auth_method 0 2026-02-17T05:58:00.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.683+0000 7f8076ffd640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8044079190 0x7f804407b650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f806c02ed90 tx=0x7f806c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:00.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.683+0000 7f80577fe640 10 client.4769.objecter ms_handle_connect 0x7f8044079190 2026-02-17T05:58:00.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.683+0000 7f80577fe640 1 -- 10.20.193.16:0/4231995972 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f8060007730 con 0x7f80780c7fe0 2026-02-17T05:58:00.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.683+0000 7f80577fe640 10 client.4769.objecter ms_dispatch2 0x7f8078000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:00.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.683+0000 7f80577fe640 3 client.4769.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:00.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.683+0000 7f80577fe640 3 client.4769.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:00.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.684+0000 7f80577fe640 20 client.4769.objecter dump_active .. 0 homeless 2026-02-17T05:58:00.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.785+0000 7f80577fe640 1 -- 10.20.193.16:0/4231995972 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8060022080 con 0x7f80780c7fe0 2026-02-17T05:58:00.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.785+0000 7f80577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:00.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.785+0000 7f80577fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:00.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.886+0000 7f807d3ce640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:00.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.886+0000 7f807d3ce640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:00.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.886+0000 7f807d3ce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:00.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.886+0000 7f807d3ce640 1 -- 10.20.193.16:0/4231995972 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f80781573f0 con 0x7f80780c7fe0 2026-02-17T05:58:00.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.888+0000 7f80577fe640 1 -- 10.20.193.16:0/4231995972 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8060061440 con 0x7f80780c7fe0 2026-02-17T05:58:00.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.888+0000 7f80577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:00.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.888+0000 7f80577fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:00.889 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:00.889 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 10 librados: watch_flush enter 2026-02-17T05:58:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 10 librados: watch_flush exit 2026-02-17T05:58:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 20 client.4769.objecter shutdown clearing up homeless session... 2026-02-17T05:58:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 10 client.4769.objecter successfully canceled tick 2026-02-17T05:58:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 -- 10.20.193.16:0/4231995972 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8044079190 msgr2=0x7f804407b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8044079190 0x7f804407b650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f806c02ed90 tx=0x7f806c033000 comp rx=0 tx=0).stop 2026-02-17T05:58:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 10 monclient: shutdown 2026-02-17T05:58:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 -- 10.20.193.16:0/4231995972 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f80780c7fe0 msgr2=0x7f80781f6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f80780c7fe0 0x7f80781f6a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f806000c480 tx=0x7f806000c950 comp rx=0 tx=0).stop 2026-02-17T05:58:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 -- 10.20.193.16:0/4231995972 shutdown_connections 2026-02-17T05:58:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8044079190 0x7f804407b650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f80781536f0 0x7f80780cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f80780c7fe0 0x7f80781f6a20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 --2- 10.20.193.16:0/4231995972 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f80780700f0 0x7f80781e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 -- 10.20.193.16:0/4231995972 >> 10.20.193.16:0/4231995972 conn(0x7f80780cd770 msgr2=0x7f807814f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 -- 10.20.193.16:0/4231995972 shutdown_connections 2026-02-17T05:58:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 -- 10.20.193.16:0/4231995972 wait complete. 2026-02-17T05:58:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:00.890+0000 7f807d3ce640 1 librados: shutdown 2026-02-17T05:58:03.837 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:03:42.664 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:03.865 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:03:42.690 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:03.866 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:03:42.693 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:03.867 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:03:42.692 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:03.874 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:03:42.700 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:03.875 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:03:42.703 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:03.898 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:03:42.687 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:03.899 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:03:42.688 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:03.901 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:03.908 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:03:42.697 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:03.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.969+0000 7f33750fa640 10 monclient: get_monmap_and_config 2026-02-17T05:58:03.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.969+0000 7f33750fa640 10 monclient: build_initial_monmap 2026-02-17T05:58:03.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.969+0000 7f33750fa640 10 monclient: monmap: 2026-02-17T05:58:03.970 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:03.971 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:03.971 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:03.970945+0000 2026-02-17T05:58:03.971 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:03.970945+0000 2026-02-17T05:58:03.971 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:03.971 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:03.971 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:03.971 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:03.971 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:03.971 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:03.972 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:03.972 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:03.972 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:03.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.970+0000 7f33750fa640 1 Processor -- start 2026-02-17T05:58:03.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.970+0000 7f33750fa640 1 -- start start 2026-02-17T05:58:03.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.970+0000 7f33750fa640 10 monclient: init 2026-02-17T05:58:03.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.970+0000 7f33750fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:03.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.970+0000 7f33750fa640 10 monclient: _add_conns ranks=[4,3,2,0,5,1] 2026-02-17T05:58:03.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f33700700f0 0x7f33700704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 10 monclient(hunting): picked mon.noname-b con 0x7f33700700f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f33700c7fe0 0x7f33700c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 10 monclient(hunting): picked mon.noname-a con 0x7f33700c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f33701536f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33700c8970 con 0x7f33701536f0 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f33700c4820 con 0x7f33700c7fe0 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f33700c4b00 con 0x7f33700700f0 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:03.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f33750fa640 10 monclient(hunting): authenticate will time out at 1136.540041s 2026-02-17T05:58:03.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336e575640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f33700c7fe0 0x7f33700c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336ed76640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f33700700f0 0x7f33700704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336e575640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f33700c7fe0 0x7f33700c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44928/0 (socket says 10.20.193.16:44928) 2026-02-17T05:58:03.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336e575640 1 -- 10.20.193.16:0/2697054386 learned_addr learned my addr 10.20.193.16:0/2697054386 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:03.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336e575640 10 monclient(hunting): get_auth_request con 0x7f33700c7fe0 auth_method 0 2026-02-17T05:58:03.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336ed76640 10 monclient(hunting): get_auth_request con 0x7f33700700f0 auth_method 0 2026-02-17T05:58:03.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.971+0000 7f336e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336e575640 10 monclient(hunting): handle_auth_done global_id 4647 payload 274 2026-02-17T05:58:03.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336e575640 1 -- 10.20.193.16:0/2697054386 >> v2:10.20.193.128:3300/0 conn(0x7f33700700f0 msgr2=0x7f33700704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:03.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336e575640 1 --2- 10.20.193.16:0/2697054386 >> v2:10.20.193.128:3300/0 conn(0x7f33700700f0 0x7f33700704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336e575640 1 -- 10.20.193.16:0/2697054386 >> v1:10.20.193.148:6789/0 conn(0x7f33701536f0 legacy=0x7f3370153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:03.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336e575640 10 monclient: _finish_hunting 0 2026-02-17T05:58:03.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336e575640 1 monclient: found mon.noname-a 2026-02-17T05:58:03.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:03.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336e575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:03.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.972+0000 7f336e575640 1 -- 10.20.193.16:0/2697054386 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33700c4de0 con 0x7f33700c7fe0 2026-02-17T05:58:03.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.973+0000 7f336e575640 1 --2- 10.20.193.16:0/2697054386 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 0x7f33700c8430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3370070b90 tx=0x7f335802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a1f1fd21a5b514 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 1 -- 10.20.193.16:0/2697054386 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335803c070 con 0x7f33700c7fe0 2026-02-17T05:58:03.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:03.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 10 monclient: dump: 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:03.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 10 monclient: _finish_auth 0 2026-02-17T05:58:03.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 10 monclient: _check_auth_tickets 2026-02-17T05:58:03.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:03.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 1 -- 10.20.193.16:0/2697054386 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335802fa90 con 0x7f33700c7fe0 2026-02-17T05:58:03.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f33750fa640 5 monclient: authenticate success, global_id 4647 2026-02-17T05:58:03.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f33750fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:03.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:03.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 1 -- 10.20.193.16:0/2697054386 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335802fe60 con 0x7f33700c7fe0 2026-02-17T05:58:03.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:03.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f336dd74640 10 monclient: dump: 2026-02-17T05:58:03.989 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:03.989 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:03.989 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:03.989 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:03.989 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.989 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:03.989 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:03.989 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:03.989 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f33750fa640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.975+0000 7f33750fa640 4 set_mon_vals no callback set 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 10 monclient: shutdown 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 -- 10.20.193.16:0/2697054386 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 msgr2=0x7f33700c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 --2- 10.20.193.16:0/2697054386 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 0x7f33700c8430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3370070b90 tx=0x7f335802ee60 comp rx=0 tx=0).stop 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 -- 10.20.193.16:0/2697054386 shutdown_connections 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 --2- 10.20.193.16:0/2697054386 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 0x7f33700c8430 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 --2- 10.20.193.16:0/2697054386 >> v2:10.20.193.128:3300/0 conn(0x7f33700700f0 0x7f33700704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 -- 10.20.193.16:0/2697054386 >> 10.20.193.16:0/2697054386 conn(0x7f33700cd770 msgr2=0x7f33700cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:03.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 -- 10.20.193.16:0/2697054386 shutdown_connections 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 -- 10.20.193.16:0/2697054386 wait complete. 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 10 monclient: build_initial_monmap 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 10 monclient: monmap: 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:03.991 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 librados: starting msgr at 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 librados: starting objecter 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 Processor -- start 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 -- start start 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 librados: setting wanted keys 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 1 librados: calling monclient init 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.976+0000 7f33750fa640 10 monclient: init 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f33700700f0 0x7f33701529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 10 monclient(hunting): picked mon.noname-b con 0x7f33700700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 0x7f3370155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 10 monclient(hunting): picked mon.noname-a con 0x7f33700c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33701536f0 0x7f3370155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 10 monclient(hunting): picked mon.noname-c con 0x7f33701536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33700c5680 con 0x7f33700c7fe0 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f33700c53a0 con 0x7f33700700f0 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3370074c00 con 0x7f33701536f0 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f33750fa640 10 monclient(hunting): authenticate will time out at 1136.546311s 2026-02-17T05:58:03.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336e575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 0x7f3370155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336ed76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f33700700f0 0x7f33701529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336e575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 0x7f3370155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44938/0 (socket says 10.20.193.16:44938) 2026-02-17T05:58:03.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336e575640 1 -- 10.20.193.16:0/3865141732 learned_addr learned my addr 10.20.193.16:0/3865141732 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:03.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336e575640 10 monclient(hunting): get_auth_request con 0x7f33700c7fe0 auth_method 0 2026-02-17T05:58:03.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336ed76640 10 monclient(hunting): get_auth_request con 0x7f33700700f0 auth_method 0 2026-02-17T05:58:03.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.977+0000 7f336e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.978+0000 7f336ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.978+0000 7f336ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.978+0000 7f336ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.978+0000 7f336f577640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33701536f0 0x7f3370155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.979+0000 7f336e575640 10 monclient(hunting): handle_auth_done global_id 4650 payload 824 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.979+0000 7f336e575640 1 -- 10.20.193.16:0/3865141732 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33701536f0 msgr2=0x7f3370155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.979+0000 7f336e575640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33701536f0 0x7f3370155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.979+0000 7f336e575640 1 -- 10.20.193.16:0/3865141732 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f33700700f0 msgr2=0x7f33701529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.979+0000 7f336e575640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f33700700f0 0x7f33701529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.979+0000 7f336e575640 10 monclient: _finish_hunting 0 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.979+0000 7f336e575640 1 monclient: found mon.noname-a 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.979+0000 7f336e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.979+0000 7f336e575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.979+0000 7f336e575640 1 -- 10.20.193.16:0/3865141732 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33701f1780 con 0x7f33700c7fe0 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.980+0000 7f336e575640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 0x7f3370155970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f335802f370 tx=0x7f33580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.980+0000 7f334f7fe640 10 client.?.objecter ms_handle_connect 0x7f33700c7fe0 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.980+0000 7f334f7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 1 -- 10.20.193.16:0/3865141732 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335803c040 con 0x7f33700c7fe0 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 10 monclient: dump: 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:03.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 1 -- 10.20.193.16:0/3865141732 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3358042410 con 0x7f33700c7fe0 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 5 monclient: authenticate success, global_id 4650 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 1 -- 10.20.193.16:0/3865141732 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358042800 con 0x7f33700c7fe0 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f336d573640 4 set_mon_vals no callback set 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f334f7fe640 10 monclient: dump: 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:03.998 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:03.999 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:03.999 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.999 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:03.999 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:03.999 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:03.999 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:03.999 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:03.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 10 monclient: _renew_subs 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 1 -- 10.20.193.16:0/3865141732 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33701f1a60 con 0x7f33700c7fe0 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 10 client.4650.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 10 monclient: _renew_subs 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 1 -- 10.20.193.16:0/3865141732 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33701f2020 con 0x7f33700c7fe0 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 1 librados: init done 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:04.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.982+0000 7f33750fa640 1 -- 10.20.193.16:0/3865141732 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33700704d0 con 0x7f33700c7fe0 2026-02-17T05:58:04.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.994+0000 7f334f7fe640 1 -- 10.20.193.16:0/3865141732 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f33580429a0 con 0x7f33700c7fe0 2026-02-17T05:58:04.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.994+0000 7f334f7fe640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f333c0791c0 0x7f333c07b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.994+0000 7f336ed76640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f333c0791c0 0x7f333c07b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.994+0000 7f336ed76640 10 monclient: get_auth_request con 0x7f333c0791c0 auth_method 0 2026-02-17T05:58:04.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.995+0000 7f334f7fe640 10 client.4650.objecter ms_handle_connect 0x7f333c0791c0 2026-02-17T05:58:04.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.995+0000 7f336ed76640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f333c0791c0 0x7f333c07b680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3364009ba0 tx=0x7f3364009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.996+0000 7f334f7fe640 1 -- 10.20.193.16:0/3865141732 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f3358083970 con 0x7f33700c7fe0 2026-02-17T05:58:04.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.996+0000 7f334f7fe640 10 client.4650.objecter ms_dispatch2 0x7f3370000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:04.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.996+0000 7f334f7fe640 3 client.4650.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:04.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.996+0000 7f334f7fe640 3 client.4650.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:04.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:03.997+0000 7f334f7fe640 20 client.4650.objecter dump_active .. 0 homeless 2026-02-17T05:58:04.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.101+0000 7f334f7fe640 1 -- 10.20.193.16:0/3865141732 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3358088030 con 0x7f33700c7fe0 2026-02-17T05:58:04.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.101+0000 7f334f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.101+0000 7f334f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:04.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.204+0000 7f33750fa640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:04.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.204+0000 7f33750fa640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:04.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.204+0000 7f33750fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:04.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.204+0000 7f33750fa640 1 -- 10.20.193.16:0/3865141732 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f33701573f0 con 0x7f33700c7fe0 2026-02-17T05:58:04.207 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.207+0000 7f334f7fe640 1 -- 10.20.193.16:0/3865141732 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3358087930 con 0x7f33700c7fe0 2026-02-17T05:58:04.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.207+0000 7f334f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:04.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.207+0000 7f334f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:04.208 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:04.208 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:04.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 10 librados: watch_flush enter 2026-02-17T05:58:04.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 10 librados: watch_flush exit 2026-02-17T05:58:04.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 20 client.4650.objecter shutdown clearing up homeless session... 2026-02-17T05:58:04.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 10 client.4650.objecter successfully canceled tick 2026-02-17T05:58:04.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 -- 10.20.193.16:0/3865141732 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f333c0791c0 msgr2=0x7f333c07b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f333c0791c0 0x7f333c07b680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3364009ba0 tx=0x7f3364009290 comp rx=0 tx=0).stop 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 10 monclient: shutdown 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 -- 10.20.193.16:0/3865141732 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 msgr2=0x7f3370155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 0x7f3370155970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f335802f370 tx=0x7f33580047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 -- 10.20.193.16:0/3865141732 shutdown_connections 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f333c0791c0 0x7f333c07b680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f33701536f0 0x7f3370155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f33700c7fe0 0x7f3370155970 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 --2- 10.20.193.16:0/3865141732 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f33700700f0 0x7f33701529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 -- 10.20.193.16:0/3865141732 >> 10.20.193.16:0/3865141732 conn(0x7f33700cd770 msgr2=0x7f337014eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.208+0000 7f33750fa640 1 -- 10.20.193.16:0/3865141732 shutdown_connections 2026-02-17T05:58:04.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.209+0000 7f33750fa640 1 -- 10.20.193.16:0/3865141732 wait complete. 2026-02-17T05:58:04.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:04.209+0000 7f33750fa640 1 librados: shutdown 2026-02-17T05:58:07.219 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:07.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.289+0000 7f0bbe51b640 10 monclient: get_monmap_and_config 2026-02-17T05:58:07.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.289+0000 7f0bbe51b640 10 monclient: build_initial_monmap 2026-02-17T05:58:07.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.289+0000 7f0bbe51b640 10 monclient: monmap: 2026-02-17T05:58:07.290 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:07.290 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:07.290 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:07.290048+0000 2026-02-17T05:58:07.290 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:07.290048+0000 2026-02-17T05:58:07.290 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:07.290 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:07.290 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:07.290 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:07.291 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:07.291 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:07.291 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:07.291 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:07.291 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:07.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.289+0000 7f0bbe51b640 1 Processor -- start 2026-02-17T05:58:07.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.289+0000 7f0bbe51b640 1 -- start start 2026-02-17T05:58:07.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.289+0000 7f0bbe51b640 10 monclient: init 2026-02-17T05:58:07.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:07.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 10 monclient: _add_conns ranks=[4,5,0,2,1,3] 2026-02-17T05:58:07.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f0bb814f020 0x7f0bb806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:07.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 10 monclient(hunting): picked mon.noname-b con 0x7f0bb814f020 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:07.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f0bb806e120 0x7f0bb815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:07.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 10 monclient(hunting): picked mon.noname-c con 0x7f0bb806e120 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:07.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0bb814f400 addr v1:10.20.193.16:6789/0 2026-02-17T05:58:07.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb80c5b10 con 0x7f0bb814f400 2026-02-17T05:58:07.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:07.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb806e620 con 0x7f0bb814f020 2026-02-17T05:58:07.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:07.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb806e900 con 0x7f0bb806e120 2026-02-17T05:58:07.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:07.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbe51b640 10 monclient(hunting): authenticate will time out at 1139.859176s 2026-02-17T05:58:07.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbca91640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f0bb814f400 0x7f0bb80c6010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:51048/0 (socket says 10.20.193.16:51048) 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bbca91640 1 -- 10.20.193.16:0/4210143799 learned_addr learned my addr 10.20.193.16:0/4210143799 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bb7fff640 1 --2- 10.20.193.16:0/4210143799 >> v2:10.20.193.128:3300/0 conn(0x7f0bb814f020 0x7f0bb806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.290+0000 7f0bb77fe640 1 --2- 10.20.193.16:0/4210143799 >> v2:10.20.193.148:3300/0 conn(0x7f0bb806e120 0x7f0bb815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb7fff640 10 monclient(hunting): get_auth_request con 0x7f0bb814f020 auth_method 0 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb77fe640 10 monclient(hunting): get_auth_request con 0x7f0bb806e120 auth_method 0 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:07.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:07.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:07.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:07.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:07.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:07.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:07.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:07.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb6ffd640 1 -- 10.20.193.16:0/4210143799 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685572745 0 0) 0x7f0bb80c5b10 con 0x7f0bb814f400 2026-02-17T05:58:07.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb6ffd640 10 monclient(hunting): _negotiate 2026-02-17T05:58:07.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:07.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:07.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.291+0000 7f0bb6ffd640 10 monclient(hunting): my global_id is 4653 2026-02-17T05:58:07.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.292+0000 7f0bb6ffd640 1 -- 10.20.193.16:0/4210143799 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b9c003620 con 0x7f0bb814f400 2026-02-17T05:58:07.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.292+0000 7f0bb7fff640 10 monclient(hunting): handle_auth_done global_id 4705 payload 274 2026-02-17T05:58:07.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.292+0000 7f0bb7fff640 1 -- 10.20.193.16:0/4210143799 >> v2:10.20.193.148:3300/0 conn(0x7f0bb806e120 msgr2=0x7f0bb815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:07.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.292+0000 7f0bb7fff640 1 --2- 10.20.193.16:0/4210143799 >> v2:10.20.193.148:3300/0 conn(0x7f0bb806e120 0x7f0bb815c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.292+0000 7f0bb7fff640 1 -- 10.20.193.16:0/4210143799 >> v1:10.20.193.16:6789/0 conn(0x7f0bb814f400 legacy=0x7f0bb80c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:07.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.292+0000 7f0bb7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:07.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.292+0000 7f0bb7fff640 1 monclient: found mon.noname-b 2026-02-17T05:58:07.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.292+0000 7f0bb7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:07.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.292+0000 7f0bb7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:58:07.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.292+0000 7f0bb7fff640 1 -- 10.20.193.16:0/4210143799 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb806ebe0 con 0x7f0bb814f020 2026-02-17T05:58:07.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.293+0000 7f0bb7fff640 1 --2- 10.20.193.16:0/4210143799 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bb814f020 0x7f0bb806dbe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8156400 tx=0x7f0ba402eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28e6e9687c309ef6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:07.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 1 -- 10.20.193.16:0/4210143799 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba403c070 con 0x7f0bb814f020 2026-02-17T05:58:07.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:07.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:07.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 10 monclient: dump: 2026-02-17T05:58:07.308 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:07.308 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:07.308 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:07.308 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:07.308 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:07.308 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:07.308 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:07.308 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:07.308 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 1 -- 10.20.193.16:0/4210143799 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba402fb80 con 0x7f0bb814f020 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bbe51b640 5 monclient: authenticate success, global_id 4705 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 1 -- 10.20.193.16:0/4210143799 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4037440 con 0x7f0bb814f020 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bbe51b640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bbe51b640 4 set_mon_vals no callback set 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:07.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.295+0000 7f0bb6ffd640 10 monclient: dump: 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:07.310 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 10 monclient: shutdown 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 1 -- 10.20.193.16:0/4210143799 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bb814f020 msgr2=0x7f0bb806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 1 --2- 10.20.193.16:0/4210143799 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bb814f020 0x7f0bb806dbe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8156400 tx=0x7f0ba402eeb0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 1 -- 10.20.193.16:0/4210143799 shutdown_connections 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 1 --2- 10.20.193.16:0/4210143799 >> v2:10.20.193.148:3300/0 conn(0x7f0bb806e120 0x7f0bb815c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 1 --2- 10.20.193.16:0/4210143799 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bb814f020 0x7f0bb806dbe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 1 -- 10.20.193.16:0/4210143799 >> 10.20.193.16:0/4210143799 conn(0x7f0bb80cd770 msgr2=0x7f0bb80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 1 -- 10.20.193.16:0/4210143799 shutdown_connections 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 1 -- 10.20.193.16:0/4210143799 wait complete. 2026-02-17T05:58:07.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 10 monclient: build_initial_monmap 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.296+0000 7f0bbe51b640 10 monclient: monmap: 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:07.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 librados: starting msgr at 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 librados: starting objecter 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 Processor -- start 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 -- start start 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 librados: setting wanted keys 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 librados: calling monclient init 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient: init 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bb806e120 0x7f0bb8142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient(hunting): picked mon.noname-a con 0x7f0bb806e120 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:07.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bb814f020 0x7f0bb81f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient(hunting): picked mon.noname-c con 0x7f0bb814f020 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bb814f400 0x7f0bb81ed110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient(hunting): picked mon.noname-b con 0x7f0bb814f400 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb80c6720 con 0x7f0bb806e120 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb806f1a0 con 0x7f0bb814f400 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb80c68a0 con 0x7f0bb814f020 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bbe51b640 10 monclient(hunting): authenticate will time out at 1139.866581s 2026-02-17T05:58:07.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bb7fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bb806e120 0x7f0bb8142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:07.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bb77fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bb814f020 0x7f0bb81f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:07.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bb7fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bb806e120 0x7f0bb8142bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44942/0 (socket says 10.20.193.16:44942) 2026-02-17T05:58:07.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bb7fff640 1 -- 10.20.193.16:0/3095767242 learned_addr learned my addr 10.20.193.16:0/3095767242 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:07.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bb7fff640 10 monclient(hunting): get_auth_request con 0x7f0bb806e120 auth_method 0 2026-02-17T05:58:07.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:07.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bb7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:07.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.297+0000 7f0bb7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:07.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bbca91640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bb814f400 0x7f0bb81ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:07.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bb77fe640 10 monclient(hunting): get_auth_request con 0x7f0bb814f020 auth_method 0 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bb77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bb77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bbca91640 10 monclient(hunting): get_auth_request con 0x7f0bb814f400 auth_method 0 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bbca91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bbca91640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bbca91640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:07.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bbca91640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bbca91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.298+0000 7f0bbca91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.299+0000 7f0bb7fff640 10 monclient(hunting): handle_auth_done global_id 4656 payload 824 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.299+0000 7f0bb7fff640 1 -- 10.20.193.16:0/3095767242 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bb814f020 msgr2=0x7f0bb81f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.299+0000 7f0bb7fff640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bb814f020 0x7f0bb81f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.299+0000 7f0bb7fff640 1 -- 10.20.193.16:0/3095767242 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bb814f400 msgr2=0x7f0bb81ed110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.299+0000 7f0bb7fff640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bb814f400 0x7f0bb81ed110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.299+0000 7f0bb7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.299+0000 7f0bb7fff640 1 monclient: found mon.noname-a 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.299+0000 7f0bb7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.299+0000 7f0bb7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.299+0000 7f0bb7fff640 1 -- 10.20.193.16:0/3095767242 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb81edc00 con 0x7f0bb806e120 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.300+0000 7f0bb7fff640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bb806e120 0x7f0bb8142bc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ba403a040 tx=0x7f0ba40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.300+0000 7f0bb4ff9640 10 client.?.objecter ms_handle_connect 0x7f0bb806e120 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.300+0000 7f0bb4ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 1 -- 10.20.193.16:0/3095767242 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4046020 con 0x7f0bb806e120 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:07.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 10 monclient: dump: 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:07.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 10 monclient: _finish_auth 0 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 1 -- 10.20.193.16:0/3095767242 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba4042430 con 0x7f0bb806e120 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bbe51b640 5 monclient: authenticate success, global_id 4656 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bbe51b640 10 monclient: _renew_subs 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bbe51b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bbe51b640 1 -- 10.20.193.16:0/3095767242 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb81edee0 con 0x7f0bb806e120 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 1 -- 10.20.193.16:0/3095767242 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba403c040 con 0x7f0bb806e120 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:07.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb4ff9640 10 monclient: dump: 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:07.321 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:07.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bb67fc640 4 set_mon_vals no callback set 2026-02-17T05:58:07.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bbe51b640 10 client.4656.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:07.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bbe51b640 10 monclient: _renew_subs 2026-02-17T05:58:07.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bbe51b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:07.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bbe51b640 1 -- 10.20.193.16:0/3095767242 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb8154760 con 0x7f0bb806e120 2026-02-17T05:58:07.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.302+0000 7f0bbe51b640 1 librados: init done 2026-02-17T05:58:07.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.303+0000 7f0bbe51b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:07.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.303+0000 7f0bbe51b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:07.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.303+0000 7f0bbe51b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:07.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.303+0000 7f0bbe51b640 1 -- 10.20.193.16:0/3095767242 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb806f510 con 0x7f0bb806e120 2026-02-17T05:58:07.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.314+0000 7f0bb4ff9640 1 -- 10.20.193.16:0/3095767242 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f0ba4053070 con 0x7f0bb806e120 2026-02-17T05:58:07.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.315+0000 7f0bb4ff9640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0b88079190 0x7f0b8807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:07.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.315+0000 7f0bb77fe640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0b88079190 0x7f0b8807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:07.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.315+0000 7f0bb77fe640 10 monclient: get_auth_request con 0x7f0b88079190 auth_method 0 2026-02-17T05:58:07.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.315+0000 7f0bb77fe640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0b88079190 0x7f0b8807b650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80c77a0 tx=0x7f0ba8009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:07.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.315+0000 7f0bb4ff9640 10 client.4656.objecter ms_handle_connect 0x7f0b88079190 2026-02-17T05:58:07.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.316+0000 7f0bb4ff9640 1 -- 10.20.193.16:0/3095767242 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f0ba40836e0 con 0x7f0bb806e120 2026-02-17T05:58:07.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.316+0000 7f0bb4ff9640 10 client.4656.objecter ms_dispatch2 0x7f0bb8000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:07.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.316+0000 7f0bb4ff9640 3 client.4656.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:07.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.316+0000 7f0bb4ff9640 3 client.4656.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:07.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.316+0000 7f0bb4ff9640 20 client.4656.objecter dump_active .. 0 homeless 2026-02-17T05:58:07.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.418+0000 7f0bb4ff9640 1 -- 10.20.193.16:0/3095767242 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0bb806f510 con 0x7f0bb806e120 2026-02-17T05:58:07.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.418+0000 7f0bb4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:07.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.418+0000 7f0bb4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:07.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.524+0000 7f0bbe51b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:07.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.524+0000 7f0bbe51b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:07.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.524+0000 7f0bbe51b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:07.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.524+0000 7f0bbe51b640 1 -- 10.20.193.16:0/3095767242 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0bb8154270 con 0x7f0bb806e120 2026-02-17T05:58:07.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.527+0000 7f0bb4ff9640 1 -- 10.20.193.16:0/3095767242 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f0ba4050020 con 0x7f0bb806e120 2026-02-17T05:58:07.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.527+0000 7f0bb4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:07.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.527+0000 7f0bb4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:07.527 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:07.527 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:07.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 10 librados: watch_flush enter 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 10 librados: watch_flush exit 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 20 client.4656.objecter shutdown clearing up homeless session... 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 10 client.4656.objecter successfully canceled tick 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 -- 10.20.193.16:0/3095767242 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0b88079190 msgr2=0x7f0b8807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0b88079190 0x7f0b8807b650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80c77a0 tx=0x7f0ba8009290 comp rx=0 tx=0).stop 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 10 monclient: shutdown 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 -- 10.20.193.16:0/3095767242 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bb806e120 msgr2=0x7f0bb8142bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bb806e120 0x7f0bb8142bc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ba403a040 tx=0x7f0ba40047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 -- 10.20.193.16:0/3095767242 shutdown_connections 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0b88079190 0x7f0b8807b650 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bb814f400 0x7f0bb81ed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bb814f020 0x7f0bb81f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 --2- 10.20.193.16:0/3095767242 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bb806e120 0x7f0bb8142bc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 -- 10.20.193.16:0/3095767242 >> 10.20.193.16:0/3095767242 conn(0x7f0bb80cd770 msgr2=0x7f0bb814b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 -- 10.20.193.16:0/3095767242 shutdown_connections 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 -- 10.20.193.16:0/3095767242 wait complete. 2026-02-17T05:58:07.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:07.528+0000 7f0bbe51b640 1 librados: shutdown 2026-02-17T05:58:10.538 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:10.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.608+0000 7f8b65171640 10 monclient: get_monmap_and_config 2026-02-17T05:58:10.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.608+0000 7f8b65171640 10 monclient: build_initial_monmap 2026-02-17T05:58:10.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.608+0000 7f8b65171640 10 monclient: monmap: 2026-02-17T05:58:10.609 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:10.609 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:10.609 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:10.609326+0000 2026-02-17T05:58:10.609 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:10.609326+0000 2026-02-17T05:58:10.609 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:10.609 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:10.610 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:10.610 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:10.610 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:10.610 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:10.610 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:10.610 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:10.610 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:10.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.608+0000 7f8b65171640 1 Processor -- start 2026-02-17T05:58:10.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.608+0000 7f8b65171640 1 -- start start 2026-02-17T05:58:10.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.608+0000 7f8b65171640 10 monclient: init 2026-02-17T05:58:10.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:10.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 10 monclient: _add_conns ranks=[1,5,4,0,2,3] 2026-02-17T05:58:10.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f8b600700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f8b600c86f0 0x7f8b600704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 10 monclient(hunting): picked mon.noname-c con 0x7f8b600c86f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f8b601536f0 0x7f8b60153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 10 monclient(hunting): picked mon.noname-b con 0x7f8b601536f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b600c4820 con 0x7f8b600700f0 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f8b600c4b00 con 0x7f8b601536f0 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f8b600c4de0 con 0x7f8b600c86f0 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b65171640 10 monclient(hunting): authenticate will time out at 1143.178280s 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b5e575640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f8b600c86f0 0x7f8b600704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b5f577640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f8b601536f0 0x7f8b60153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.609+0000 7f8b5e575640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f8b600c86f0 0x7f8b600704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:53970/0 (socket says 10.20.193.16:53970) 2026-02-17T05:58:10.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5e575640 1 -- 10.20.193.16:0/3174697638 learned_addr learned my addr 10.20.193.16:0/3174697638 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5e575640 10 monclient(hunting): get_auth_request con 0x7f8b600c86f0 auth_method 0 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5f577640 10 monclient(hunting): get_auth_request con 0x7f8b601536f0 auth_method 0 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5f577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5f577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5f577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5f577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5f577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.610+0000 7f8b5f577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:10.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.611+0000 7f8b5e575640 10 monclient(hunting): handle_auth_done global_id 4778 payload 274 2026-02-17T05:58:10.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.611+0000 7f8b5e575640 1 -- 10.20.193.16:0/3174697638 >> v2:10.20.193.128:3300/0 conn(0x7f8b601536f0 msgr2=0x7f8b60153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:10.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.611+0000 7f8b5e575640 1 --2- 10.20.193.16:0/3174697638 >> v2:10.20.193.128:3300/0 conn(0x7f8b601536f0 0x7f8b60153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:10.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.611+0000 7f8b5e575640 1 -- 10.20.193.16:0/3174697638 >> v1:10.20.193.128:6789/0 conn(0x7f8b600700f0 legacy=0x7f8b600c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:10.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.611+0000 7f8b5e575640 10 monclient: _finish_hunting 0 2026-02-17T05:58:10.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.611+0000 7f8b5e575640 1 monclient: found mon.noname-c 2026-02-17T05:58:10.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.611+0000 7f8b5e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:10.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.611+0000 7f8b5e575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:10.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.611+0000 7f8b5e575640 1 -- 10.20.193.16:0/3174697638 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b600c50c0 con 0x7f8b600c86f0 2026-02-17T05:58:10.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.612+0000 7f8b5e575640 1 --2- 10.20.193.16:0/3174697638 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600c86f0 0x7f8b600704d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b600c5b80 tx=0x7f8b4802ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269244a3ff686e45 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:10.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.613+0000 7f8b5dd74640 1 -- 10.20.193.16:0/3174697638 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4803c070 con 0x7f8b600c86f0 2026-02-17T05:58:10.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.613+0000 7f8b5dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:10.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.613+0000 7f8b5dd74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:10.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.613+0000 7f8b5dd74640 10 monclient: dump: 2026-02-17T05:58:10.619 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:10.619 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:10.619 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:10.619 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:10.619 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:10.619 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.613+0000 7f8b5dd74640 10 monclient: _finish_auth 0 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.613+0000 7f8b5dd74640 10 monclient: _check_auth_tickets 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.613+0000 7f8b5dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.613+0000 7f8b65171640 5 monclient: authenticate success, global_id 4778 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.613+0000 7f8b5dd74640 1 -- 10.20.193.16:0/3174697638 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4802fb30 con 0x7f8b600c86f0 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b5dd74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b5dd74640 1 -- 10.20.193.16:0/3174697638 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48037440 con 0x7f8b600c86f0 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 4 set_mon_vals no callback set 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b5dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b5dd74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b5dd74640 10 monclient: dump: 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:10.620 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 10 monclient: shutdown 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 1 -- 10.20.193.16:0/3174697638 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600c86f0 msgr2=0x7f8b600704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 1 --2- 10.20.193.16:0/3174697638 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600c86f0 0x7f8b600704d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b600c5b80 tx=0x7f8b4802ee60 comp rx=0 tx=0).stop 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 1 -- 10.20.193.16:0/3174697638 shutdown_connections 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 1 --2- 10.20.193.16:0/3174697638 >> v2:10.20.193.128:3300/0 conn(0x7f8b601536f0 0x7f8b60153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 1 --2- 10.20.193.16:0/3174697638 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600c86f0 0x7f8b600704d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 1 -- 10.20.193.16:0/3174697638 >> 10.20.193.16:0/3174697638 conn(0x7f8b600cd770 msgr2=0x7f8b600cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 1 -- 10.20.193.16:0/3174697638 shutdown_connections 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 1 -- 10.20.193.16:0/3174697638 wait complete. 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 10 monclient: build_initial_monmap 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.614+0000 7f8b65171640 10 monclient: monmap: 2026-02-17T05:58:10.621 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 librados: starting msgr at 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 librados: starting objecter 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 Processor -- start 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 -- start start 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 librados: setting wanted keys 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 librados: calling monclient init 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient: init 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:58:10.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600700f0 0x7f8b60141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient(hunting): picked mon.noname-c con 0x7f8b600700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8b600c86f0 0x7f8b601f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient(hunting): picked mon.noname-a con 0x7f8b600c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8b601536f0 0x7f8b60145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient(hunting): picked mon.noname-b con 0x7f8b601536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60154060 con 0x7f8b600c86f0 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f8b600c5680 con 0x7f8b601536f0 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60074c00 con 0x7f8b600700f0 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b65171640 10 monclient(hunting): authenticate will time out at 1143.184653s 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b5ed76640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600700f0 0x7f8b60141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.615+0000 7f8b5e575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8b600c86f0 0x7f8b601f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5f577640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8b601536f0 0x7f8b60145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5ed76640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600700f0 0x7f8b60141e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:53976/0 (socket says 10.20.193.16:53976) 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5ed76640 1 -- 10.20.193.16:0/641498754 learned_addr learned my addr 10.20.193.16:0/641498754 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5ed76640 10 monclient(hunting): get_auth_request con 0x7f8b600700f0 auth_method 0 2026-02-17T05:58:10.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5e575640 10 monclient(hunting): get_auth_request con 0x7f8b600c86f0 auth_method 0 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5f577640 10 monclient(hunting): get_auth_request con 0x7f8b601536f0 auth_method 0 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5f577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5f577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5f577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5f577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5f577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.616+0000 7f8b5f577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.617+0000 7f8b5ed76640 10 monclient(hunting): handle_auth_done global_id 4781 payload 824 2026-02-17T05:58:10.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.617+0000 7f8b5ed76640 1 -- 10.20.193.16:0/641498754 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8b601536f0 msgr2=0x7f8b60145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.617+0000 7f8b5ed76640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8b601536f0 0x7f8b60145e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.617+0000 7f8b5ed76640 1 -- 10.20.193.16:0/641498754 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8b600c86f0 msgr2=0x7f8b601f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.617+0000 7f8b5ed76640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8b600c86f0 0x7f8b601f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.617+0000 7f8b5ed76640 10 monclient: _finish_hunting 0 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.617+0000 7f8b5ed76640 1 monclient: found mon.noname-c 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.617+0000 7f8b5ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.617+0000 7f8b5ed76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.617+0000 7f8b5ed76640 1 -- 10.20.193.16:0/641498754 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b601423a0 con 0x7f8b600700f0 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.618+0000 7f8b5ed76640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600700f0 0x7f8b60141e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c480 tx=0x7f8b5400c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.618+0000 7f8b3f7fe640 10 client.?.objecter ms_handle_connect 0x7f8b600700f0 2026-02-17T05:58:10.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.618+0000 7f8b3f7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:10.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b3f7fe640 1 -- 10.20.193.16:0/641498754 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54017070 con 0x7f8b600700f0 2026-02-17T05:58:10.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:10.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b3f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:10.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b3f7fe640 10 monclient: dump: 2026-02-17T05:58:10.626 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:10.626 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:10.626 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:10.626 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b3f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b3f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b3f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b65171640 5 monclient: authenticate success, global_id 4781 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b65171640 10 monclient: _renew_subs 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b65171640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b65171640 1 -- 10.20.193.16:0/641498754 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b601f7780 con 0x7f8b600700f0 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b65171640 10 client.4781.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b65171640 10 monclient: _renew_subs 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b65171640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b65171640 1 -- 10.20.193.16:0/641498754 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b601f7d40 con 0x7f8b600700f0 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.619+0000 7f8b65171640 1 librados: init done 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b65171640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b65171640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b65171640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:10.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b65171640 1 -- 10.20.193.16:0/641498754 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b600704d0 con 0x7f8b600700f0 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b3f7fe640 1 -- 10.20.193.16:0/641498754 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b540052d0 con 0x7f8b600700f0 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b3f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b3f7fe640 1 -- 10.20.193.16:0/641498754 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54021060 con 0x7f8b600700f0 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b3f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b5d573640 4 set_mon_vals no callback set 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.620+0000 7f8b3f7fe640 10 monclient: dump: 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:10.628 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:10.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.725+0000 7f8b3f7fe640 1 -- 10.20.193.16:0/641498754 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f8b54024080 con 0x7f8b600700f0 2026-02-17T05:58:10.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.726+0000 7f8b3f7fe640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8b2c079170 0x7f8b2c07b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:10.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.726+0000 7f8b5e575640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8b2c079170 0x7f8b2c07b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:10.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.726+0000 7f8b5e575640 10 monclient: get_auth_request con 0x7f8b2c079170 auth_method 0 2026-02-17T05:58:10.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.726+0000 7f8b5e575640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8b2c079170 0x7f8b2c07b630 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8b60070b90 tx=0x7f8b48039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:10.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.726+0000 7f8b3f7fe640 10 client.4781.objecter ms_handle_connect 0x7f8b2c079170 2026-02-17T05:58:10.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.727+0000 7f8b3f7fe640 1 -- 10.20.193.16:0/641498754 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f8b5405d660 con 0x7f8b600700f0 2026-02-17T05:58:10.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.727+0000 7f8b3f7fe640 10 client.4781.objecter ms_dispatch2 0x7f8b60000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:10.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.727+0000 7f8b3f7fe640 3 client.4781.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:10.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.727+0000 7f8b3f7fe640 3 client.4781.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:10.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.727+0000 7f8b3f7fe640 20 client.4781.objecter dump_active .. 0 homeless 2026-02-17T05:58:10.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.733+0000 7f8b3f7fe640 1 -- 10.20.193.16:0/641498754 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8b54061620 con 0x7f8b600700f0 2026-02-17T05:58:10.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.733+0000 7f8b3f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:10.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.733+0000 7f8b3f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:10.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.831+0000 7f8b65171640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:10.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.831+0000 7f8b65171640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:10.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.831+0000 7f8b65171640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:10.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.831+0000 7f8b65171640 1 -- 10.20.193.16:0/641498754 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8b601573f0 con 0x7f8b600700f0 2026-02-17T05:58:10.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b3f7fe640 1 -- 10.20.193.16:0/641498754 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8b54065920 con 0x7f8b600700f0 2026-02-17T05:58:10.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b3f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:10.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b3f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:10.834 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:10.834 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b65171640 10 librados: watch_flush enter 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b65171640 10 librados: watch_flush exit 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b65171640 20 client.4781.objecter shutdown clearing up homeless session... 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b65171640 10 client.4781.objecter successfully canceled tick 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b65171640 1 -- 10.20.193.16:0/641498754 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8b2c079170 msgr2=0x7f8b2c07b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b65171640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8b2c079170 0x7f8b2c07b630 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8b60070b90 tx=0x7f8b48039040 comp rx=0 tx=0).stop 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b65171640 10 monclient: shutdown 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b65171640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b65171640 1 -- 10.20.193.16:0/641498754 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600700f0 msgr2=0x7f8b60141e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.834+0000 7f8b65171640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600700f0 0x7f8b60141e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c480 tx=0x7f8b5400c950 comp rx=0 tx=0).stop 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.835+0000 7f8b65171640 1 -- 10.20.193.16:0/641498754 shutdown_connections 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.835+0000 7f8b65171640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8b2c079170 0x7f8b2c07b630 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.835+0000 7f8b65171640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8b601536f0 0x7f8b60145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.835+0000 7f8b65171640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8b600c86f0 0x7f8b601f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:10.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.835+0000 7f8b65171640 1 --2- 10.20.193.16:0/641498754 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8b600700f0 0x7f8b60141e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:10.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.835+0000 7f8b65171640 1 -- 10.20.193.16:0/641498754 >> 10.20.193.16:0/641498754 conn(0x7f8b600cd770 msgr2=0x7f8b600c6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:10.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.835+0000 7f8b65171640 1 -- 10.20.193.16:0/641498754 shutdown_connections 2026-02-17T05:58:10.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.835+0000 7f8b65171640 1 -- 10.20.193.16:0/641498754 wait complete. 2026-02-17T05:58:10.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:10.835+0000 7f8b65171640 1 librados: shutdown 2026-02-17T05:58:13.844 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:13.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.913+0000 7f1f3dea7640 10 monclient: get_monmap_and_config 2026-02-17T05:58:13.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.913+0000 7f1f3dea7640 10 monclient: build_initial_monmap 2026-02-17T05:58:13.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.913+0000 7f1f3dea7640 10 monclient: monmap: 2026-02-17T05:58:13.915 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:13.915 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:13.915 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:13.914941+0000 2026-02-17T05:58:13.915 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:13.914941+0000 2026-02-17T05:58:13.915 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 1 Processor -- start 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 1 -- start start 2026-02-17T05:58:13.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 10 monclient: init 2026-02-17T05:58:13.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:13.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 10 monclient: _add_conns ranks=[0,4,5,3,2,1] 2026-02-17T05:58:13.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1f380700f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1f380c86f0 0x7f1f380704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 10 monclient(hunting): picked mon.noname-b con 0x7f1f380c86f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f1f381536f0 0x7f1f38153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 10 monclient(hunting): picked mon.noname-c con 0x7f1f381536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f380c4820 con 0x7f1f380700f0 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f1f380c4b00 con 0x7f1f380c86f0 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f1f380c4de0 con 0x7f1f381536f0 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.914+0000 7f1f3dea7640 10 monclient(hunting): authenticate will time out at 1146.483898s 2026-02-17T05:58:13.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.915+0000 7f1f377fe640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f1f380700f0 0x7f1f380c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:46896/0 (socket says 10.20.193.16:46896) 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.915+0000 7f1f377fe640 1 -- 10.20.193.16:0/2266018013 learned_addr learned my addr 10.20.193.16:0/2266018013 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.915+0000 7f1f37fff640 1 --2- 10.20.193.16:0/2266018013 >> v2:10.20.193.148:3300/0 conn(0x7f1f381536f0 0x7f1f38153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.915+0000 7f1f37fff640 10 monclient(hunting): get_auth_request con 0x7f1f381536f0 auth_method 0 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.915+0000 7f1f37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.915+0000 7f1f37fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.915+0000 7f1f37fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.915+0000 7f1f37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.915+0000 7f1f37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.915+0000 7f1f37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.916+0000 7f1f36ffd640 1 --2- 10.20.193.16:0/2266018013 >> v2:10.20.193.128:3300/0 conn(0x7f1f380c86f0 0x7f1f380704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:13.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.916+0000 7f1f36ffd640 10 monclient(hunting): get_auth_request con 0x7f1f380c86f0 auth_method 0 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.916+0000 7f1f36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.916+0000 7f1f36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.916+0000 7f1f36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.916+0000 7f1f36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.916+0000 7f1f36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.916+0000 7f1f36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.917+0000 7f1f37fff640 10 monclient(hunting): handle_auth_done global_id 4784 payload 274 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.917+0000 7f1f37fff640 1 -- 10.20.193.16:0/2266018013 >> v2:10.20.193.128:3300/0 conn(0x7f1f380c86f0 msgr2=0x7f1f380704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.917+0000 7f1f37fff640 1 --2- 10.20.193.16:0/2266018013 >> v2:10.20.193.128:3300/0 conn(0x7f1f380c86f0 0x7f1f380704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.917+0000 7f1f37fff640 1 -- 10.20.193.16:0/2266018013 >> v1:10.20.193.16:6789/0 conn(0x7f1f380700f0 legacy=0x7f1f380c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.917+0000 7f1f37fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.917+0000 7f1f37fff640 1 monclient: found mon.noname-c 2026-02-17T05:58:13.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.917+0000 7f1f37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:13.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.917+0000 7f1f37fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:13.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.917+0000 7f1f37fff640 1 -- 10.20.193.16:0/2266018013 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f380c50c0 con 0x7f1f381536f0 2026-02-17T05:58:13.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.919+0000 7f1f37fff640 1 --2- 10.20.193.16:0/2266018013 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f381536f0 0x7f1f38153b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f380c9540 tx=0x7f1f2c02f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1861b60d0fc480 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:13.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 1 -- 10.20.193.16:0/2266018013 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c03c070 con 0x7f1f381536f0 2026-02-17T05:58:13.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:13.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:13.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 10 monclient: dump: 2026-02-17T05:58:13.930 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:13.930 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:13.930 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f3dea7640 5 monclient: authenticate success, global_id 4784 2026-02-17T05:58:13.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 1 -- 10.20.193.16:0/2266018013 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f2c02fe40 con 0x7f1f381536f0 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 1 -- 10.20.193.16:0/2266018013 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c037660 con 0x7f1f381536f0 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f3dea7640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f3dea7640 4 set_mon_vals no callback set 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f367fc640 10 monclient: dump: 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:13.932 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f3dea7640 10 monclient: shutdown 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f3dea7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f3dea7640 1 -- 10.20.193.16:0/2266018013 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f381536f0 msgr2=0x7f1f38153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.921+0000 7f1f3dea7640 1 --2- 10.20.193.16:0/2266018013 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f381536f0 0x7f1f38153b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f380c9540 tx=0x7f1f2c02f130 comp rx=0 tx=0).stop 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 -- 10.20.193.16:0/2266018013 shutdown_connections 2026-02-17T05:58:13.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 --2- 10.20.193.16:0/2266018013 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f381536f0 0x7f1f38153b20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 --2- 10.20.193.16:0/2266018013 >> v2:10.20.193.128:3300/0 conn(0x7f1f380c86f0 0x7f1f380704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 -- 10.20.193.16:0/2266018013 >> 10.20.193.16:0/2266018013 conn(0x7f1f380cd770 msgr2=0x7f1f380cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 -- 10.20.193.16:0/2266018013 shutdown_connections 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 -- 10.20.193.16:0/2266018013 wait complete. 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 10 monclient: build_initial_monmap 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 10 monclient: monmap: 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:13.934 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 librados: starting msgr at 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 librados: starting objecter 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 Processor -- start 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 -- start start 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 librados: setting wanted keys 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 1 librados: calling monclient init 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.922+0000 7f1f3dea7640 10 monclient: init 2026-02-17T05:58:13.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f380700f0 0x7f1f38156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 10 monclient(hunting): picked mon.noname-c con 0x7f1f380700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f380c86f0 0x7f1f381542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 10 monclient(hunting): picked mon.noname-a con 0x7f1f380c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f381536f0 0x7f1f3814e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 10 monclient(hunting): picked mon.noname-b con 0x7f1f381536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38154060 con 0x7f1f380c86f0 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1f380c5680 con 0x7f1f381536f0 2026-02-17T05:58:13.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38074c00 con 0x7f1f380700f0 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f3dea7640 10 monclient(hunting): authenticate will time out at 1146.492265s 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f36ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f380c86f0 0x7f1f381542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f377fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f380700f0 0x7f1f38156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f36ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f380c86f0 0x7f1f381542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:57032/0 (socket says 10.20.193.16:57032) 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f36ffd640 1 -- 10.20.193.16:0/1293806162 learned_addr learned my addr 10.20.193.16:0/1293806162 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f36ffd640 10 monclient(hunting): get_auth_request con 0x7f1f380c86f0 auth_method 0 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:13.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f37fff640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f381536f0 0x7f1f3814e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f377fe640 10 monclient(hunting): get_auth_request con 0x7f1f380700f0 auth_method 0 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f377fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f377fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f37fff640 10 monclient(hunting): get_auth_request con 0x7f1f381536f0 auth_method 0 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f37fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.923+0000 7f1f37fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.924+0000 7f1f36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.924+0000 7f1f36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:13.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.924+0000 7f1f36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.924+0000 7f1f377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.924+0000 7f1f377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.924+0000 7f1f377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.924+0000 7f1f37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.924+0000 7f1f37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.924+0000 7f1f37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.925+0000 7f1f37fff640 10 monclient(hunting): handle_auth_done global_id 4723 payload 824 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.925+0000 7f1f37fff640 1 -- 10.20.193.16:0/1293806162 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f380700f0 msgr2=0x7f1f38156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.925+0000 7f1f37fff640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f380700f0 0x7f1f38156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.925+0000 7f1f37fff640 1 -- 10.20.193.16:0/1293806162 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f380c86f0 msgr2=0x7f1f381542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.925+0000 7f1f37fff640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f380c86f0 0x7f1f381542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.925+0000 7f1f37fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:13.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.925+0000 7f1f37fff640 1 monclient: found mon.noname-b 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.925+0000 7f1f37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.925+0000 7f1f37fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.925+0000 7f1f37fff640 1 -- 10.20.193.16:0/1293806162 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f38155530 con 0x7f1f381536f0 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.926+0000 7f1f37fff640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f381536f0 0x7f1f3814e6d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c036000 tx=0x7f1f2c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.926+0000 7f1f13fff640 10 client.?.objecter ms_handle_connect 0x7f1f381536f0 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.926+0000 7f1f13fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 1 -- 10.20.193.16:0/1293806162 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c047020 con 0x7f1f381536f0 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 10 monclient: dump: 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:13.940 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 10 monclient: _finish_auth 0 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 1 -- 10.20.193.16:0/1293806162 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f2c042470 con 0x7f1f381536f0 2026-02-17T05:58:13.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 1 -- 10.20.193.16:0/1293806162 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c03c040 con 0x7f1f381536f0 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f35ffb640 4 set_mon_vals no callback set 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f13fff640 10 monclient: dump: 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:13.942 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 5 monclient: authenticate success, global_id 4723 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 10 monclient: _renew_subs 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 1 -- 10.20.193.16:0/1293806162 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f38154c90 con 0x7f1f381536f0 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 10 client.4723.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 10 monclient: _renew_subs 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:13.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 1 -- 10.20.193.16:0/1293806162 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f38155250 con 0x7f1f381536f0 2026-02-17T05:58:13.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 1 librados: init done 2026-02-17T05:58:13.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:13.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:13.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:13.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.928+0000 7f1f3dea7640 1 -- 10.20.193.16:0/1293806162 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f380704d0 con 0x7f1f381536f0 2026-02-17T05:58:13.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.939+0000 7f1f13fff640 1 -- 10.20.193.16:0/1293806162 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1f2c042880 con 0x7f1f381536f0 2026-02-17T05:58:13.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.939+0000 7f1f13fff640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f08079190 0x7f1f0807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:13.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.940+0000 7f1f377fe640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f08079190 0x7f1f0807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:13.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.940+0000 7f1f377fe640 10 monclient: get_auth_request con 0x7f1f08079190 auth_method 0 2026-02-17T05:58:13.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.940+0000 7f1f377fe640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f08079190 0x7f1f0807b650 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1f24002770 tx=0x7f1f24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:13.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.940+0000 7f1f13fff640 10 client.4723.objecter ms_handle_connect 0x7f1f08079190 2026-02-17T05:58:13.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.940+0000 7f1f13fff640 1 -- 10.20.193.16:0/1293806162 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1f2c084030 con 0x7f1f381536f0 2026-02-17T05:58:13.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.940+0000 7f1f13fff640 10 client.4723.objecter ms_dispatch2 0x7f1f38000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:13.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.940+0000 7f1f13fff640 3 client.4723.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:13.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.940+0000 7f1f13fff640 3 client.4723.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:13.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:13.940+0000 7f1f13fff640 20 client.4723.objecter dump_active .. 0 homeless 2026-02-17T05:58:14.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.045+0000 7f1f13fff640 1 -- 10.20.193.16:0/1293806162 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1f2c051070 con 0x7f1f381536f0 2026-02-17T05:58:14.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.045+0000 7f1f13fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:14.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.045+0000 7f1f13fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:14.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.144+0000 7f1f3dea7640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:14.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.144+0000 7f1f3dea7640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:14.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.144+0000 7f1f3dea7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:14.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.144+0000 7f1f3dea7640 1 -- 10.20.193.16:0/1293806162 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1f38153e40 con 0x7f1f381536f0 2026-02-17T05:58:14.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f13fff640 1 -- 10.20.193.16:0/1293806162 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1f2c087ff0 con 0x7f1f381536f0 2026-02-17T05:58:14.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f13fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:14.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f13fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:14.146 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:14.146 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 10 librados: watch_flush enter 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 10 librados: watch_flush exit 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 20 client.4723.objecter shutdown clearing up homeless session... 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 10 client.4723.objecter successfully canceled tick 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 -- 10.20.193.16:0/1293806162 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f08079190 msgr2=0x7f1f0807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f08079190 0x7f1f0807b650 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1f24002770 tx=0x7f1f24009040 comp rx=0 tx=0).stop 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 10 monclient: shutdown 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 -- 10.20.193.16:0/1293806162 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f381536f0 msgr2=0x7f1f3814e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f381536f0 0x7f1f3814e6d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c036000 tx=0x7f1f2c004b40 comp rx=0 tx=0).stop 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 -- 10.20.193.16:0/1293806162 shutdown_connections 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1f08079190 0x7f1f0807b650 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1f381536f0 0x7f1f3814e6d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1f380c86f0 0x7f1f381542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 --2- 10.20.193.16:0/1293806162 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1f380700f0 0x7f1f38156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 -- 10.20.193.16:0/1293806162 >> 10.20.193.16:0/1293806162 conn(0x7f1f380cd770 msgr2=0x7f1f38144d30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 -- 10.20.193.16:0/1293806162 shutdown_connections 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 -- 10.20.193.16:0/1293806162 wait complete. 2026-02-17T05:58:14.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:14.147+0000 7f1f3dea7640 1 librados: shutdown 2026-02-17T05:58:17.157 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:17.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.226+0000 7f4c4a32c640 10 monclient: get_monmap_and_config 2026-02-17T05:58:17.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.226+0000 7f4c4a32c640 10 monclient: build_initial_monmap 2026-02-17T05:58:17.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.226+0000 7f4c4a32c640 10 monclient: monmap: 2026-02-17T05:58:17.227 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:17.227 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:17.227 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:17.227305+0000 2026-02-17T05:58:17.227 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:17.227305+0000 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.226+0000 7f4c4a32c640 1 Processor -- start 2026-02-17T05:58:17.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.226+0000 7f4c4a32c640 1 -- start start 2026-02-17T05:58:17.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.226+0000 7f4c4a32c640 10 monclient: init 2026-02-17T05:58:17.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:17.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 10 monclient: _add_conns ranks=[4,2,0,1,3,5] 2026-02-17T05:58:17.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f4c440700f0 0x7f4c440704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 10 monclient(hunting): picked mon.noname-b con 0x7f4c440700f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4c440c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4c441536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c440c4820 con 0x7f4c441536f0 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c440c4b00 con 0x7f4c440c7fe0 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f4c440c4de0 con 0x7f4c440700f0 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c4a32c640 10 monclient(hunting): authenticate will time out at 1149.796274s 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c488a2640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f4c441536f0 0x7f4c44153b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:46906/0 (socket says 10.20.193.16:46906) 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c488a2640 1 -- 10.20.193.16:0/3472873356 learned_addr learned my addr 10.20.193.16:0/3472873356 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:58:17.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c43fff640 1 --2- 10.20.193.16:0/3472873356 >> v2:10.20.193.128:3300/0 conn(0x7f4c440700f0 0x7f4c440704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c43fff640 10 monclient(hunting): get_auth_request con 0x7f4c440700f0 auth_method 0 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c43fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.227+0000 7f4c43fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.228+0000 7f4c43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.228+0000 7f4c43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.228+0000 7f4c43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.228+0000 7f4c42ffd640 1 -- 10.20.193.16:0/3472873356 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2722754074 0 0) 0x7f4c440c4820 con 0x7f4c441536f0 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.228+0000 7f4c42ffd640 10 monclient(hunting): _negotiate 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.228+0000 7f4c42ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.228+0000 7f4c42ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:17.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.228+0000 7f4c42ffd640 10 monclient(hunting): my global_id is 4668 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.228+0000 7f4c42ffd640 1 -- 10.20.193.16:0/3472873356 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c28003620 con 0x7f4c441536f0 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.229+0000 7f4c43fff640 10 monclient(hunting): handle_auth_done global_id 4726 payload 274 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.229+0000 7f4c43fff640 1 -- 10.20.193.16:0/3472873356 >> v1:10.20.193.148:6789/0 conn(0x7f4c440c7fe0 legacy=0x7f4c440c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.229+0000 7f4c42ffd640 1 -- 10.20.193.16:0/3472873356 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394699719 0 0) 0x7f4c440c4b00 con 0x7f4c440c7fe0 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.229+0000 7f4c43fff640 1 -- 10.20.193.16:0/3472873356 >> v1:10.20.193.16:6789/0 conn(0x7f4c441536f0 legacy=0x7f4c44153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.229+0000 7f4c43fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.229+0000 7f4c43fff640 1 monclient: found mon.noname-b 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.229+0000 7f4c43fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.229+0000 7f4c43fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.229+0000 7f4c43fff640 1 -- 10.20.193.16:0/3472873356 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c440c50c0 con 0x7f4c440700f0 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.229+0000 7f4c42ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T05:58:17.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.230+0000 7f4c43fff640 1 --2- 10.20.193.16:0/3472873356 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c440700f0 0x7f4c440704d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4c300098a0 tx=0x7f4c3002edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8570956faf059526 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:17.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 1 -- 10.20.193.16:0/3472873356 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3003c070 con 0x7f4c440700f0 2026-02-17T05:58:17.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:17.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:17.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 10 monclient: dump: 2026-02-17T05:58:17.241 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:17.241 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:17.241 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:17.241 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:17.241 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:17.241 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:17.241 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 10 monclient: _finish_auth 0 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 5 monclient: authenticate success, global_id 4726 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 1 -- 10.20.193.16:0/3472873356 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c3002fae0 con 0x7f4c440700f0 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:17.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 4 set_mon_vals no callback set 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 1 -- 10.20.193.16:0/3472873356 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30037440 con 0x7f4c440700f0 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c42ffd640 10 monclient: dump: 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:17.243 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 10 monclient: shutdown 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 1 -- 10.20.193.16:0/3472873356 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c440700f0 msgr2=0x7f4c440704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 1 --2- 10.20.193.16:0/3472873356 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c440700f0 0x7f4c440704d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4c300098a0 tx=0x7f4c3002edd0 comp rx=0 tx=0).stop 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 1 -- 10.20.193.16:0/3472873356 shutdown_connections 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 1 --2- 10.20.193.16:0/3472873356 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c440700f0 0x7f4c440704d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 1 -- 10.20.193.16:0/3472873356 >> 10.20.193.16:0/3472873356 conn(0x7f4c440cd770 msgr2=0x7f4c440cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 1 -- 10.20.193.16:0/3472873356 shutdown_connections 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 1 -- 10.20.193.16:0/3472873356 wait complete. 2026-02-17T05:58:17.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 10 monclient: build_initial_monmap 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 10 monclient: monmap: 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:17.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 1 librados: starting msgr at 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.232+0000 7f4c4a32c640 1 librados: starting objecter 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 1 Processor -- start 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 1 -- start start 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 1 librados: setting wanted keys 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 1 librados: calling monclient init 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient: init 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c440700f0 0x7f4c441f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient(hunting): picked mon.noname-c con 0x7f4c440700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c440c7fe0 0x7f4c440cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient(hunting): picked mon.noname-b con 0x7f4c440c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:17.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c441536f0 0x7f4c441e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:17.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient(hunting): picked mon.noname-a con 0x7f4c441536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:17.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:17.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c44074c00 con 0x7f4c441536f0 2026-02-17T05:58:17.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:17.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f4c440c5680 con 0x7f4c440c7fe0 2026-02-17T05:58:17.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:17.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c44074d80 con 0x7f4c440700f0 2026-02-17T05:58:17.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:17.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c4a32c640 10 monclient(hunting): authenticate will time out at 1149.802599s 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c488a2640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c441536f0 0x7f4c441e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c437fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c440c7fe0 0x7f4c440cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.233+0000 7f4c43fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c440700f0 0x7f4c441f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c488a2640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c441536f0 0x7f4c441e6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:57048/0 (socket says 10.20.193.16:57048) 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c488a2640 1 -- 10.20.193.16:0/1347288147 learned_addr learned my addr 10.20.193.16:0/1347288147 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c488a2640 10 monclient(hunting): get_auth_request con 0x7f4c441536f0 auth_method 0 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c488a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c488a2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c488a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c437fe640 10 monclient(hunting): get_auth_request con 0x7f4c440c7fe0 auth_method 0 2026-02-17T05:58:17.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c437fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c43fff640 10 monclient(hunting): get_auth_request con 0x7f4c440700f0 auth_method 0 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c43fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c43fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c488a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c488a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c488a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:17.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.234+0000 7f4c43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.235+0000 7f4c437fe640 10 monclient(hunting): handle_auth_done global_id 4729 payload 824 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.235+0000 7f4c437fe640 1 -- 10.20.193.16:0/1347288147 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c440700f0 msgr2=0x7f4c441f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.235+0000 7f4c437fe640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c440700f0 0x7f4c441f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.235+0000 7f4c437fe640 1 -- 10.20.193.16:0/1347288147 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c441536f0 msgr2=0x7f4c441e6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.235+0000 7f4c437fe640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c441536f0 0x7f4c441e6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.235+0000 7f4c437fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.235+0000 7f4c437fe640 1 monclient: found mon.noname-b 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.235+0000 7f4c437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.235+0000 7f4c437fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:58:17.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.235+0000 7f4c437fe640 1 -- 10.20.193.16:0/1347288147 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c441e61b0 con 0x7f4c440c7fe0 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.236+0000 7f4c437fe640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c440c7fe0 0x7f4c440cc570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400c400 tx=0x7f4c3400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.236+0000 7f4c40ff9640 10 client.?.objecter ms_handle_connect 0x7f4c440c7fe0 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.236+0000 7f4c40ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 1 -- 10.20.193.16:0/1347288147 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34017070 con 0x7f4c440c7fe0 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 10 monclient: dump: 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:17.251 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 10 monclient: _finish_auth 0 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 1 -- 10.20.193.16:0/1347288147 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c34005500 con 0x7f4c440c7fe0 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 1 -- 10.20.193.16:0/1347288147 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34003930 con 0x7f4c440c7fe0 2026-02-17T05:58:17.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c40ff9640 10 monclient: dump: 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:17.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c427fc640 4 set_mon_vals no callback set 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 5 monclient: authenticate success, global_id 4729 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 10 monclient: _renew_subs 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 1 -- 10.20.193.16:0/1347288147 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c441f7390 con 0x7f4c440c7fe0 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 10 client.4729.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 10 monclient: _renew_subs 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 1 -- 10.20.193.16:0/1347288147 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c441f7950 con 0x7f4c440c7fe0 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 1 librados: init done 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:17.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:17.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.238+0000 7f4c4a32c640 1 -- 10.20.193.16:0/1347288147 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4414f0e0 con 0x7f4c440c7fe0 2026-02-17T05:58:17.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.250+0000 7f4c40ff9640 1 -- 10.20.193.16:0/1347288147 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f4c34003ad0 con 0x7f4c440c7fe0 2026-02-17T05:58:17.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.250+0000 7f4c40ff9640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4c14079170 0x7f4c1407b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:17.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.250+0000 7f4c43fff640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4c14079170 0x7f4c1407b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:17.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.250+0000 7f4c43fff640 10 monclient: get_auth_request con 0x7f4c14079170 auth_method 0 2026-02-17T05:58:17.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.250+0000 7f4c40ff9640 10 client.4729.objecter ms_handle_connect 0x7f4c14079170 2026-02-17T05:58:17.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.250+0000 7f4c43fff640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4c14079170 0x7f4c1407b630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4c30039000 tx=0x7f4c30033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:17.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.252+0000 7f4c40ff9640 1 -- 10.20.193.16:0/1347288147 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f4c34003df0 con 0x7f4c440c7fe0 2026-02-17T05:58:17.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.252+0000 7f4c40ff9640 10 client.4729.objecter ms_dispatch2 0x7f4c44000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:17.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.252+0000 7f4c40ff9640 3 client.4729.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:17.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.252+0000 7f4c40ff9640 3 client.4729.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:17.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.252+0000 7f4c40ff9640 20 client.4729.objecter dump_active .. 0 homeless 2026-02-17T05:58:17.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.355+0000 7f4c40ff9640 1 -- 10.20.193.16:0/1347288147 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4c4414f0e0 con 0x7f4c440c7fe0 2026-02-17T05:58:17.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.355+0000 7f4c40ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:17.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.355+0000 7f4c40ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:17.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.454+0000 7f4c4a32c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:17.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.454+0000 7f4c4a32c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:17.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.454+0000 7f4c4a32c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:17.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.454+0000 7f4c4a32c640 1 -- 10.20.193.16:0/1347288147 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4c441e5fa0 con 0x7f4c440c7fe0 2026-02-17T05:58:17.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.456+0000 7f4c40ff9640 1 -- 10.20.193.16:0/1347288147 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f4c34021020 con 0x7f4c440c7fe0 2026-02-17T05:58:17.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.456+0000 7f4c40ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:17.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.456+0000 7f4c40ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:17.458 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:17.458 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:17.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.457+0000 7f4c4a32c640 10 librados: watch_flush enter 2026-02-17T05:58:17.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.457+0000 7f4c4a32c640 10 librados: watch_flush exit 2026-02-17T05:58:17.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.457+0000 7f4c4a32c640 20 client.4729.objecter shutdown clearing up homeless session... 2026-02-17T05:58:17.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.457+0000 7f4c4a32c640 10 client.4729.objecter successfully canceled tick 2026-02-17T05:58:17.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.457+0000 7f4c4a32c640 1 -- 10.20.193.16:0/1347288147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4c14079170 msgr2=0x7f4c1407b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:17.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.457+0000 7f4c4a32c640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4c14079170 0x7f4c1407b630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4c30039000 tx=0x7f4c30033000 comp rx=0 tx=0).stop 2026-02-17T05:58:17.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.457+0000 7f4c4a32c640 10 monclient: shutdown 2026-02-17T05:58:17.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.457+0000 7f4c4a32c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:17.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.457+0000 7f4c4a32c640 1 -- 10.20.193.16:0/1347288147 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c440c7fe0 msgr2=0x7f4c440cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:17.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.457+0000 7f4c4a32c640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c440c7fe0 0x7f4c440cc570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400c400 tx=0x7f4c3400c8d0 comp rx=0 tx=0).stop 2026-02-17T05:58:17.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.458+0000 7f4c4a32c640 1 -- 10.20.193.16:0/1347288147 shutdown_connections 2026-02-17T05:58:17.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.458+0000 7f4c4a32c640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4c14079170 0x7f4c1407b630 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:17.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.458+0000 7f4c4a32c640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c440c7fe0 0x7f4c440cc570 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:17.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.458+0000 7f4c4a32c640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c441536f0 0x7f4c441e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:17.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.458+0000 7f4c4a32c640 1 --2- 10.20.193.16:0/1347288147 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c440700f0 0x7f4c441f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:17.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.458+0000 7f4c4a32c640 1 -- 10.20.193.16:0/1347288147 >> 10.20.193.16:0/1347288147 conn(0x7f4c440cd770 msgr2=0x7f4c4414f500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:17.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.458+0000 7f4c4a32c640 1 -- 10.20.193.16:0/1347288147 shutdown_connections 2026-02-17T05:58:17.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.458+0000 7f4c4a32c640 1 -- 10.20.193.16:0/1347288147 wait complete. 2026-02-17T05:58:17.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:17.458+0000 7f4c4a32c640 1 librados: shutdown 2026-02-17T05:58:20.468 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:20.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.536+0000 7fac50c76640 10 monclient: get_monmap_and_config 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.536+0000 7fac50c76640 10 monclient: build_initial_monmap 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.536+0000 7fac50c76640 10 monclient: monmap: 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:20.537507+0000 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:20.537507+0000 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:20.537 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.536+0000 7fac50c76640 1 Processor -- start 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 1 -- start start 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 10 monclient: init 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 10 monclient: _add_conns ranks=[1,2,4,5,3,0] 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fac4c0700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fac4c0c86f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fac4c153e00 0x7fac4c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 10 monclient(hunting): picked mon.noname-b con 0x7fac4c153e00 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac4c0c4820 con 0x7fac4c0700f0 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac4c0c4b00 con 0x7fac4c0c86f0 2026-02-17T05:58:20.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fac4c0c4de0 con 0x7fac4c153e00 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac50c76640 10 monclient(hunting): authenticate will time out at 1153.106575s 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac49d74640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fac4c0c86f0 0x7fac4c1536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:59790/0 (socket says 10.20.193.16:59790) 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.537+0000 7fac49d74640 1 -- 10.20.193.16:0/3825511002 learned_addr learned my addr 10.20.193.16:0/3825511002 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.538+0000 7fac4ad76640 1 --2- 10.20.193.16:0/3825511002 >> v2:10.20.193.128:3300/0 conn(0x7fac4c153e00 0x7fac4c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.538+0000 7fac4ad76640 10 monclient(hunting): get_auth_request con 0x7fac4c153e00 auth_method 0 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.538+0000 7fac4ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.538+0000 7fac4ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.538+0000 7fac4ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.538+0000 7fac4ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.538+0000 7fac4ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.538+0000 7fac4ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.539+0000 7fac49573640 1 -- 10.20.193.16:0/3825511002 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4245447568 0 0) 0x7fac4c0c4b00 con 0x7fac4c0c86f0 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.539+0000 7fac49573640 10 monclient(hunting): _negotiate 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.539+0000 7fac49573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.539+0000 7fac49573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.539+0000 7fac49573640 10 monclient(hunting): my global_id is 4796 2026-02-17T05:58:20.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.539+0000 7fac49573640 1 -- 10.20.193.16:0/3825511002 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac38003620 con 0x7fac4c0c86f0 2026-02-17T05:58:20.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.540+0000 7fac4ad76640 10 monclient(hunting): handle_auth_done global_id 4732 payload 274 2026-02-17T05:58:20.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.540+0000 7fac4ad76640 1 -- 10.20.193.16:0/3825511002 >> v1:10.20.193.148:6789/0 conn(0x7fac4c0c86f0 legacy=0x7fac4c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.540+0000 7fac4ad76640 1 -- 10.20.193.16:0/3825511002 >> v1:10.20.193.128:6789/0 conn(0x7fac4c0700f0 legacy=0x7fac4c0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.540+0000 7fac4ad76640 10 monclient: _finish_hunting 0 2026-02-17T05:58:20.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.540+0000 7fac4ad76640 1 monclient: found mon.noname-b 2026-02-17T05:58:20.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.540+0000 7fac4ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:20.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.540+0000 7fac4ad76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:58:20.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.540+0000 7fac4ad76640 1 -- 10.20.193.16:0/3825511002 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c0c50c0 con 0x7fac4c153e00 2026-02-17T05:58:20.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.542+0000 7fac4ad76640 1 --2- 10.20.193.16:0/3825511002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c153e00 0x7fac4c0704d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009d90 tx=0x7fac3c02f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dbc7894aa30913b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 1 -- 10.20.193.16:0/3825511002 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c03c070 con 0x7fac4c153e00 2026-02-17T05:58:20.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:20.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 10 monclient: dump: 2026-02-17T05:58:20.550 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:20.550 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:20.550 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:20.550 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:20.550 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.550 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:20.550 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:20.550 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 10 monclient: _finish_auth 0 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 10 monclient: _check_auth_tickets 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 1 -- 10.20.193.16:0/3825511002 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3c037440 con 0x7fac4c153e00 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 5 monclient: authenticate success, global_id 4732 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 1 -- 10.20.193.16:0/3825511002 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c037810 con 0x7fac4c153e00 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:20.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 4 set_mon_vals no callback set 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac49573640 10 monclient: dump: 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:20.552 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 10 monclient: shutdown 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 1 -- 10.20.193.16:0/3825511002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c153e00 msgr2=0x7fac4c0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 1 --2- 10.20.193.16:0/3825511002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c153e00 0x7fac4c0704d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009d90 tx=0x7fac3c02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 1 -- 10.20.193.16:0/3825511002 shutdown_connections 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 1 --2- 10.20.193.16:0/3825511002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c153e00 0x7fac4c0704d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 1 -- 10.20.193.16:0/3825511002 >> 10.20.193.16:0/3825511002 conn(0x7fac4c0cd770 msgr2=0x7fac4c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 1 -- 10.20.193.16:0/3825511002 shutdown_connections 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.544+0000 7fac50c76640 1 -- 10.20.193.16:0/3825511002 wait complete. 2026-02-17T05:58:20.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.545+0000 7fac50c76640 10 monclient: build_initial_monmap 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.545+0000 7fac50c76640 10 monclient: monmap: 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:20.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.545+0000 7fac50c76640 1 librados: starting msgr at 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.545+0000 7fac50c76640 1 librados: starting objecter 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.545+0000 7fac50c76640 1 Processor -- start 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.545+0000 7fac50c76640 1 -- start start 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.545+0000 7fac50c76640 1 librados: setting wanted keys 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.545+0000 7fac50c76640 1 librados: calling monclient init 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.545+0000 7fac50c76640 10 monclient: init 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c0700f0 0x7fac4c1f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 10 monclient(hunting): picked mon.noname-a con 0x7fac4c0700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0c86f0 0x7fac4c1481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 10 monclient(hunting): picked mon.noname-b con 0x7fac4c0c86f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c153e00 0x7fac4c151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 10 monclient(hunting): picked mon.noname-c con 0x7fac4c153e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c074c00 con 0x7fac4c0700f0 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c0c5680 con 0x7fac4c0c86f0 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c074d80 con 0x7fac4c153e00 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac50c76640 10 monclient(hunting): authenticate will time out at 1153.115090s 2026-02-17T05:58:20.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac49d74640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0c86f0 0x7fac4c1481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4a575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c0700f0 0x7fac4c1f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4ad76640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c153e00 0x7fac4c151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4a575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c0700f0 0x7fac4c1f4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33018/0 (socket says 10.20.193.16:33018) 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4a575640 1 -- 10.20.193.16:0/2586500414 learned_addr learned my addr 10.20.193.16:0/2586500414 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4a575640 10 monclient(hunting): get_auth_request con 0x7fac4c0700f0 auth_method 0 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4ad76640 10 monclient(hunting): get_auth_request con 0x7fac4c153e00 auth_method 0 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac49d74640 10 monclient(hunting): get_auth_request con 0x7fac4c0c86f0 auth_method 0 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac49d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac49d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac49d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.546+0000 7fac4a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.547+0000 7fac49d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.547+0000 7fac49d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.547+0000 7fac49d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.547+0000 7fac4ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.547+0000 7fac4ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:20.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.547+0000 7fac4ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:20.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.548+0000 7fac4ad76640 10 monclient(hunting): handle_auth_done global_id 4799 payload 824 2026-02-17T05:58:20.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.548+0000 7fac4ad76640 1 -- 10.20.193.16:0/2586500414 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0c86f0 msgr2=0x7fac4c1481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.548+0000 7fac4ad76640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0c86f0 0x7fac4c1481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.548+0000 7fac4ad76640 1 -- 10.20.193.16:0/2586500414 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c0700f0 msgr2=0x7fac4c1f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.548+0000 7fac4ad76640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c0700f0 0x7fac4c1f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.548+0000 7fac4ad76640 10 monclient: _finish_hunting 0 2026-02-17T05:58:20.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.548+0000 7fac4ad76640 1 monclient: found mon.noname-c 2026-02-17T05:58:20.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.548+0000 7fac4ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:20.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.548+0000 7fac4ad76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:20.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.548+0000 7fac4ad76640 1 -- 10.20.193.16:0/2586500414 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c146b90 con 0x7fac4c153e00 2026-02-17T05:58:20.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.549+0000 7fac4ad76640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c153e00 0x7fac4c151580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fac3c036000 tx=0x7fac3c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.549+0000 7fac32ffd640 10 client.?.objecter ms_handle_connect 0x7fac4c153e00 2026-02-17T05:58:20.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.549+0000 7fac32ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:20.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 1 -- 10.20.193.16:0/2586500414 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c047020 con 0x7fac4c153e00 2026-02-17T05:58:20.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:20.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 10 monclient: dump: 2026-02-17T05:58:20.562 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:20.562 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:20.562 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:20.562 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:20.562 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.562 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:20.563 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:20.563 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:20.563 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:20.563 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:20.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 10 monclient: _finish_auth 0 2026-02-17T05:58:20.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:20.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:20.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 1 -- 10.20.193.16:0/2586500414 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3c03d040 con 0x7fac4c153e00 2026-02-17T05:58:20.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 5 monclient: authenticate success, global_id 4799 2026-02-17T05:58:20.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:20.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 1 -- 10.20.193.16:0/2586500414 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c03c050 con 0x7fac4c153e00 2026-02-17T05:58:20.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:20.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac32ffd640 10 monclient: dump: 2026-02-17T05:58:20.564 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac48d72640 4 set_mon_vals no callback set 2026-02-17T05:58:20.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 10 monclient: _renew_subs 2026-02-17T05:58:20.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:20.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 1 -- 10.20.193.16:0/2586500414 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4c146e70 con 0x7fac4c153e00 2026-02-17T05:58:20.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 10 client.4799.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:20.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 10 monclient: _renew_subs 2026-02-17T05:58:20.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:20.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 1 -- 10.20.193.16:0/2586500414 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4c147430 con 0x7fac4c153e00 2026-02-17T05:58:20.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 1 librados: init done 2026-02-17T05:58:20.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:20.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:20.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:20.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.551+0000 7fac50c76640 1 -- 10.20.193.16:0/2586500414 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4c0704d0 con 0x7fac4c153e00 2026-02-17T05:58:20.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.656+0000 7fac32ffd640 1 -- 10.20.193.16:0/2586500414 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fac3c041a30 con 0x7fac4c153e00 2026-02-17T05:58:20.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.657+0000 7fac32ffd640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac180791c0 0x7fac1807b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.657+0000 7fac4a575640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac180791c0 0x7fac1807b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.657+0000 7fac4a575640 10 monclient: get_auth_request con 0x7fac180791c0 auth_method 0 2026-02-17T05:58:20.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.657+0000 7fac4a575640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac180791c0 0x7fac1807b680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac40002770 tx=0x7fac40002990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.657+0000 7fac32ffd640 10 client.4799.objecter ms_handle_connect 0x7fac180791c0 2026-02-17T05:58:20.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.659+0000 7fac32ffd640 1 -- 10.20.193.16:0/2586500414 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fac3c041d50 con 0x7fac4c153e00 2026-02-17T05:58:20.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.659+0000 7fac32ffd640 10 client.4799.objecter ms_dispatch2 0x7fac4c000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:20.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.659+0000 7fac32ffd640 3 client.4799.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:20.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.659+0000 7fac32ffd640 3 client.4799.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:20.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.659+0000 7fac32ffd640 20 client.4799.objecter dump_active .. 0 homeless 2026-02-17T05:58:20.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.666+0000 7fac32ffd640 1 -- 10.20.193.16:0/2586500414 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fac3c087160 con 0x7fac4c153e00 2026-02-17T05:58:20.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.666+0000 7fac32ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:20.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.666+0000 7fac32ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:20.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.765+0000 7fac50c76640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:20.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.765+0000 7fac50c76640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:20.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.765+0000 7fac50c76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:20.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.765+0000 7fac50c76640 1 -- 10.20.193.16:0/2586500414 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fac4c0c5df0 con 0x7fac4c153e00 2026-02-17T05:58:20.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.768+0000 7fac32ffd640 1 -- 10.20.193.16:0/2586500414 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fac3c08b460 con 0x7fac4c153e00 2026-02-17T05:58:20.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.768+0000 7fac32ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:20.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.768+0000 7fac32ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:20.769 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:20.769 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:20.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 10 librados: watch_flush enter 2026-02-17T05:58:20.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 10 librados: watch_flush exit 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 20 client.4799.objecter shutdown clearing up homeless session... 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 10 client.4799.objecter successfully canceled tick 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 -- 10.20.193.16:0/2586500414 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac180791c0 msgr2=0x7fac1807b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac180791c0 0x7fac1807b680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac40002770 tx=0x7fac40002990 comp rx=0 tx=0).stop 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 10 monclient: shutdown 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 -- 10.20.193.16:0/2586500414 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c153e00 msgr2=0x7fac4c151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c153e00 0x7fac4c151580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fac3c036000 tx=0x7fac3c004b40 comp rx=0 tx=0).stop 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 -- 10.20.193.16:0/2586500414 shutdown_connections 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac180791c0 0x7fac1807b680 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0c86f0 0x7fac4c1481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c153e00 0x7fac4c151580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 --2- 10.20.193.16:0/2586500414 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c0700f0 0x7fac4c1f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 -- 10.20.193.16:0/2586500414 >> 10.20.193.16:0/2586500414 conn(0x7fac4c0cd770 msgr2=0x7fac4c14ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:20.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 -- 10.20.193.16:0/2586500414 shutdown_connections 2026-02-17T05:58:20.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 -- 10.20.193.16:0/2586500414 wait complete. 2026-02-17T05:58:20.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:20.769+0000 7fac50c76640 1 librados: shutdown 2026-02-17T05:58:23.779 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:23.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.846+0000 7f14b2a54640 10 monclient: get_monmap_and_config 2026-02-17T05:58:23.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.846+0000 7f14b2a54640 10 monclient: build_initial_monmap 2026-02-17T05:58:23.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.846+0000 7f14b2a54640 10 monclient: monmap: 2026-02-17T05:58:23.848 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:23.848 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:23.848 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:23.847973+0000 2026-02-17T05:58:23.848 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:23.847973+0000 2026-02-17T05:58:23.848 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:23.848 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:23.848 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:23.849 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:23.849 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:23.849 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:23.849 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:23.849 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:23.849 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:23.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.847+0000 7f14b2a54640 1 Processor -- start 2026-02-17T05:58:23.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.847+0000 7f14b2a54640 1 -- start start 2026-02-17T05:58:23.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.847+0000 7f14b2a54640 10 monclient: init 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 10 monclient: _add_conns ranks=[1,5,2,0,3,4] 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f14ac0700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f14ac0c86f0 0x7f14ac0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 10 monclient(hunting): picked mon.noname-c con 0x7f14ac0c86f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f14ac1536f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14ac0c4820 con 0x7f14ac0700f0 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14ac0c4b00 con 0x7f14ac1536f0 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f14ac0c4de0 con 0x7f14ac0c86f0 2026-02-17T05:58:23.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:23.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14b2a54640 10 monclient(hunting): authenticate will time out at 1156.417234s 2026-02-17T05:58:23.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14ab7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f14ac0c86f0 0x7f14ac0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:23.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14abfff640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f14ac0700f0 0x7f14ac0c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:45194/0 (socket says 10.20.193.16:45194) 2026-02-17T05:58:23.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.848+0000 7f14abfff640 1 -- 10.20.193.16:0/2428293513 learned_addr learned my addr 10.20.193.16:0/2428293513 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:58:23.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.849+0000 7f14ab7fe640 10 monclient(hunting): get_auth_request con 0x7f14ac0c86f0 auth_method 0 2026-02-17T05:58:23.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.849+0000 7f14ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:23.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.849+0000 7f14ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:23.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.849+0000 7f14ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:23.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.849+0000 7f14ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:23.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.849+0000 7f14ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:23.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.849+0000 7f14ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:23.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14aaffd640 1 -- 10.20.193.16:0/2428293513 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379630597 0 0) 0x7f14ac0c4820 con 0x7f14ac0700f0 2026-02-17T05:58:23.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14aaffd640 10 monclient(hunting): _negotiate 2026-02-17T05:58:23.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14aaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:23.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14aaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:23.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14aaffd640 10 monclient(hunting): my global_id is 4741 2026-02-17T05:58:23.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14aaffd640 1 -- 10.20.193.16:0/2428293513 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1490003620 con 0x7f14ac0700f0 2026-02-17T05:58:23.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14ab7fe640 10 monclient(hunting): handle_auth_done global_id 4802 payload 274 2026-02-17T05:58:23.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14ab7fe640 1 -- 10.20.193.16:0/2428293513 >> v1:10.20.193.148:6789/0 conn(0x7f14ac1536f0 legacy=0x7f14ac153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:23.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14ab7fe640 1 -- 10.20.193.16:0/2428293513 >> v1:10.20.193.128:6789/0 conn(0x7f14ac0700f0 legacy=0x7f14ac0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:23.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14ab7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:23.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14ab7fe640 1 monclient: found mon.noname-c 2026-02-17T05:58:23.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:23.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14ab7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:23.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.850+0000 7f14ab7fe640 1 -- 10.20.193.16:0/2428293513 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac0c50c0 con 0x7f14ac0c86f0 2026-02-17T05:58:23.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.852+0000 7f14ab7fe640 1 --2- 10.20.193.16:0/2428293513 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac0c86f0 0x7f14ac0704d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f149c0098a0 tx=0x7f149c02edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c297df72f741c856 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:23.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 1 -- 10.20.193.16:0/2428293513 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c03c070 con 0x7f14ac0c86f0 2026-02-17T05:58:23.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:23.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:23.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 10 monclient: dump: 2026-02-17T05:58:23.862 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:23.862 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:23.862 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:23.862 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:23.862 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:23.862 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:23.863 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:23.863 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:23.863 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:23.863 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:23.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 10 monclient: _finish_auth 0 2026-02-17T05:58:23.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:23.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:23.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14b2a54640 5 monclient: authenticate success, global_id 4802 2026-02-17T05:58:23.863 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14b2a54640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:23.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 1 -- 10.20.193.16:0/2428293513 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149c02fae0 con 0x7f14ac0c86f0 2026-02-17T05:58:23.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:23.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14b2a54640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:23.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 1 -- 10.20.193.16:0/2428293513 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c037440 con 0x7f14ac0c86f0 2026-02-17T05:58:23.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14b2a54640 4 set_mon_vals no callback set 2026-02-17T05:58:23.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:23.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:23.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.854+0000 7f14aaffd640 10 monclient: dump: 2026-02-17T05:58:23.864 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:23.864 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 10 monclient: shutdown 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:23.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 -- 10.20.193.16:0/2428293513 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac0c86f0 msgr2=0x7f14ac0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 --2- 10.20.193.16:0/2428293513 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac0c86f0 0x7f14ac0704d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f149c0098a0 tx=0x7f149c02edd0 comp rx=0 tx=0).stop 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 -- 10.20.193.16:0/2428293513 shutdown_connections 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 --2- 10.20.193.16:0/2428293513 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac0c86f0 0x7f14ac0704d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 -- 10.20.193.16:0/2428293513 >> 10.20.193.16:0/2428293513 conn(0x7f14ac0cd770 msgr2=0x7f14ac0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 -- 10.20.193.16:0/2428293513 shutdown_connections 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 -- 10.20.193.16:0/2428293513 wait complete. 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 10 monclient: build_initial_monmap 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 10 monclient: monmap: 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:23.866 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:23.867 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:23.867 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:23.867 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:23.867 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:23.867 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:23.867 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:23.867 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:23.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 librados: starting msgr at 2026-02-17T05:58:23.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 librados: starting objecter 2026-02-17T05:58:23.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 Processor -- start 2026-02-17T05:58:23.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 -- start start 2026-02-17T05:58:23.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 librados: setting wanted keys 2026-02-17T05:58:23.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 1 librados: calling monclient init 2026-02-17T05:58:23.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.855+0000 7f14b2a54640 10 monclient: init 2026-02-17T05:58:23.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:23.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:58:23.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f14ac0700f0 0x7f14ac1f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:23.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 10 monclient(hunting): picked mon.noname-b con 0x7f14ac0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:23.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f14ac0c86f0 0x7f14ac1f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:23.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 10 monclient(hunting): picked mon.noname-a con 0x7f14ac0c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac1536f0 0x7f14ac147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 10 monclient(hunting): picked mon.noname-c con 0x7f14ac1536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac074c00 con 0x7f14ac0c86f0 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac0c5680 con 0x7f14ac0700f0 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac074d80 con 0x7f14ac1536f0 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b2a54640 10 monclient(hunting): authenticate will time out at 1156.425434s 2026-02-17T05:58:23.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.856+0000 7f14b0fca640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac1536f0 0x7f14ac147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:23.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14b0fca640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac1536f0 0x7f14ac147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:60286/0 (socket says 10.20.193.16:60286) 2026-02-17T05:58:23.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14b0fca640 1 -- 10.20.193.16:0/603705002 learned_addr learned my addr 10.20.193.16:0/603705002 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:23.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14b0fca640 10 monclient(hunting): get_auth_request con 0x7f14ac1536f0 auth_method 0 2026-02-17T05:58:23.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14b0fca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:23.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14b0fca640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:23.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14b0fca640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:23.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14ab7fe640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f14ac0c86f0 0x7f14ac1f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:23.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14b0fca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:23.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14b0fca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:23.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14b0fca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.857+0000 7f14abfff640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f14ac0700f0 0x7f14ac1f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14ab7fe640 10 monclient(hunting): get_auth_request con 0x7f14ac0c86f0 auth_method 0 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14abfff640 10 monclient(hunting): get_auth_request con 0x7f14ac0700f0 auth_method 0 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14abfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14b0fca640 10 monclient(hunting): handle_auth_done global_id 4808 payload 824 2026-02-17T05:58:23.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14b0fca640 1 -- 10.20.193.16:0/603705002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f14ac0700f0 msgr2=0x7f14ac1f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:23.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14b0fca640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f14ac0700f0 0x7f14ac1f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:23.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14b0fca640 1 -- 10.20.193.16:0/603705002 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f14ac0c86f0 msgr2=0x7f14ac1f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:23.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14b0fca640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f14ac0c86f0 0x7f14ac1f2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:23.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14b0fca640 10 monclient: _finish_hunting 0 2026-02-17T05:58:23.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14b0fca640 1 monclient: found mon.noname-c 2026-02-17T05:58:23.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14b0fca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:23.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14b0fca640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:23.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.858+0000 7f14b0fca640 1 -- 10.20.193.16:0/603705002 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac1e67d0 con 0x7f14ac1536f0 2026-02-17T05:58:23.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.860+0000 7f14b0fca640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac1536f0 0x7f14ac147780 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f14a000cbd0 tx=0x7f14a000a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:23.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.860+0000 7f14a8ff9640 10 client.?.objecter ms_handle_connect 0x7f14ac1536f0 2026-02-17T05:58:23.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.860+0000 7f14a8ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:23.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 1 -- 10.20.193.16:0/603705002 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a000ad50 con 0x7f14ac1536f0 2026-02-17T05:58:23.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:23.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:23.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 10 monclient: dump: 2026-02-17T05:58:23.876 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:23.876 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:23.876 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:23.876 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:23.877 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:23.877 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:23.877 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:23.877 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:23.877 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:23.877 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:23.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 10 monclient: _finish_auth 0 2026-02-17T05:58:23.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:58:23.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:23.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 1 -- 10.20.193.16:0/603705002 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a0003a70 con 0x7f14ac1536f0 2026-02-17T05:58:23.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14b2a54640 5 monclient: authenticate success, global_id 4808 2026-02-17T05:58:23.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14b2a54640 10 monclient: _renew_subs 2026-02-17T05:58:23.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14b2a54640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:23.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14b2a54640 1 -- 10.20.193.16:0/603705002 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac1e6ab0 con 0x7f14ac1536f0 2026-02-17T05:58:23.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:23.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 1 -- 10.20.193.16:0/603705002 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0003e40 con 0x7f14ac1536f0 2026-02-17T05:58:23.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:23.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:23.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14aa7fc640 4 set_mon_vals no callback set 2026-02-17T05:58:23.878 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14a8ff9640 10 monclient: dump: 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:23.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14b2a54640 10 client.4808.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:23.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14b2a54640 10 monclient: _renew_subs 2026-02-17T05:58:23.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14b2a54640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:23.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14b2a54640 1 -- 10.20.193.16:0/603705002 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac1e7070 con 0x7f14ac1536f0 2026-02-17T05:58:23.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.862+0000 7f14b2a54640 1 librados: init done 2026-02-17T05:58:23.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.863+0000 7f14b2a54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:23.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.863+0000 7f14b2a54640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:23.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.863+0000 7f14b2a54640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:23.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.863+0000 7f14b2a54640 1 -- 10.20.193.16:0/603705002 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ac0704d0 con 0x7f14ac1536f0 2026-02-17T05:58:23.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.874+0000 7f14a8ff9640 1 -- 10.20.193.16:0/603705002 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f14a00050d0 con 0x7f14ac1536f0 2026-02-17T05:58:23.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.874+0000 7f14a8ff9640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f147c079190 0x7f147c07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:23.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.874+0000 7f14abfff640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f147c079190 0x7f147c07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:23.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.874+0000 7f14abfff640 10 monclient: get_auth_request con 0x7f147c079190 auth_method 0 2026-02-17T05:58:23.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.874+0000 7f14abfff640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f147c079190 0x7f147c07b650 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1498007fd0 tx=0x7f1498009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:23.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.874+0000 7f14a8ff9640 10 client.4808.objecter ms_handle_connect 0x7f147c079190 2026-02-17T05:58:23.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.876+0000 7f14a8ff9640 1 -- 10.20.193.16:0/603705002 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f14a005e990 con 0x7f14ac1536f0 2026-02-17T05:58:23.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.876+0000 7f14a8ff9640 10 client.4808.objecter ms_dispatch2 0x7f14ac000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:23.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.876+0000 7f14a8ff9640 3 client.4808.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:23.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.876+0000 7f14a8ff9640 3 client.4808.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:23.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.876+0000 7f14a8ff9640 20 client.4808.objecter dump_active .. 0 homeless 2026-02-17T05:58:23.890 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:04:02.717 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:23.892 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:04:02.719 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:23.902 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:04:02.729 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:23.917 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:04:02.743 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:23.921 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:04:02.748 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:23.930 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:04:02.756 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:23.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.982+0000 7f14a8ff9640 1 -- 10.20.193.16:0/603705002 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f14a0063050 con 0x7f14ac1536f0 2026-02-17T05:58:23.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.982+0000 7f14a8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:23.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:23.982+0000 7f14a8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:24.012 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:04:02.801 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:24.014 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:04:02.802 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:24.023 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:04:02.812 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:58:24.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.085+0000 7f14b2a54640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:24.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.085+0000 7f14b2a54640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:24.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.085+0000 7f14b2a54640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:24.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.085+0000 7f14b2a54640 1 -- 10.20.193.16:0/603705002 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f14ac1573f0 con 0x7f14ac1536f0 2026-02-17T05:58:24.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14a8ff9640 1 -- 10.20.193.16:0/603705002 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f14a0062950 con 0x7f14ac1536f0 2026-02-17T05:58:24.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14a8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:24.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14a8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:24.091 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:24.092 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 10 librados: watch_flush enter 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 10 librados: watch_flush exit 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 20 client.4808.objecter shutdown clearing up homeless session... 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 10 client.4808.objecter successfully canceled tick 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 -- 10.20.193.16:0/603705002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f147c079190 msgr2=0x7f147c07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f147c079190 0x7f147c07b650 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1498007fd0 tx=0x7f1498009040 comp rx=0 tx=0).stop 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 10 monclient: shutdown 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 -- 10.20.193.16:0/603705002 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac1536f0 msgr2=0x7f14ac147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac1536f0 0x7f14ac147780 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f14a000cbd0 tx=0x7f14a000a430 comp rx=0 tx=0).stop 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 -- 10.20.193.16:0/603705002 shutdown_connections 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f147c079190 0x7f147c07b650 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f14ac1536f0 0x7f14ac147780 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f14ac0c86f0 0x7f14ac1f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 --2- 10.20.193.16:0/603705002 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f14ac0700f0 0x7f14ac1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 -- 10.20.193.16:0/603705002 >> 10.20.193.16:0/603705002 conn(0x7f14ac0cd770 msgr2=0x7f14ac153b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 -- 10.20.193.16:0/603705002 shutdown_connections 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 -- 10.20.193.16:0/603705002 wait complete. 2026-02-17T05:58:24.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:24.092+0000 7f14b2a54640 1 librados: shutdown 2026-02-17T05:58:27.103 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:27.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.171+0000 7fbd3d153640 10 monclient: get_monmap_and_config 2026-02-17T05:58:27.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.171+0000 7fbd3d153640 10 monclient: build_initial_monmap 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.171+0000 7fbd3d153640 10 monclient: monmap: 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:27.172728+0000 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:27.172728+0000 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:27.173 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:27.174 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:27.174 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:27.174 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:27.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 1 Processor -- start 2026-02-17T05:58:27.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 1 -- start start 2026-02-17T05:58:27.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient: init 2026-02-17T05:58:27.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:27.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient: _add_conns ranks=[5,4,3,2,1,0] 2026-02-17T05:58:27.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fbd380700f0 0x7fbd380704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:27.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient(hunting): picked mon.noname-c con 0x7fbd380700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:27.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fbd380c7fe0 0x7fbd380c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:27.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient(hunting): picked mon.noname-b con 0x7fbd380c7fe0 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fbd381536f0 0x7fbd38153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient(hunting): picked mon.noname-a con 0x7fbd381536f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbd380c8970 con 0x7fbd381536f0 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fbd380c4820 con 0x7fbd380c7fe0 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fbd380c4b00 con 0x7fbd380700f0 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.172+0000 7fbd3d153640 10 monclient(hunting): authenticate will time out at 1159.741678s 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd37577640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fbd381536f0 0x7fbd38153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:27.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36575640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fbd380c7fe0 0x7fbd380c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36d76640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fbd380700f0 0x7fbd380704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd37577640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fbd381536f0 0x7fbd38153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33038/0 (socket says 10.20.193.16:33038) 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd37577640 1 -- 10.20.193.16:0/2514945313 learned_addr learned my addr 10.20.193.16:0/2514945313 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd37577640 10 monclient(hunting): get_auth_request con 0x7fbd381536f0 auth_method 0 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd37577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd37577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd37577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36575640 10 monclient(hunting): get_auth_request con 0x7fbd380c7fe0 auth_method 0 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:27.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36d76640 10 monclient(hunting): get_auth_request con 0x7fbd380700f0 auth_method 0 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd37577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd37577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd37577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.173+0000 7fbd36575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd36575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:27.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd37577640 10 monclient(hunting): handle_auth_done global_id 4680 payload 274 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd37577640 1 -- 10.20.193.16:0/2514945313 >> v2:10.20.193.148:3300/0 conn(0x7fbd380700f0 msgr2=0x7fbd380704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd37577640 1 --2- 10.20.193.16:0/2514945313 >> v2:10.20.193.148:3300/0 conn(0x7fbd380700f0 0x7fbd380704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd37577640 1 -- 10.20.193.16:0/2514945313 >> v2:10.20.193.128:3300/0 conn(0x7fbd380c7fe0 msgr2=0x7fbd380c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd37577640 1 --2- 10.20.193.16:0/2514945313 >> v2:10.20.193.128:3300/0 conn(0x7fbd380c7fe0 0x7fbd380c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd37577640 10 monclient: _finish_hunting 0 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd37577640 1 monclient: found mon.noname-a 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd37577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd37577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.174+0000 7fbd37577640 1 -- 10.20.193.16:0/2514945313 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd380c4de0 con 0x7fbd381536f0 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.175+0000 7fbd37577640 1 --2- 10.20.193.16:0/2514945313 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 0x7fbd38153b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd380c5b80 tx=0x7fbd2802f200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eca14c1af0b6271 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 1 -- 10.20.193.16:0/2514945313 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2802fc70 con 0x7fbd381536f0 2026-02-17T05:58:27.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 10 monclient: dump: 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:27.188 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 10 monclient: _finish_auth 0 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 10 monclient: _check_auth_tickets 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 1 -- 10.20.193.16:0/2514945313 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2802fe10 con 0x7fbd381536f0 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 1 -- 10.20.193.16:0/2514945313 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28037610 con 0x7fbd381536f0 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd35d74640 10 monclient: dump: 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:27.189 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd3d153640 5 monclient: authenticate success, global_id 4680 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd3d153640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd3d153640 4 set_mon_vals no callback set 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd3d153640 10 monclient: shutdown 2026-02-17T05:58:27.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd3d153640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd3d153640 1 -- 10.20.193.16:0/2514945313 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 msgr2=0x7fbd38153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.177+0000 7fbd3d153640 1 --2- 10.20.193.16:0/2514945313 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 0x7fbd38153b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd380c5b80 tx=0x7fbd2802f200 comp rx=0 tx=0).stop 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 -- 10.20.193.16:0/2514945313 shutdown_connections 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 --2- 10.20.193.16:0/2514945313 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 0x7fbd38153b00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 --2- 10.20.193.16:0/2514945313 >> v2:10.20.193.128:3300/0 conn(0x7fbd380c7fe0 0x7fbd380c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 --2- 10.20.193.16:0/2514945313 >> v2:10.20.193.148:3300/0 conn(0x7fbd380700f0 0x7fbd380704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 -- 10.20.193.16:0/2514945313 >> 10.20.193.16:0/2514945313 conn(0x7fbd380cd770 msgr2=0x7fbd380cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 -- 10.20.193.16:0/2514945313 shutdown_connections 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 -- 10.20.193.16:0/2514945313 wait complete. 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 10 monclient: build_initial_monmap 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 10 monclient: monmap: 2026-02-17T05:58:27.191 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 librados: starting msgr at 2026-02-17T05:58:27.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 librados: starting objecter 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 Processor -- start 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 -- start start 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 librados: setting wanted keys 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 1 librados: calling monclient init 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.178+0000 7fbd3d153640 10 monclient: init 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbd380700f0 0x7fbd38072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 10 monclient(hunting): picked mon.noname-b con 0x7fbd380700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbd380c7fe0 0x7fbd380731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 10 monclient(hunting): picked mon.noname-c con 0x7fbd380c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 0x7fbd3813d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:27.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 10 monclient(hunting): picked mon.noname-a con 0x7fbd381536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd380c5680 con 0x7fbd381536f0 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fbd380c53a0 con 0x7fbd380700f0 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd38154040 con 0x7fbd380c7fe0 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd3d153640 10 monclient(hunting): authenticate will time out at 1159.748314s 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd37577640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 0x7fbd3813d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd37577640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 0x7fbd3813d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33042/0 (socket says 10.20.193.16:33042) 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd37577640 1 -- 10.20.193.16:0/2717506826 learned_addr learned my addr 10.20.193.16:0/2717506826 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd37577640 10 monclient(hunting): get_auth_request con 0x7fbd381536f0 auth_method 0 2026-02-17T05:58:27.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd37577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd37577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd37577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd36575640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbd380c7fe0 0x7fbd380731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd37577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd37577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd37577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd36d76640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbd380700f0 0x7fbd38072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd36575640 10 monclient(hunting): get_auth_request con 0x7fbd380c7fe0 auth_method 0 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd36575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd36575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.179+0000 7fbd36575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:27.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.180+0000 7fbd36d76640 10 monclient(hunting): get_auth_request con 0x7fbd380700f0 auth_method 0 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.180+0000 7fbd36d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.180+0000 7fbd36d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.180+0000 7fbd36d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.180+0000 7fbd36575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.180+0000 7fbd36575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.180+0000 7fbd36575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.180+0000 7fbd36d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.180+0000 7fbd36d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.180+0000 7fbd36d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.181+0000 7fbd37577640 10 monclient(hunting): handle_auth_done global_id 4683 payload 824 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.181+0000 7fbd37577640 1 -- 10.20.193.16:0/2717506826 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbd380c7fe0 msgr2=0x7fbd380731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:27.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.181+0000 7fbd37577640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbd380c7fe0 0x7fbd380731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.181+0000 7fbd37577640 1 -- 10.20.193.16:0/2717506826 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbd380700f0 msgr2=0x7fbd38072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:27.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.181+0000 7fbd37577640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbd380700f0 0x7fbd38072c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.181+0000 7fbd37577640 10 monclient: _finish_hunting 0 2026-02-17T05:58:27.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.181+0000 7fbd37577640 1 monclient: found mon.noname-a 2026-02-17T05:58:27.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.181+0000 7fbd37577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:27.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.181+0000 7fbd37577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:27.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.181+0000 7fbd37577640 1 -- 10.20.193.16:0/2717506826 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd380cc7c0 con 0x7fbd381536f0 2026-02-17T05:58:27.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.182+0000 7fbd37577640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 0x7fbd3813d7d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd28030fd0 tx=0x7fbd28004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:27.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.182+0000 7fbd177fe640 10 client.?.objecter ms_handle_connect 0x7fbd381536f0 2026-02-17T05:58:27.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.182+0000 7fbd177fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:27.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 1 -- 10.20.193.16:0/2717506826 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28046070 con 0x7fbd381536f0 2026-02-17T05:58:27.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:27.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:27.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 10 monclient: dump: 2026-02-17T05:58:27.200 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:27.200 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:27.200 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:27.200 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:27.200 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:27.200 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 1 -- 10.20.193.16:0/2717506826 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd28042660 con 0x7fbd381536f0 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 1 -- 10.20.193.16:0/2717506826 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28009510 con 0x7fbd381536f0 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:27.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd177fe640 10 monclient: dump: 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd35573640 4 set_mon_vals no callback set 2026-02-17T05:58:27.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd3d153640 5 monclient: authenticate success, global_id 4683 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd3d153640 10 monclient: _renew_subs 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd3d153640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd3d153640 1 -- 10.20.193.16:0/2717506826 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd380cb180 con 0x7fbd381536f0 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd3d153640 10 client.4683.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd3d153640 10 monclient: _renew_subs 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd3d153640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd3d153640 1 -- 10.20.193.16:0/2717506826 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd380cb740 con 0x7fbd381536f0 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.183+0000 7fbd3d153640 1 librados: init done 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.184+0000 7fbd3d153640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.184+0000 7fbd3d153640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.184+0000 7fbd3d153640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:27.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.184+0000 7fbd3d153640 1 -- 10.20.193.16:0/2717506826 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd38074c00 con 0x7fbd381536f0 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.195+0000 7fbd177fe640 1 -- 10.20.193.16:0/2717506826 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fbd28052060 con 0x7fbd381536f0 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.196+0000 7fbd177fe640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbd04079190 0x7fbd0407b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.196+0000 7fbd36d76640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbd04079190 0x7fbd0407b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.196+0000 7fbd36d76640 10 monclient: get_auth_request con 0x7fbd04079190 auth_method 0 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.196+0000 7fbd36d76640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbd04079190 0x7fbd0407b650 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbd3806bf90 tx=0x7fbd2c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.196+0000 7fbd177fe640 10 client.4683.objecter ms_handle_connect 0x7fbd04079190 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.198+0000 7fbd177fe640 1 -- 10.20.193.16:0/2717506826 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fbd280833f0 con 0x7fbd381536f0 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.198+0000 7fbd177fe640 10 client.4683.objecter ms_dispatch2 0x7fbd38000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.198+0000 7fbd177fe640 3 client.4683.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.198+0000 7fbd177fe640 3 client.4683.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:27.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.198+0000 7fbd177fe640 20 client.4683.objecter dump_active .. 0 homeless 2026-02-17T05:58:27.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.301+0000 7fbd177fe640 1 -- 10.20.193.16:0/2717506826 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbd280873b0 con 0x7fbd381536f0 2026-02-17T05:58:27.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.301+0000 7fbd177fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:27.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.301+0000 7fbd177fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:27.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.401+0000 7fbd3d153640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:27.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.401+0000 7fbd3d153640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:27.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.401+0000 7fbd3d153640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:27.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.401+0000 7fbd3d153640 1 -- 10.20.193.16:0/2717506826 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbd381573f0 con 0x7fbd381536f0 2026-02-17T05:58:27.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.404+0000 7fbd177fe640 1 -- 10.20.193.16:0/2717506826 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fbd2808b6b0 con 0x7fbd381536f0 2026-02-17T05:58:27.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.404+0000 7fbd177fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:27.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.404+0000 7fbd177fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:27.405 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:27.405 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:27.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 10 librados: watch_flush enter 2026-02-17T05:58:27.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 10 librados: watch_flush exit 2026-02-17T05:58:27.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 20 client.4683.objecter shutdown clearing up homeless session... 2026-02-17T05:58:27.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 10 client.4683.objecter successfully canceled tick 2026-02-17T05:58:27.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 -- 10.20.193.16:0/2717506826 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbd04079190 msgr2=0x7fbd0407b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:27.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbd04079190 0x7fbd0407b650 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbd3806bf90 tx=0x7fbd2c009290 comp rx=0 tx=0).stop 2026-02-17T05:58:27.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 10 monclient: shutdown 2026-02-17T05:58:27.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:27.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 -- 10.20.193.16:0/2717506826 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 msgr2=0x7fbd3813d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:27.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 0x7fbd3813d7d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd28030fd0 tx=0x7fbd28004b40 comp rx=0 tx=0).stop 2026-02-17T05:58:27.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 -- 10.20.193.16:0/2717506826 shutdown_connections 2026-02-17T05:58:27.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbd04079190 0x7fbd0407b650 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbd381536f0 0x7fbd3813d7d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbd380c7fe0 0x7fbd380731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 --2- 10.20.193.16:0/2717506826 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbd380700f0 0x7fbd38072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:27.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 -- 10.20.193.16:0/2717506826 >> 10.20.193.16:0/2717506826 conn(0x7fbd380cd770 msgr2=0x7fbd3814e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:27.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 -- 10.20.193.16:0/2717506826 shutdown_connections 2026-02-17T05:58:27.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 -- 10.20.193.16:0/2717506826 wait complete. 2026-02-17T05:58:27.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:27.405+0000 7fbd3d153640 1 librados: shutdown 2026-02-17T05:58:30.414 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:30.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.482+0000 7fb52b04c640 10 monclient: get_monmap_and_config 2026-02-17T05:58:30.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.482+0000 7fb52b04c640 10 monclient: build_initial_monmap 2026-02-17T05:58:30.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.482+0000 7fb52b04c640 10 monclient: monmap: 2026-02-17T05:58:30.484 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:30.484 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:30.484 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:30.484219+0000 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:30.484219+0000 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.482+0000 7fb52b04c640 1 Processor -- start 2026-02-17T05:58:30.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.482+0000 7fb52b04c640 1 -- start start 2026-02-17T05:58:30.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.482+0000 7fb52b04c640 10 monclient: init 2026-02-17T05:58:30.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:30.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 10 monclient: _add_conns ranks=[3,1,4,2,0,5] 2026-02-17T05:58:30.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb524155ac0 0x7fb524073890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:30.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 10 monclient(hunting): picked mon.noname-a con 0x7fb524155ac0 addr v2:10.20.193.16:3300/0 2026-02-17T05:58:30.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb524155ea0 addr v1:10.20.193.128:6789/0 2026-02-17T05:58:30.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fb524070590 0x7fb52415d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:30.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 10 monclient(hunting): picked mon.noname-b con 0x7fb524070590 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:30.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb524072510 con 0x7fb524155ea0 2026-02-17T05:58:30.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:30.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb5240c8d10 con 0x7fb524155ac0 2026-02-17T05:58:30.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fb5240727f0 con 0x7fb524070590 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb52b04c640 10 monclient(hunting): authenticate will time out at 1163.053281s 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb5295c2640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fb524070590 0x7fb52415d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb528dc1640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb524155ac0 0x7fb524073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb528dc1640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb524155ac0 0x7fb524073890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:41598/0 (socket says 10.20.193.16:41598) 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb528dc1640 1 -- 10.20.193.16:0/2310324917 learned_addr learned my addr 10.20.193.16:0/2310324917 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb528dc1640 10 monclient(hunting): get_auth_request con 0x7fb524155ac0 auth_method 0 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb528dc1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb528dc1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:30.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.483+0000 7fb528dc1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.484+0000 7fb5295c2640 10 monclient(hunting): get_auth_request con 0x7fb524070590 auth_method 0 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.484+0000 7fb5295c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.484+0000 7fb5295c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.484+0000 7fb5295c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.484+0000 7fb528dc1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.484+0000 7fb528dc1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.484+0000 7fb528dc1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.484+0000 7fb5295c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.484+0000 7fb5295c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.484+0000 7fb5295c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.485+0000 7fb528dc1640 10 monclient(hunting): handle_auth_done global_id 4686 payload 274 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.485+0000 7fb528dc1640 1 -- 10.20.193.16:0/2310324917 >> v2:10.20.193.128:3300/0 conn(0x7fb524070590 msgr2=0x7fb52415d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:30.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.485+0000 7fb528dc1640 1 --2- 10.20.193.16:0/2310324917 >> v2:10.20.193.128:3300/0 conn(0x7fb524070590 0x7fb52415d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:30.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.485+0000 7fb528dc1640 1 -- 10.20.193.16:0/2310324917 >> v1:10.20.193.128:6789/0 conn(0x7fb524155ea0 legacy=0x7fb52406fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:30.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.485+0000 7fb528dc1640 10 monclient: _finish_hunting 0 2026-02-17T05:58:30.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.485+0000 7fb528dc1640 1 monclient: found mon.noname-a 2026-02-17T05:58:30.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.485+0000 7fb528dc1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:30.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.485+0000 7fb528dc1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:30.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.485+0000 7fb528dc1640 1 -- 10.20.193.16:0/2310324917 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb524072ad0 con 0x7fb524155ac0 2026-02-17T05:58:30.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.485+0000 7fb5295c2640 1 --2- 10.20.193.16:0/2310324917 >> v2:10.20.193.128:3300/0 conn(0x7fb524070590 0x7fb52415d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:58:30.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.486+0000 7fb528dc1640 1 --2- 10.20.193.16:0/2310324917 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 0x7fb524073890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb52406f470 tx=0x7fb51402eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b786080aa2e5688 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:30.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 1 -- 10.20.193.16:0/2310324917 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51403c070 con 0x7fb524155ac0 2026-02-17T05:58:30.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:30.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:30.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 10 monclient: dump: 2026-02-17T05:58:30.499 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:30.499 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:30.499 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:30.499 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 5 monclient: authenticate success, global_id 4686 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 1 -- 10.20.193.16:0/2310324917 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51402fbe0 con 0x7fb524155ac0 2026-02-17T05:58:30.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 1 -- 10.20.193.16:0/2310324917 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514037440 con 0x7fb524155ac0 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb51b7fe640 10 monclient: dump: 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:30.501 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 4 set_mon_vals no callback set 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 10 monclient: shutdown 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 1 -- 10.20.193.16:0/2310324917 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 msgr2=0x7fb524073890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 1 --2- 10.20.193.16:0/2310324917 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 0x7fb524073890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb52406f470 tx=0x7fb51402eed0 comp rx=0 tx=0).stop 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 1 -- 10.20.193.16:0/2310324917 shutdown_connections 2026-02-17T05:58:30.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 1 --2- 10.20.193.16:0/2310324917 >> v2:10.20.193.128:3300/0 conn(0x7fb524070590 0x7fb52415d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 1 --2- 10.20.193.16:0/2310324917 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 0x7fb524073890 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 1 -- 10.20.193.16:0/2310324917 >> 10.20.193.16:0/2310324917 conn(0x7fb5240cd770 msgr2=0x7fb5240cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 1 -- 10.20.193.16:0/2310324917 shutdown_connections 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.488+0000 7fb52b04c640 1 -- 10.20.193.16:0/2310324917 wait complete. 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient: build_initial_monmap 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient: monmap: 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:30.503 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 librados: starting msgr at 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 librados: starting objecter 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 Processor -- start 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 -- start start 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 librados: setting wanted keys 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 librados: calling monclient init 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient: init 2026-02-17T05:58:30.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb524070590 0x7fb5241e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient(hunting): picked mon.noname-b con 0x7fb524070590 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 0x7fb5241f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient(hunting): picked mon.noname-a con 0x7fb524155ac0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb524155ea0 0x7fb52414d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient(hunting): picked mon.noname-c con 0x7fb524155ea0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb524073370 con 0x7fb524155ac0 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fb524073090 con 0x7fb524070590 2026-02-17T05:58:30.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb52415cb30 con 0x7fb524155ea0 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.489+0000 7fb52b04c640 10 monclient(hunting): authenticate will time out at 1163.059826s 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb51bfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 0x7fb5241f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb528dc1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb524070590 0x7fb5241e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb51bfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 0x7fb5241f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:41600/0 (socket says 10.20.193.16:41600) 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb51bfff640 1 -- 10.20.193.16:0/3638158438 learned_addr learned my addr 10.20.193.16:0/3638158438 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb51bfff640 10 monclient(hunting): get_auth_request con 0x7fb524155ac0 auth_method 0 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb51bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb51bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb51bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:30.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb5295c2640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb524155ea0 0x7fb52414d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb528dc1640 10 monclient(hunting): get_auth_request con 0x7fb524070590 auth_method 0 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb528dc1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb528dc1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb528dc1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb51bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb51bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb51bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb5295c2640 10 monclient(hunting): get_auth_request con 0x7fb524155ea0 auth_method 0 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb5295c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb5295c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb5295c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:30.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb528dc1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb528dc1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb528dc1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb5295c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb5295c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.490+0000 7fb5295c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.491+0000 7fb51bfff640 10 monclient(hunting): handle_auth_done global_id 4689 payload 824 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.491+0000 7fb51bfff640 1 -- 10.20.193.16:0/3638158438 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb524155ea0 msgr2=0x7fb52414d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.491+0000 7fb51bfff640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb524155ea0 0x7fb52414d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.491+0000 7fb51bfff640 1 -- 10.20.193.16:0/3638158438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb524070590 msgr2=0x7fb5241e7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.491+0000 7fb51bfff640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb524070590 0x7fb5241e7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.491+0000 7fb51bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:30.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.491+0000 7fb51bfff640 1 monclient: found mon.noname-a 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.491+0000 7fb51bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.491+0000 7fb51bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.491+0000 7fb51bfff640 1 -- 10.20.193.16:0/3638158438 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5241f4c00 con 0x7fb524155ac0 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.492+0000 7fb51bfff640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 0x7fb5241f3d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c480 tx=0x7fb50c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.492+0000 7fb5197fa640 10 client.?.objecter ms_handle_connect 0x7fb524155ac0 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.492+0000 7fb5197fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 1 -- 10.20.193.16:0/3638158438 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c017070 con 0x7fb524155ac0 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 10 monclient: dump: 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:30.509 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 10 monclient: _finish_auth 0 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 10 monclient: _check_auth_tickets 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 1 -- 10.20.193.16:0/3638158438 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c0052f0 con 0x7fb524155ac0 2026-02-17T05:58:30.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 1 -- 10.20.193.16:0/3638158438 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c0036f0 con 0x7fb524155ac0 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb5197fa640 10 monclient: dump: 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:30.511 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb51affd640 4 set_mon_vals no callback set 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb52b04c640 5 monclient: authenticate success, global_id 4689 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb52b04c640 10 monclient: _renew_subs 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb52b04c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb52b04c640 1 -- 10.20.193.16:0/3638158438 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5241e73a0 con 0x7fb524155ac0 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb52b04c640 10 client.4689.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb52b04c640 10 monclient: _renew_subs 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb52b04c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb52b04c640 1 -- 10.20.193.16:0/3638158438 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5241f4920 con 0x7fb524155ac0 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.494+0000 7fb52b04c640 1 librados: init done 2026-02-17T05:58:30.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.495+0000 7fb52b04c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:30.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.495+0000 7fb52b04c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:30.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.495+0000 7fb52b04c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:30.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.495+0000 7fb52b04c640 1 -- 10.20.193.16:0/3638158438 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb524073890 con 0x7fb524155ac0 2026-02-17T05:58:30.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.507+0000 7fb5197fa640 1 -- 10.20.193.16:0/3638158438 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fb50c003890 con 0x7fb524155ac0 2026-02-17T05:58:30.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.507+0000 7fb5197fa640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4f40791c0 0x7fb4f407b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:30.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.507+0000 7fb528dc1640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4f40791c0 0x7fb4f407b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:30.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.507+0000 7fb528dc1640 10 monclient: get_auth_request con 0x7fb4f40791c0 auth_method 0 2026-02-17T05:58:30.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.507+0000 7fb5197fa640 10 client.4689.objecter ms_handle_connect 0x7fb4f40791c0 2026-02-17T05:58:30.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.507+0000 7fb528dc1640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4f40791c0 0x7fb4f407b680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb524070f40 tx=0x7fb514039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:30.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.611+0000 7fb5197fa640 1 -- 10.20.193.16:0/3638158438 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fb50c05d5a0 con 0x7fb524155ac0 2026-02-17T05:58:30.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.611+0000 7fb5197fa640 10 client.4689.objecter ms_dispatch2 0x7fb524000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:30.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.611+0000 7fb5197fa640 3 client.4689.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:30.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.611+0000 7fb5197fa640 3 client.4689.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:30.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.611+0000 7fb5197fa640 20 client.4689.objecter dump_active .. 0 homeless 2026-02-17T05:58:30.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.611+0000 7fb5197fa640 1 -- 10.20.193.16:0/3638158438 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb50c021020 con 0x7fb524155ac0 2026-02-17T05:58:30.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.611+0000 7fb5197fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:30.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.611+0000 7fb5197fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:30.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.712+0000 7fb52b04c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:30.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.712+0000 7fb52b04c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:30.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.712+0000 7fb52b04c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:30.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.712+0000 7fb52b04c640 1 -- 10.20.193.16:0/3638158438 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb52415f030 con 0x7fb524155ac0 2026-02-17T05:58:30.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.714+0000 7fb5197fa640 1 -- 10.20.193.16:0/3638158438 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fb50c061560 con 0x7fb524155ac0 2026-02-17T05:58:30.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.714+0000 7fb5197fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:30.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.714+0000 7fb5197fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:30.715 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:30.715 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:30.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 10 librados: watch_flush enter 2026-02-17T05:58:30.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 10 librados: watch_flush exit 2026-02-17T05:58:30.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 20 client.4689.objecter shutdown clearing up homeless session... 2026-02-17T05:58:30.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 10 client.4689.objecter successfully canceled tick 2026-02-17T05:58:30.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 -- 10.20.193.16:0/3638158438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4f40791c0 msgr2=0x7fb4f407b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:30.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4f40791c0 0x7fb4f407b680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb524070f40 tx=0x7fb514039040 comp rx=0 tx=0).stop 2026-02-17T05:58:30.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 10 monclient: shutdown 2026-02-17T05:58:30.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:30.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 -- 10.20.193.16:0/3638158438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 msgr2=0x7fb5241f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:30.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 0x7fb5241f3d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c480 tx=0x7fb50c00c950 comp rx=0 tx=0).stop 2026-02-17T05:58:30.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 -- 10.20.193.16:0/3638158438 shutdown_connections 2026-02-17T05:58:30.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb4f40791c0 0x7fb4f407b680 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:30.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb524155ea0 0x7fb52414d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:30.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb524155ac0 0x7fb5241f3d00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:30.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 --2- 10.20.193.16:0/3638158438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb524070590 0x7fb5241e7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:30.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 -- 10.20.193.16:0/3638158438 >> 10.20.193.16:0/3638158438 conn(0x7fb5240cd770 msgr2=0x7fb52415e510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:30.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 -- 10.20.193.16:0/3638158438 shutdown_connections 2026-02-17T05:58:30.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 -- 10.20.193.16:0/3638158438 wait complete. 2026-02-17T05:58:30.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:30.716+0000 7fb52b04c640 1 librados: shutdown 2026-02-17T05:58:33.726 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:33.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.795+0000 7f5557a64640 10 monclient: get_monmap_and_config 2026-02-17T05:58:33.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.795+0000 7f5557a64640 10 monclient: build_initial_monmap 2026-02-17T05:58:33.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.795+0000 7f5557a64640 10 monclient: monmap: 2026-02-17T05:58:33.797 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:33.797 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:33.797 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:33.796953+0000 2026-02-17T05:58:33.797 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:33.796953+0000 2026-02-17T05:58:33.797 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:33.797 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:33.797 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:33.798 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:33.798 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:33.798 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:33.798 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:33.798 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:33.798 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:33.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 1 Processor -- start 2026-02-17T05:58:33.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 1 -- start start 2026-02-17T05:58:33.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 10 monclient: init 2026-02-17T05:58:33.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:33.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 10 monclient: _add_conns ranks=[1,2,5,3,0,4] 2026-02-17T05:58:33.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f55500700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:58:33.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f55500c86f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:58:33.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f5550153e00 0x7f55500704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:33.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 10 monclient(hunting): picked mon.noname-c con 0x7f5550153e00 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:33.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55500c4820 con 0x7f55500700f0 2026-02-17T05:58:33.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55500c4b00 con 0x7f55500c86f0 2026-02-17T05:58:33.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f55500c4de0 con 0x7f5550153e00 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.796+0000 7f5557a64640 10 monclient(hunting): authenticate will time out at 1166.365947s 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.797+0000 7f55557d9640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f55500700f0 0x7f55500c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:40008/0 (socket says 10.20.193.16:40008) 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.797+0000 7f55557d9640 1 -- 10.20.193.16:0/2907268376 learned_addr learned my addr 10.20.193.16:0/2907268376 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.797+0000 7f5555fda640 1 --2- 10.20.193.16:0/2907268376 >> v2:10.20.193.148:3300/0 conn(0x7f5550153e00 0x7f55500704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.797+0000 7f5555fda640 10 monclient(hunting): get_auth_request con 0x7f5550153e00 auth_method 0 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.797+0000 7f5555fda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.797+0000 7f5555fda640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.797+0000 7f5555fda640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.798+0000 7f5555fda640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:33.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.798+0000 7f5555fda640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.798+0000 7f5555fda640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.798+0000 7f553ffff640 1 -- 10.20.193.16:0/2907268376 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 882469577 0 0) 0x7f55500c4820 con 0x7f55500700f0 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.798+0000 7f553ffff640 10 monclient(hunting): _negotiate 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.798+0000 7f553ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.798+0000 7f553ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.798+0000 7f553ffff640 10 monclient(hunting): my global_id is 4762 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.798+0000 7f553ffff640 1 -- 10.20.193.16:0/2907268376 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5538003620 con 0x7f55500700f0 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.799+0000 7f5555fda640 10 monclient(hunting): handle_auth_done global_id 4820 payload 274 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.799+0000 7f5555fda640 1 -- 10.20.193.16:0/2907268376 >> v1:10.20.193.148:6789/0 conn(0x7f55500c86f0 legacy=0x7f55501536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.799+0000 7f5555fda640 1 -- 10.20.193.16:0/2907268376 >> v1:10.20.193.128:6789/0 conn(0x7f55500700f0 legacy=0x7f55500c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.799+0000 7f5555fda640 10 monclient: _finish_hunting 0 2026-02-17T05:58:33.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.799+0000 7f5555fda640 1 monclient: found mon.noname-c 2026-02-17T05:58:33.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.799+0000 7f5555fda640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:33.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.799+0000 7f5555fda640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:33.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.799+0000 7f5555fda640 1 -- 10.20.193.16:0/2907268376 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55500c50c0 con 0x7f5550153e00 2026-02-17T05:58:33.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.801+0000 7f5555fda640 1 --2- 10.20.193.16:0/2907268376 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5550153e00 0x7f55500704d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f554c009d90 tx=0x7f554c02f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=505f8cc720a9df90 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:33.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 1 -- 10.20.193.16:0/2907268376 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c03c070 con 0x7f5550153e00 2026-02-17T05:58:33.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:33.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 10 monclient: dump: 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:33.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 10 monclient: _finish_auth 0 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 1 -- 10.20.193.16:0/2907268376 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554c037440 con 0x7f5550153e00 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 5 monclient: authenticate success, global_id 4820 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 1 -- 10.20.193.16:0/2907268376 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c037810 con 0x7f5550153e00 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 4 set_mon_vals no callback set 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:33.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f553ffff640 10 monclient: dump: 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:33.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 10 monclient: shutdown 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 1 -- 10.20.193.16:0/2907268376 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5550153e00 msgr2=0x7f55500704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 1 --2- 10.20.193.16:0/2907268376 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5550153e00 0x7f55500704d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f554c009d90 tx=0x7f554c02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 1 -- 10.20.193.16:0/2907268376 shutdown_connections 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 1 --2- 10.20.193.16:0/2907268376 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5550153e00 0x7f55500704d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 1 -- 10.20.193.16:0/2907268376 >> 10.20.193.16:0/2907268376 conn(0x7f55500cd770 msgr2=0x7f55500cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 1 -- 10.20.193.16:0/2907268376 shutdown_connections 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 1 -- 10.20.193.16:0/2907268376 wait complete. 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 10 monclient: build_initial_monmap 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.803+0000 7f5557a64640 10 monclient: monmap: 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:33.812 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 librados: starting msgr at 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 librados: starting objecter 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 Processor -- start 2026-02-17T05:58:33.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 -- start start 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 librados: setting wanted keys 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 librados: calling monclient init 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient: init 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55500700f0 0x7f55501f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient(hunting): picked mon.noname-a con 0x7f55500700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55500c86f0 0x7f555006e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient(hunting): picked mon.noname-b con 0x7f55500c86f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5550153e00 0x7f55501f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient(hunting): picked mon.noname-c con 0x7f5550153e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:33.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5550074c00 con 0x7f55500700f0 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f55500c5680 con 0x7f55500c86f0 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5550074d80 con 0x7f5550153e00 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f5557a64640 10 monclient(hunting): authenticate will time out at 1166.373599s 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.804+0000 7f55557d9640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55500700f0 0x7f55501f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.805+0000 7f55557d9640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55500700f0 0x7f55501f6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:41604/0 (socket says 10.20.193.16:41604) 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.805+0000 7f55557d9640 1 -- 10.20.193.16:0/1858058375 learned_addr learned my addr 10.20.193.16:0/1858058375 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.805+0000 7f55557d9640 10 monclient(hunting): get_auth_request con 0x7f55500700f0 auth_method 0 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.805+0000 7f55557d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:33.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.805+0000 7f55557d9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.805+0000 7f55557d9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.805+0000 7f55557d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.805+0000 7f55557d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.805+0000 7f55557d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.806+0000 7f55557d9640 10 monclient(hunting): handle_auth_done global_id 4692 payload 824 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.807+0000 7f55557d9640 1 -- 10.20.193.16:0/1858058375 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5550153e00 msgr2=0x7f55501f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.807+0000 7f55557d9640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5550153e00 0x7f55501f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.807+0000 7f55557d9640 1 -- 10.20.193.16:0/1858058375 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55500c86f0 msgr2=0x7f555006e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.807+0000 7f55557d9640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55500c86f0 0x7f555006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.807+0000 7f55557d9640 10 monclient: _finish_hunting 0 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.807+0000 7f55557d9640 1 monclient: found mon.noname-a 2026-02-17T05:58:33.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.807+0000 7f55557d9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:33.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.807+0000 7f55557d9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:33.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.807+0000 7f55557d9640 1 -- 10.20.193.16:0/1858058375 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555006e7c0 con 0x7f55500700f0 2026-02-17T05:58:33.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.808+0000 7f55557d9640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55500700f0 0x7f55501f6dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f554000c4d0 tx=0x7f554000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:33.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.808+0000 7f553dffb640 10 client.?.objecter ms_handle_connect 0x7f55500700f0 2026-02-17T05:58:33.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.808+0000 7f553dffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:33.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 1 -- 10.20.193.16:0/1858058375 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540017070 con 0x7f55500700f0 2026-02-17T05:58:33.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:33.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:33.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 10 monclient: dump: 2026-02-17T05:58:33.819 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:33.819 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:33.819 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:33.819 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 10 monclient: _finish_auth 0 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 1 -- 10.20.193.16:0/1858058375 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5540005de0 con 0x7f55500700f0 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:33.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 1 -- 10.20.193.16:0/1858058375 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55400072b0 con 0x7f55500700f0 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 5 monclient: authenticate success, global_id 4692 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 10 monclient: _renew_subs 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553f7fe640 4 set_mon_vals no callback set 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 1 -- 10.20.193.16:0/1858058375 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55501f7cd0 con 0x7f55500700f0 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f553dffb640 10 monclient: dump: 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:33.821 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 10 client.4692.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 10 monclient: _renew_subs 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 1 -- 10.20.193.16:0/1858058375 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55501f7430 con 0x7f55500700f0 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 1 librados: init done 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:33.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:33.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:33.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.810+0000 7f5557a64640 1 -- 10.20.193.16:0/1858058375 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55500704d0 con 0x7f55500700f0 2026-02-17T05:58:33.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.822+0000 7f553dffb640 1 -- 10.20.193.16:0/1858058375 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f5540007450 con 0x7f55500700f0 2026-02-17T05:58:33.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.822+0000 7f553dffb640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5520079190 0x7f552007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:33.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.822+0000 7f5554fd8640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5520079190 0x7f552007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:33.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.822+0000 7f5554fd8640 10 monclient: get_auth_request con 0x7f5520079190 auth_method 0 2026-02-17T05:58:33.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.822+0000 7f5554fd8640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5520079190 0x7f552007b650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5544002770 tx=0x7f5544007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:33.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.822+0000 7f553dffb640 10 client.4692.objecter ms_handle_connect 0x7f5520079190 2026-02-17T05:58:33.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.823+0000 7f553dffb640 1 -- 10.20.193.16:0/1858058375 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f5540007770 con 0x7f55500700f0 2026-02-17T05:58:33.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.823+0000 7f553dffb640 10 client.4692.objecter ms_dispatch2 0x7f5550000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:33.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.823+0000 7f553dffb640 3 client.4692.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:33.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.823+0000 7f553dffb640 3 client.4692.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:33.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.824+0000 7f553dffb640 20 client.4692.objecter dump_active .. 0 homeless 2026-02-17T05:58:33.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.935+0000 7f553dffb640 1 -- 10.20.193.16:0/1858058375 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5540021020 con 0x7f55500700f0 2026-02-17T05:58:33.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.935+0000 7f553dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:33.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:33.935+0000 7f553dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:34.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.036+0000 7f5557a64640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:34.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.036+0000 7f5557a64640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:34.038 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.036+0000 7f5557a64640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:34.038 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.036+0000 7f5557a64640 1 -- 10.20.193.16:0/1858058375 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5550140a30 con 0x7f55500700f0 2026-02-17T05:58:34.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.050+0000 7f553dffb640 1 -- 10.20.193.16:0/1858058375 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f5540061d60 con 0x7f55500700f0 2026-02-17T05:58:34.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.050+0000 7f553dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:34.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.050+0000 7f553dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:34.051 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:34.051 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 10 librados: watch_flush enter 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 10 librados: watch_flush exit 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 20 client.4692.objecter shutdown clearing up homeless session... 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 10 client.4692.objecter successfully canceled tick 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 -- 10.20.193.16:0/1858058375 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5520079190 msgr2=0x7f552007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5520079190 0x7f552007b650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5544002770 tx=0x7f5544007ab0 comp rx=0 tx=0).stop 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 10 monclient: shutdown 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 -- 10.20.193.16:0/1858058375 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55500700f0 msgr2=0x7f55501f6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55500700f0 0x7f55501f6dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f554000c4d0 tx=0x7f554000c9a0 comp rx=0 tx=0).stop 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 -- 10.20.193.16:0/1858058375 shutdown_connections 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5520079190 0x7f552007b650 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55500c86f0 0x7f555006e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5550153e00 0x7f55501f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 --2- 10.20.193.16:0/1858058375 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55500700f0 0x7f55501f6dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 -- 10.20.193.16:0/1858058375 >> 10.20.193.16:0/1858058375 conn(0x7f55500cd770 msgr2=0x7f555014e560 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 -- 10.20.193.16:0/1858058375 shutdown_connections 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.052+0000 7f5557a64640 1 -- 10.20.193.16:0/1858058375 wait complete. 2026-02-17T05:58:34.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:34.053+0000 7f5557a64640 1 librados: shutdown 2026-02-17T05:58:37.062 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:37.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.130+0000 7f01ee50a640 10 monclient: get_monmap_and_config 2026-02-17T05:58:37.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.130+0000 7f01ee50a640 10 monclient: build_initial_monmap 2026-02-17T05:58:37.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.130+0000 7f01ee50a640 10 monclient: monmap: 2026-02-17T05:58:37.131 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:37.131 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:37.131 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:37.131168+0000 2026-02-17T05:58:37.131 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:37.131168+0000 2026-02-17T05:58:37.131 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.130+0000 7f01ee50a640 1 Processor -- start 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.130+0000 7f01ee50a640 1 -- start start 2026-02-17T05:58:37.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.130+0000 7f01ee50a640 10 monclient: init 2026-02-17T05:58:37.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:37.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 10 monclient: _add_conns ranks=[2,3,1,5,0,4] 2026-02-17T05:58:37.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f01e80700f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:58:37.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f01e80c86f0 0x7f01e80704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:37.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 10 monclient(hunting): picked mon.noname-a con 0x7f01e80c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:58:37.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f01e81536f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:58:37.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e80c4820 con 0x7f01e81536f0 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e80c4b00 con 0x7f01e80700f0 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f01e80c4de0 con 0x7f01e80c86f0 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01ee50a640 10 monclient(hunting): authenticate will time out at 1169.700135s 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01e77fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f01e80c86f0 0x7f01e80704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01e7fff640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f01e80700f0 0x7f01e80c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:60286/0 (socket says 10.20.193.16:60286) 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01e7fff640 1 -- 10.20.193.16:0/1992548067 learned_addr learned my addr 10.20.193.16:0/1992548067 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01e77fe640 10 monclient(hunting): get_auth_request con 0x7f01e80c86f0 auth_method 0 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:37.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01e77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.131+0000 7f01e77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.132+0000 7f01e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.132+0000 7f01e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.132+0000 7f01e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e6ffd640 1 -- 10.20.193.16:0/1992548067 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1433079913 0 0) 0x7f01e80c4820 con 0x7f01e81536f0 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e6ffd640 10 monclient(hunting): _negotiate 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e6ffd640 10 monclient(hunting): my global_id is 4765 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e6ffd640 1 -- 10.20.193.16:0/1992548067 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01cc003620 con 0x7f01e81536f0 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e77fe640 10 monclient(hunting): handle_auth_done global_id 4695 payload 274 2026-02-17T05:58:37.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e77fe640 1 -- 10.20.193.16:0/1992548067 >> v1:10.20.193.148:6789/0 conn(0x7f01e80700f0 legacy=0x7f01e80c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:37.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e77fe640 1 -- 10.20.193.16:0/1992548067 >> v1:10.20.193.128:6789/0 conn(0x7f01e81536f0 legacy=0x7f01e8153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:37.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e77fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:37.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e77fe640 1 monclient: found mon.noname-a 2026-02-17T05:58:37.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:37.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:37.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.133+0000 7f01e77fe640 1 -- 10.20.193.16:0/1992548067 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e80c50c0 con 0x7f01e80c86f0 2026-02-17T05:58:37.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.134+0000 7f01e77fe640 1 --2- 10.20.193.16:0/1992548067 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01e80c86f0 0x7f01e80704d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f01d80098a0 tx=0x7f01d802edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a56a83455219479 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:37.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 1 -- 10.20.193.16:0/1992548067 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d803c070 con 0x7f01e80c86f0 2026-02-17T05:58:37.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:37.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:37.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 10 monclient: dump: 2026-02-17T05:58:37.145 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:37.145 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:37.145 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:37.145 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:37.145 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:37.145 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 1 -- 10.20.193.16:0/1992548067 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d802fae0 con 0x7f01e80c86f0 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 1 -- 10.20.193.16:0/1992548067 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8037440 con 0x7f01e80c86f0 2026-02-17T05:58:37.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 5 monclient: authenticate success, global_id 4695 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 4 set_mon_vals no callback set 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01e6ffd640 10 monclient: dump: 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:37.147 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 10 monclient: shutdown 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 1 -- 10.20.193.16:0/1992548067 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01e80c86f0 msgr2=0x7f01e80704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 1 --2- 10.20.193.16:0/1992548067 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01e80c86f0 0x7f01e80704d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f01d80098a0 tx=0x7f01d802edd0 comp rx=0 tx=0).stop 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 1 -- 10.20.193.16:0/1992548067 shutdown_connections 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 1 --2- 10.20.193.16:0/1992548067 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01e80c86f0 0x7f01e80704d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 1 -- 10.20.193.16:0/1992548067 >> 10.20.193.16:0/1992548067 conn(0x7f01e80cd770 msgr2=0x7f01e80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 1 -- 10.20.193.16:0/1992548067 shutdown_connections 2026-02-17T05:58:37.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 1 -- 10.20.193.16:0/1992548067 wait complete. 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 10 monclient: build_initial_monmap 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 10 monclient: monmap: 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:37.149 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 1 librados: starting msgr at 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.136+0000 7f01ee50a640 1 librados: starting objecter 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 1 Processor -- start 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 1 -- start start 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 1 librados: setting wanted keys 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 1 librados: calling monclient init 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient: init 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01e80700f0 0x7f01e806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient(hunting): picked mon.noname-a con 0x7f01e80700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f01e80c86f0 0x7f01e81e7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient(hunting): picked mon.noname-c con 0x7f01e80c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01e81536f0 0x7f01e8152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:37.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient(hunting): picked mon.noname-b con 0x7f01e81536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8074c00 con 0x7f01e80700f0 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f01e80c5680 con 0x7f01e81536f0 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8074d80 con 0x7f01e80c86f0 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01ee50a640 10 monclient(hunting): authenticate will time out at 1169.706490s 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01e7fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01e80700f0 0x7f01e806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01e7fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01e80700f0 0x7f01e806ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:41630/0 (socket says 10.20.193.16:41630) 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01e7fff640 1 -- 10.20.193.16:0/3481057522 learned_addr learned my addr 10.20.193.16:0/3481057522 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01e7fff640 10 monclient(hunting): get_auth_request con 0x7f01e80700f0 auth_method 0 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:37.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01e7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01e7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01e77fe640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f01e80c86f0 0x7f01e81e7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.137+0000 7f01eca80640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01e81536f0 0x7f01e8152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01eca80640 10 monclient(hunting): get_auth_request con 0x7f01e81536f0 auth_method 0 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01eca80640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01eca80640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01eca80640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01e77fe640 10 monclient(hunting): get_auth_request con 0x7f01e80c86f0 auth_method 0 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01e77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01e77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01eca80640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01eca80640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:37.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01eca80640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.138+0000 7f01e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.139+0000 7f01eca80640 10 monclient(hunting): handle_auth_done global_id 4768 payload 824 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.139+0000 7f01eca80640 1 -- 10.20.193.16:0/3481057522 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f01e80c86f0 msgr2=0x7f01e81e7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.139+0000 7f01eca80640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f01e80c86f0 0x7f01e81e7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.139+0000 7f01eca80640 1 -- 10.20.193.16:0/3481057522 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01e80700f0 msgr2=0x7f01e806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.139+0000 7f01eca80640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01e80700f0 0x7f01e806ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.139+0000 7f01eca80640 10 monclient: _finish_hunting 0 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.139+0000 7f01eca80640 1 monclient: found mon.noname-b 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.139+0000 7f01eca80640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:37.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.139+0000 7f01eca80640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.139+0000 7f01eca80640 1 -- 10.20.193.16:0/3481057522 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e806d4c0 con 0x7f01e81536f0 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.140+0000 7f01eca80640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01e81536f0 0x7f01e8152700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00cb40 tx=0x7f01dc00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.140+0000 7f01e4ff9640 10 client.?.objecter ms_handle_connect 0x7f01e81536f0 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.140+0000 7f01e4ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.141+0000 7f01e4ff9640 1 -- 10.20.193.16:0/3481057522 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00ad50 con 0x7f01e81536f0 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.141+0000 7f01e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.141+0000 7f01e4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.141+0000 7f01e4ff9640 10 monclient: dump: 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:37.154 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.141+0000 7f01e4ff9640 10 monclient: _finish_auth 0 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.141+0000 7f01e4ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.141+0000 7f01e4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.141+0000 7f01ee50a640 5 monclient: authenticate success, global_id 4768 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 10 monclient: _renew_subs 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 1 -- 10.20.193.16:0/3481057522 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e806d7a0 con 0x7f01e81536f0 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 10 client.4768.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 10 monclient: _renew_subs 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 1 -- 10.20.193.16:0/3481057522 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e806dd60 con 0x7f01e81536f0 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 1 librados: init done 2026-02-17T05:58:37.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01e4ff9640 1 -- 10.20.193.16:0/3481057522 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01dc003a20 con 0x7f01e81536f0 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01ee50a640 1 -- 10.20.193.16:0/3481057522 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e80704d0 con 0x7f01e81536f0 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01e4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01e4ff9640 1 -- 10.20.193.16:0/3481057522 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc003dd0 con 0x7f01e81536f0 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01e67fc640 4 set_mon_vals no callback set 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01e4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.142+0000 7f01e4ff9640 10 monclient: dump: 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:37.156 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:37.157 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:37.157 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:37.157 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:37.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.153+0000 7f01e4ff9640 1 -- 10.20.193.16:0/3481057522 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f01dc0050d0 con 0x7f01e81536f0 2026-02-17T05:58:37.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.153+0000 7f01e4ff9640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01b8079140 0x7f01b807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:37.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.154+0000 7f01e7fff640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01b8079140 0x7f01b807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:37.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.154+0000 7f01e7fff640 10 monclient: get_auth_request con 0x7f01b8079140 auth_method 0 2026-02-17T05:58:37.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.154+0000 7f01e7fff640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01b8079140 0x7f01b807b600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01d4002770 tx=0x7f01d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:37.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.154+0000 7f01e4ff9640 10 client.4768.objecter ms_handle_connect 0x7f01b8079140 2026-02-17T05:58:37.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.250+0000 7f01e4ff9640 1 -- 10.20.193.16:0/3481057522 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f01dc05e800 con 0x7f01e81536f0 2026-02-17T05:58:37.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.250+0000 7f01e4ff9640 10 client.4768.objecter ms_dispatch2 0x7f01e8000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:37.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.250+0000 7f01e4ff9640 3 client.4768.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:37.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.250+0000 7f01e4ff9640 3 client.4768.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:37.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.250+0000 7f01e4ff9640 20 client.4768.objecter dump_active .. 0 homeless 2026-02-17T05:58:37.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.257+0000 7f01e4ff9640 1 -- 10.20.193.16:0/3481057522 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f01dc0627c0 con 0x7f01e81536f0 2026-02-17T05:58:37.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.257+0000 7f01e4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:37.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.257+0000 7f01e4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:37.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.355+0000 7f01ee50a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:37.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.355+0000 7f01ee50a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:37.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.355+0000 7f01ee50a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:37.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.355+0000 7f01ee50a640 1 -- 10.20.193.16:0/3481057522 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f01e81573f0 con 0x7f01e81536f0 2026-02-17T05:58:37.359 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.358+0000 7f01e4ff9640 1 -- 10.20.193.16:0/3481057522 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f01dc066ac0 con 0x7f01e81536f0 2026-02-17T05:58:37.359 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.358+0000 7f01e4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:37.359 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.358+0000 7f01e4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:37.359 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:37.359 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:37.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 10 librados: watch_flush enter 2026-02-17T05:58:37.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 10 librados: watch_flush exit 2026-02-17T05:58:37.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 20 client.4768.objecter shutdown clearing up homeless session... 2026-02-17T05:58:37.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 10 client.4768.objecter successfully canceled tick 2026-02-17T05:58:37.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 -- 10.20.193.16:0/3481057522 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01b8079140 msgr2=0x7f01b807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:37.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01b8079140 0x7f01b807b600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01d4002770 tx=0x7f01d4009040 comp rx=0 tx=0).stop 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 10 monclient: shutdown 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 -- 10.20.193.16:0/3481057522 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01e81536f0 msgr2=0x7f01e8152700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01e81536f0 0x7f01e8152700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00cb40 tx=0x7f01dc00a430 comp rx=0 tx=0).stop 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 -- 10.20.193.16:0/3481057522 shutdown_connections 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f01b8079140 0x7f01b807b600 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01e81536f0 0x7f01e8152700 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f01e80c86f0 0x7f01e81e7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 --2- 10.20.193.16:0/3481057522 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01e80700f0 0x7f01e806ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 -- 10.20.193.16:0/3481057522 >> 10.20.193.16:0/3481057522 conn(0x7f01e80cd770 msgr2=0x7f01e814dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:37.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 -- 10.20.193.16:0/3481057522 shutdown_connections 2026-02-17T05:58:37.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 -- 10.20.193.16:0/3481057522 wait complete. 2026-02-17T05:58:37.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:37.359+0000 7f01ee50a640 1 librados: shutdown 2026-02-17T05:58:40.368 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:40.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.441+0000 7f0be1c91640 10 monclient: get_monmap_and_config 2026-02-17T05:58:40.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.441+0000 7f0be1c91640 10 monclient: build_initial_monmap 2026-02-17T05:58:40.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.441+0000 7f0be1c91640 10 monclient: monmap: 2026-02-17T05:58:40.444 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:40.444 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:40.444 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:40.442330+0000 2026-02-17T05:58:40.444 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:40.442330+0000 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.441+0000 7f0be1c91640 1 Processor -- start 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.441+0000 7f0be1c91640 1 -- start start 2026-02-17T05:58:40.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.441+0000 7f0be1c91640 10 monclient: init 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 10 monclient: _add_conns ranks=[0,5,3,1,2,4] 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0bdc14f020 addr v1:10.20.193.16:6789/0 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f0bdc14f400 0x7f0bdc15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 10 monclient(hunting): picked mon.noname-c con 0x7f0bdc14f400 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f0bdc15c920 0x7f0bdc0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 10 monclient(hunting): picked mon.noname-a con 0x7f0bdc15c920 addr v2:10.20.193.16:3300/0 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bdc0c5b10 con 0x7f0bdc14f020 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0bdc15cd20 con 0x7f0bdc15c920 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:40.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f0bdc06e620 con 0x7f0bdc14f400 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0be1c91640 10 monclient(hunting): authenticate will time out at 1173.011310s 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0bdbfff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f0bdc15c920 0x7f0bdc0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0bdaffd640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f0bdc14f400 0x7f0bdc15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0bdbfff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f0bdc15c920 0x7f0bdc0c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:49470/0 (socket says 10.20.193.16:49470) 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0bdbfff640 1 -- 10.20.193.16:0/578241087 learned_addr learned my addr 10.20.193.16:0/578241087 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0bdbfff640 10 monclient(hunting): get_auth_request con 0x7f0bdc15c920 auth_method 0 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0bdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0bdbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.442+0000 7f0bdbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.443+0000 7f0bdaffd640 10 monclient(hunting): get_auth_request con 0x7f0bdc14f400 auth_method 0 2026-02-17T05:58:40.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.443+0000 7f0bdaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:40.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.443+0000 7f0bdaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:40.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.443+0000 7f0bdaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:40.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.443+0000 7f0bdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:40.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.443+0000 7f0bdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:40.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.443+0000 7f0bdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:40.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.443+0000 7f0bdaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:40.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.443+0000 7f0bdaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:40.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.443+0000 7f0bdaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:40.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.444+0000 7f0bdbfff640 10 monclient(hunting): handle_auth_done global_id 4701 payload 274 2026-02-17T05:58:40.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.444+0000 7f0bdbfff640 1 -- 10.20.193.16:0/578241087 >> v2:10.20.193.148:3300/0 conn(0x7f0bdc14f400 msgr2=0x7f0bdc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:40.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.444+0000 7f0bdbfff640 1 --2- 10.20.193.16:0/578241087 >> v2:10.20.193.148:3300/0 conn(0x7f0bdc14f400 0x7f0bdc15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.444+0000 7f0bdbfff640 1 -- 10.20.193.16:0/578241087 >> v1:10.20.193.16:6789/0 conn(0x7f0bdc14f020 legacy=0x7f0bdc06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:40.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.444+0000 7f0bdbfff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:40.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.444+0000 7f0bdbfff640 1 monclient: found mon.noname-a 2026-02-17T05:58:40.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.444+0000 7f0bdbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:40.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.444+0000 7f0bdbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:40.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.444+0000 7f0bdbfff640 1 -- 10.20.193.16:0/578241087 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc06e900 con 0x7f0bdc15c920 2026-02-17T05:58:40.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.445+0000 7f0bdbfff640 1 --2- 10.20.193.16:0/578241087 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bdc15c920 0x7f0bdc0c6010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc156400 tx=0x7f0bd002f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39bf98a1bca74e7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:40.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 1 -- 10.20.193.16:0/578241087 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd002fd70 con 0x7f0bdc15c920 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 10 monclient: dump: 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:40.455 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 5 monclient: authenticate success, global_id 4701 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 1 -- 10.20.193.16:0/578241087 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd00373f0 con 0x7f0bdc15c920 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 1 -- 10.20.193.16:0/578241087 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00377c0 con 0x7f0bdc15c920 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:40.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 4 set_mon_vals no callback set 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0bda7fc640 10 monclient: dump: 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:40.457 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 10 monclient: shutdown 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 1 -- 10.20.193.16:0/578241087 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bdc15c920 msgr2=0x7f0bdc0c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 1 --2- 10.20.193.16:0/578241087 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bdc15c920 0x7f0bdc0c6010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc156400 tx=0x7f0bd002f310 comp rx=0 tx=0).stop 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 1 -- 10.20.193.16:0/578241087 shutdown_connections 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 1 --2- 10.20.193.16:0/578241087 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bdc15c920 0x7f0bdc0c6010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.447+0000 7f0be1c91640 1 --2- 10.20.193.16:0/578241087 >> v2:10.20.193.148:3300/0 conn(0x7f0bdc14f400 0x7f0bdc15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.448+0000 7f0be1c91640 1 -- 10.20.193.16:0/578241087 >> 10.20.193.16:0/578241087 conn(0x7f0bdc0cd770 msgr2=0x7f0bdc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.448+0000 7f0be1c91640 1 -- 10.20.193.16:0/578241087 shutdown_connections 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.448+0000 7f0be1c91640 1 -- 10.20.193.16:0/578241087 wait complete. 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.448+0000 7f0be1c91640 10 monclient: build_initial_monmap 2026-02-17T05:58:40.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.448+0000 7f0be1c91640 10 monclient: monmap: 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 librados: starting msgr at 2026-02-17T05:58:40.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 librados: starting objecter 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 Processor -- start 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 -- start start 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 librados: setting wanted keys 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 librados: calling monclient init 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient: init 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bdc14f020 0x7f0bdc15b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient(hunting): picked mon.noname-b con 0x7f0bdc14f020 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bdc14f400 0x7f0bdc1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient(hunting): picked mon.noname-c con 0x7f0bdc14f400 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bdc15c920 0x7f0bdc14aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:40.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient(hunting): picked mon.noname-a con 0x7f0bdc15c920 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc06f1a0 con 0x7f0bdc15c920 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc06eec0 con 0x7f0bdc14f020 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc06e2f0 con 0x7f0bdc14f400 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0be1c91640 10 monclient(hunting): authenticate will time out at 1173.018530s 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0bdbfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bdc15c920 0x7f0bdc14aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0bdbfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bdc15c920 0x7f0bdc14aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:49474/0 (socket says 10.20.193.16:49474) 2026-02-17T05:58:40.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0bdbfff640 1 -- 10.20.193.16:0/2728521283 learned_addr learned my addr 10.20.193.16:0/2728521283 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0bdbfff640 10 monclient(hunting): get_auth_request con 0x7f0bdc15c920 auth_method 0 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0bdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0bdbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.449+0000 7f0bdbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdaffd640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bdc14f400 0x7f0bdc1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdb7fe640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bdc14f020 0x7f0bdc15b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdaffd640 10 monclient(hunting): get_auth_request con 0x7f0bdc14f400 auth_method 0 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:40.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdb7fe640 10 monclient(hunting): get_auth_request con 0x7f0bdc14f020 auth_method 0 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.450+0000 7f0bdb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:40.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.451+0000 7f0bdaffd640 10 monclient(hunting): handle_auth_done global_id 4835 payload 824 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.451+0000 7f0bdaffd640 1 -- 10.20.193.16:0/2728521283 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bdc14f020 msgr2=0x7f0bdc15b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.451+0000 7f0bdaffd640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bdc14f020 0x7f0bdc15b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.451+0000 7f0bdaffd640 1 -- 10.20.193.16:0/2728521283 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bdc15c920 msgr2=0x7f0bdc14aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.451+0000 7f0bdaffd640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bdc15c920 0x7f0bdc14aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.451+0000 7f0bdaffd640 10 monclient: _finish_hunting 0 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.451+0000 7f0bdaffd640 1 monclient: found mon.noname-c 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.451+0000 7f0bdaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.451+0000 7f0bdaffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.451+0000 7f0bdaffd640 1 -- 10.20.193.16:0/2728521283 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc159da0 con 0x7f0bdc14f400 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.453+0000 7f0bdaffd640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bdc14f400 0x7f0bdc1f5050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00e9e0 tx=0x7f0bcc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.453+0000 7f0bb7fff640 10 client.?.objecter ms_handle_connect 0x7f0bdc14f400 2026-02-17T05:58:40.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.453+0000 7f0bb7fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 1 -- 10.20.193.16:0/2728521283 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc017070 con 0x7f0bdc14f400 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 10 monclient: dump: 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:40.467 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:40.468 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:40.468 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:40.468 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:40.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 10 monclient: _finish_auth 0 2026-02-17T05:58:40.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:40.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:40.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0be1c91640 5 monclient: authenticate success, global_id 4835 2026-02-17T05:58:40.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0be1c91640 10 monclient: _renew_subs 2026-02-17T05:58:40.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0be1c91640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0be1c91640 1 -- 10.20.193.16:0/2728521283 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc15a080 con 0x7f0bdc14f400 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0be1c91640 10 client.4835.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0be1c91640 10 monclient: _renew_subs 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0be1c91640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0be1c91640 1 -- 10.20.193.16:0/2728521283 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc15a640 con 0x7f0bdc14f400 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0be1c91640 1 librados: init done 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 1 -- 10.20.193.16:0/2728521283 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bcc005a90 con 0x7f0bdc14f400 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 1 -- 10.20.193.16:0/2728521283 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc021060 con 0x7f0bdc14f400 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:40.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bd9ffb640 4 set_mon_vals no callback set 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0bb7fff640 10 monclient: dump: 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.454+0000 7f0be1c91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:40.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.455+0000 7f0be1c91640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:40.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.455+0000 7f0be1c91640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:40.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.455+0000 7f0be1c91640 1 -- 10.20.193.16:0/2728521283 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc0c5230 con 0x7f0bdc14f400 2026-02-17T05:58:40.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.466+0000 7f0bb7fff640 1 -- 10.20.193.16:0/2728521283 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f0bcc024060 con 0x7f0bdc14f400 2026-02-17T05:58:40.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.467+0000 7f0bb7fff640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0bac079140 0x7f0bac07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:40.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.467+0000 7f0bdb7fe640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0bac079140 0x7f0bac07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:40.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.467+0000 7f0bdb7fe640 10 monclient: get_auth_request con 0x7f0bac079140 auth_method 0 2026-02-17T05:58:40.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.467+0000 7f0bdb7fe640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0bac079140 0x7f0bac07b600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80046e0 tx=0x7f0bc80091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:40.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.467+0000 7f0bb7fff640 10 client.4835.objecter ms_handle_connect 0x7f0bac079140 2026-02-17T05:58:40.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.468+0000 7f0bb7fff640 1 -- 10.20.193.16:0/2728521283 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f0bcc05e020 con 0x7f0bdc14f400 2026-02-17T05:58:40.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.468+0000 7f0bb7fff640 10 client.4835.objecter ms_dispatch2 0x7f0bdc000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:40.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.468+0000 7f0bb7fff640 3 client.4835.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:40.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.468+0000 7f0bb7fff640 3 client.4835.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:40.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.468+0000 7f0bb7fff640 20 client.4835.objecter dump_active .. 0 homeless 2026-02-17T05:58:40.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.571+0000 7f0bb7fff640 1 -- 10.20.193.16:0/2728521283 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0bdc0c5230 con 0x7f0bdc14f400 2026-02-17T05:58:40.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.571+0000 7f0bb7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:40.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.571+0000 7f0bb7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:40.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.671+0000 7f0be1c91640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:40.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.671+0000 7f0be1c91640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:40.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.671+0000 7f0be1c91640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:40.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.671+0000 7f0be1c91640 1 -- 10.20.193.16:0/2728521283 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0bdc15c3e0 con 0x7f0bdc14f400 2026-02-17T05:58:40.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.674+0000 7f0bb7fff640 1 -- 10.20.193.16:0/2728521283 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f0bcc061fe0 con 0x7f0bdc14f400 2026-02-17T05:58:40.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.674+0000 7f0bb7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:40.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.674+0000 7f0bb7fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:40.675 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:40.675 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:40.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 10 librados: watch_flush enter 2026-02-17T05:58:40.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 10 librados: watch_flush exit 2026-02-17T05:58:40.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 20 client.4835.objecter shutdown clearing up homeless session... 2026-02-17T05:58:40.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 10 client.4835.objecter successfully canceled tick 2026-02-17T05:58:40.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 -- 10.20.193.16:0/2728521283 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0bac079140 msgr2=0x7f0bac07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:40.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0bac079140 0x7f0bac07b600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80046e0 tx=0x7f0bc80091c0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 10 monclient: shutdown 2026-02-17T05:58:40.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:40.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 -- 10.20.193.16:0/2728521283 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bdc14f400 msgr2=0x7f0bdc1f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:40.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bdc14f400 0x7f0bdc1f5050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00e9e0 tx=0x7f0bcc00eeb0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 -- 10.20.193.16:0/2728521283 shutdown_connections 2026-02-17T05:58:40.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0bac079140 0x7f0bac07b600 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0bdc15c920 0x7f0bdc14aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0bdc14f400 0x7f0bdc1f5050 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 --2- 10.20.193.16:0/2728521283 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0bdc14f020 0x7f0bdc15b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:40.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 -- 10.20.193.16:0/2728521283 >> 10.20.193.16:0/2728521283 conn(0x7f0bdc0cd770 msgr2=0x7f0bdc14d420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:40.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 -- 10.20.193.16:0/2728521283 shutdown_connections 2026-02-17T05:58:40.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 -- 10.20.193.16:0/2728521283 wait complete. 2026-02-17T05:58:40.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:40.675+0000 7f0be1c91640 1 librados: shutdown 2026-02-17T05:58:43.685 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:43.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.751+0000 7fec42096640 10 monclient: get_monmap_and_config 2026-02-17T05:58:43.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.751+0000 7fec42096640 10 monclient: build_initial_monmap 2026-02-17T05:58:43.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.751+0000 7fec42096640 10 monclient: monmap: 2026-02-17T05:58:43.752 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:43.752743+0000 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:43.752743+0000 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:43.753 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:43.754 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:43.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 1 Processor -- start 2026-02-17T05:58:43.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 1 -- start start 2026-02-17T05:58:43.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 10 monclient: init 2026-02-17T05:58:43.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:43.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 10 monclient: _add_conns ranks=[5,4,1,0,2,3] 2026-02-17T05:58:43.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fec3c0700f0 0x7fec3c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:43.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 10 monclient(hunting): picked mon.noname-c con 0x7fec3c0700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:43.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fec3c0c7fe0 0x7fec3c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:43.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 10 monclient(hunting): picked mon.noname-b con 0x7fec3c0c7fe0 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:43.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fec3c1536f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:58:43.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec3c0c8970 con 0x7fec3c1536f0 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fec3c0c4820 con 0x7fec3c0c7fe0 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fec3c0c4b00 con 0x7fec3c0700f0 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.752+0000 7fec42096640 10 monclient(hunting): authenticate will time out at 1176.321806s 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3b7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fec3c0700f0 0x7fec3c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3bfff640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fec3c1536f0 0x7fec3c153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:36722/0 (socket says 10.20.193.16:36722) 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3bfff640 1 -- 10.20.193.16:0/248053038 learned_addr learned my addr 10.20.193.16:0/248053038 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3b7fe640 10 monclient(hunting): get_auth_request con 0x7fec3c0700f0 auth_method 0 2026-02-17T05:58:43.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3affd640 1 --2- 10.20.193.16:0/248053038 >> v2:10.20.193.128:3300/0 conn(0x7fec3c0c7fe0 0x7fec3c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.753+0000 7fec3b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.754+0000 7fec3affd640 10 monclient(hunting): get_auth_request con 0x7fec3c0c7fe0 auth_method 0 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.754+0000 7fec3affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.754+0000 7fec3affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.754+0000 7fec3affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.755+0000 7fec3b7fe640 10 monclient(hunting): handle_auth_done global_id 4838 payload 274 2026-02-17T05:58:43.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.755+0000 7fec3b7fe640 1 -- 10.20.193.16:0/248053038 >> v2:10.20.193.128:3300/0 conn(0x7fec3c0c7fe0 msgr2=0x7fec3c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:43.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.755+0000 7fec3b7fe640 1 --2- 10.20.193.16:0/248053038 >> v2:10.20.193.128:3300/0 conn(0x7fec3c0c7fe0 0x7fec3c0c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:43.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.755+0000 7fec3b7fe640 1 -- 10.20.193.16:0/248053038 >> v1:10.20.193.128:6789/0 conn(0x7fec3c1536f0 legacy=0x7fec3c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:43.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.755+0000 7fec3b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:43.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.755+0000 7fec3b7fe640 1 monclient: found mon.noname-c 2026-02-17T05:58:43.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.755+0000 7fec3b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:43.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.755+0000 7fec3b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:43.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.755+0000 7fec3b7fe640 1 -- 10.20.193.16:0/248053038 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c0c4de0 con 0x7fec3c0700f0 2026-02-17T05:58:43.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.756+0000 7fec3b7fe640 1 --2- 10.20.193.16:0/248053038 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 0x7fec3c0704d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec3c070b90 tx=0x7fec2802ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84ef5f94eb1cf253 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:43.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 1 -- 10.20.193.16:0/248053038 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2803c070 con 0x7fec3c0700f0 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 10 monclient: dump: 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:43.768 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 5 monclient: authenticate success, global_id 4838 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 1 -- 10.20.193.16:0/248053038 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2802fa60 con 0x7fec3c0700f0 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 1 -- 10.20.193.16:0/248053038 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2802fe30 con 0x7fec3c0700f0 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:43.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec3a7fc640 10 monclient: dump: 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:43.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 4 set_mon_vals no callback set 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 10 monclient: shutdown 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 1 -- 10.20.193.16:0/248053038 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 msgr2=0x7fec3c0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 1 --2- 10.20.193.16:0/248053038 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 0x7fec3c0704d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec3c070b90 tx=0x7fec2802ed50 comp rx=0 tx=0).stop 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 1 -- 10.20.193.16:0/248053038 shutdown_connections 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 1 --2- 10.20.193.16:0/248053038 >> v2:10.20.193.128:3300/0 conn(0x7fec3c0c7fe0 0x7fec3c0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 1 --2- 10.20.193.16:0/248053038 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 0x7fec3c0704d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 1 -- 10.20.193.16:0/248053038 >> 10.20.193.16:0/248053038 conn(0x7fec3c0cd770 msgr2=0x7fec3c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 1 -- 10.20.193.16:0/248053038 shutdown_connections 2026-02-17T05:58:43.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.758+0000 7fec42096640 1 -- 10.20.193.16:0/248053038 wait complete. 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient: build_initial_monmap 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient: monmap: 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:43.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 librados: starting msgr at 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 librados: starting objecter 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 Processor -- start 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 -- start start 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 librados: setting wanted keys 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 librados: calling monclient init 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient: init 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 0x7fec3c0ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient(hunting): picked mon.noname-c con 0x7fec3c0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fec3c0c7fe0 0x7fec3c13e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:43.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient(hunting): picked mon.noname-a con 0x7fec3c0c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fec3c1536f0 0x7fec3c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient(hunting): picked mon.noname-b con 0x7fec3c1536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c0c5680 con 0x7fec3c0c7fe0 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c0c53a0 con 0x7fec3c1536f0 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c074c00 con 0x7fec3c0700f0 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.759+0000 7fec42096640 10 monclient(hunting): authenticate will time out at 1176.328817s 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3b7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 0x7fec3c0ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:43.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3affd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fec3c0c7fe0 0x7fec3c13e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:43.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3b7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 0x7fec3c0ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:33920/0 (socket says 10.20.193.16:33920) 2026-02-17T05:58:43.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3b7fe640 1 -- 10.20.193.16:0/2455419070 learned_addr learned my addr 10.20.193.16:0/2455419070 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:43.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3b7fe640 10 monclient(hunting): get_auth_request con 0x7fec3c0700f0 auth_method 0 2026-02-17T05:58:43.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:43.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:43.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:43.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3bfff640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fec3c1536f0 0x7fec3c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:43.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3affd640 10 monclient(hunting): get_auth_request con 0x7fec3c0c7fe0 auth_method 0 2026-02-17T05:58:43.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3bfff640 10 monclient(hunting): get_auth_request con 0x7fec3c1536f0 auth_method 0 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:43.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.760+0000 7fec3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.761+0000 7fec3b7fe640 10 monclient(hunting): handle_auth_done global_id 4841 payload 824 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.761+0000 7fec3b7fe640 1 -- 10.20.193.16:0/2455419070 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fec3c1536f0 msgr2=0x7fec3c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.761+0000 7fec3b7fe640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fec3c1536f0 0x7fec3c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.761+0000 7fec3b7fe640 1 -- 10.20.193.16:0/2455419070 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fec3c0c7fe0 msgr2=0x7fec3c13e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.761+0000 7fec3b7fe640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fec3c0c7fe0 0x7fec3c13e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.761+0000 7fec3b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:43.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.761+0000 7fec3b7fe640 1 monclient: found mon.noname-c 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.761+0000 7fec3b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.761+0000 7fec3b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.761+0000 7fec3b7fe640 1 -- 10.20.193.16:0/2455419070 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c1f66f0 con 0x7fec3c0700f0 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.763+0000 7fec3b7fe640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 0x7fec3c0ccce0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec28009130 tx=0x7fec280047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.763+0000 7fec17fff640 10 client.?.objecter ms_handle_connect 0x7fec3c0700f0 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.763+0000 7fec17fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 1 -- 10.20.193.16:0/2455419070 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2803c040 con 0x7fec3c0700f0 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 10 monclient: dump: 2026-02-17T05:58:43.778 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 10 monclient: _finish_auth 0 2026-02-17T05:58:43.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 5 monclient: authenticate success, global_id 4841 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 10 monclient: _renew_subs 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 1 -- 10.20.193.16:0/2455419070 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c06d3a0 con 0x7fec3c0700f0 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 10 client.4841.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 10 monclient: _renew_subs 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 1 -- 10.20.193.16:0/2455419070 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c06d960 con 0x7fec3c0700f0 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 1 librados: init done 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:43.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec42096640 1 -- 10.20.193.16:0/2455419070 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c0704d0 con 0x7fec3c0700f0 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 1 -- 10.20.193.16:0/2455419070 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2802fa60 con 0x7fec3c0700f0 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 1 -- 10.20.193.16:0/2455419070 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28042650 con 0x7fec3c0700f0 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec17fff640 10 monclient: dump: 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:43.781 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:43.782 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:43.782 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:43.782 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:43.782 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:43.782 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:43.782 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:43.782 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:43.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.764+0000 7fec39ffb640 4 set_mon_vals no callback set 2026-02-17T05:58:43.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.872+0000 7fec17fff640 1 -- 10.20.193.16:0/2455419070 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fec280427f0 con 0x7fec3c0700f0 2026-02-17T05:58:43.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.873+0000 7fec17fff640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fec0c079140 0x7fec0c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:43.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.873+0000 7fec3affd640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fec0c079140 0x7fec0c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:43.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.873+0000 7fec3affd640 10 monclient: get_auth_request con 0x7fec0c079140 auth_method 0 2026-02-17T05:58:43.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.873+0000 7fec17fff640 10 client.4841.objecter ms_handle_connect 0x7fec0c079140 2026-02-17T05:58:43.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.873+0000 7fec3affd640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fec0c079140 0x7fec0c07b600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fec3c06f660 tx=0x7fec2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:43.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.874+0000 7fec17fff640 1 -- 10.20.193.16:0/2455419070 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fec28083920 con 0x7fec3c0700f0 2026-02-17T05:58:43.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.874+0000 7fec17fff640 10 client.4841.objecter ms_dispatch2 0x7fec3c000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:43.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.874+0000 7fec17fff640 3 client.4841.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:43.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.874+0000 7fec17fff640 3 client.4841.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:43.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.874+0000 7fec17fff640 20 client.4841.objecter dump_active .. 0 homeless 2026-02-17T05:58:43.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.887+0000 7fec17fff640 1 -- 10.20.193.16:0/2455419070 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fec3c1f66f0 con 0x7fec3c0700f0 2026-02-17T05:58:43.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.887+0000 7fec17fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:43.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.887+0000 7fec17fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:43.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.988+0000 7fec42096640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:43.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.988+0000 7fec42096640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:43.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.988+0000 7fec42096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:43.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.988+0000 7fec42096640 1 -- 10.20.193.16:0/2455419070 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fec3c1573f0 con 0x7fec3c0700f0 2026-02-17T05:58:43.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.991+0000 7fec17fff640 1 -- 10.20.193.16:0/2455419070 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fec280878e0 con 0x7fec3c0700f0 2026-02-17T05:58:43.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.991+0000 7fec17fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:43.992 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.991+0000 7fec17fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:43.992 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:43.992 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:43.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.992+0000 7fec42096640 10 librados: watch_flush enter 2026-02-17T05:58:43.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 10 librados: watch_flush exit 2026-02-17T05:58:43.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 20 client.4841.objecter shutdown clearing up homeless session... 2026-02-17T05:58:43.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 10 client.4841.objecter successfully canceled tick 2026-02-17T05:58:43.994 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 -- 10.20.193.16:0/2455419070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fec0c079140 msgr2=0x7fec0c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fec0c079140 0x7fec0c07b600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fec3c06f660 tx=0x7fec2c008040 comp rx=0 tx=0).stop 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 10 monclient: shutdown 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 -- 10.20.193.16:0/2455419070 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 msgr2=0x7fec3c0ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 0x7fec3c0ccce0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec28009130 tx=0x7fec280047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 -- 10.20.193.16:0/2455419070 shutdown_connections 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fec0c079140 0x7fec0c07b600 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fec3c1536f0 0x7fec3c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fec3c0c7fe0 0x7fec3c13e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 --2- 10.20.193.16:0/2455419070 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fec3c0700f0 0x7fec3c0ccce0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 -- 10.20.193.16:0/2455419070 >> 10.20.193.16:0/2455419070 conn(0x7fec3c0cd770 msgr2=0x7fec3c0c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:43.995 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 -- 10.20.193.16:0/2455419070 shutdown_connections 2026-02-17T05:58:43.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 -- 10.20.193.16:0/2455419070 wait complete. 2026-02-17T05:58:43.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:43.993+0000 7fec42096640 1 librados: shutdown 2026-02-17T05:58:47.002 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:47.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.068+0000 7f3ae62f1640 10 monclient: get_monmap_and_config 2026-02-17T05:58:47.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.068+0000 7f3ae62f1640 10 monclient: build_initial_monmap 2026-02-17T05:58:47.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.068+0000 7f3ae62f1640 10 monclient: monmap: 2026-02-17T05:58:47.070 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:47.070 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:47.070 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:47.069773+0000 2026-02-17T05:58:47.070 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:47.069773+0000 2026-02-17T05:58:47.070 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:47.070 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:47.070 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:47.071 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:47.071 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:47.071 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:47.071 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:47.071 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:47.071 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:47.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 1 Processor -- start 2026-02-17T05:58:47.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 1 -- start start 2026-02-17T05:58:47.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 10 monclient: init 2026-02-17T05:58:47.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 10 monclient: _add_conns ranks=[0,1,3,2,4,5] 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3ae0155e00 addr v1:10.20.193.16:6789/0 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3ae01561e0 addr v1:10.20.193.128:6789/0 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f3ae015fa10 0x7f3ae00ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 10 monclient(hunting): picked mon.noname-a con 0x7f3ae015fa10 addr v2:10.20.193.16:3300/0 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae00c96d0 con 0x7f3ae0155e00 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae0074860 con 0x7f3ae01561e0 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae00c9940 con 0x7f3ae015fa10 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:47.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.069+0000 7f3ae62f1640 10 monclient(hunting): authenticate will time out at 1179.638992s 2026-02-17T05:58:47.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.070+0000 7f3ae4867640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f3ae015fa10 0x7f3ae00ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.070+0000 7f3adf7fe640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f3ae01561e0 0x7f3ae015f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:36724/0 (socket says 10.20.193.16:36724) 2026-02-17T05:58:47.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.070+0000 7f3adf7fe640 1 -- 10.20.193.16:0/1222734659 learned_addr learned my addr 10.20.193.16:0/1222734659 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:58:47.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.070+0000 7f3ae4867640 10 monclient(hunting): get_auth_request con 0x7f3ae015fa10 auth_method 0 2026-02-17T05:58:47.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.070+0000 7f3ae4867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:47.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.070+0000 7f3ae4867640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:47.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.070+0000 7f3ae4867640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:47.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.070+0000 7f3ae4867640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:47.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.070+0000 7f3ae4867640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:47.076 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.070+0000 7f3ae4867640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.071+0000 7f3ae4867640 10 monclient(hunting): handle_auth_done global_id 4710 payload 274 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.071+0000 7f3ae4867640 1 -- 10.20.193.16:0/1222734659 >> v1:10.20.193.128:6789/0 conn(0x7f3ae01561e0 legacy=0x7f3ae015f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.071+0000 7f3ae4867640 1 -- 10.20.193.16:0/1222734659 >> v1:10.20.193.16:6789/0 conn(0x7f3ae0155e00 legacy=0x7f3ae0074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.071+0000 7f3ae4867640 10 monclient: _finish_hunting 0 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.071+0000 7f3ae4867640 1 monclient: found mon.noname-a 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.071+0000 7f3ae4867640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.072+0000 7f3ae4867640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.072+0000 7f3ae4867640 1 -- 10.20.193.16:0/1222734659 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae00c9c20 con 0x7f3ae015fa10 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.071+0000 7f3adeffd640 1 -- 10.20.193.16:0/1222734659 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3249351037 0 0) 0x7f3ad0002920 con 0x7f3ae01561e0 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.072+0000 7f3adeffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T05:58:47.077 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.072+0000 7f3ae4867640 1 --2- 10.20.193.16:0/1222734659 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae015fa10 0x7f3ae00ca700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009d90 tx=0x7f3ad402f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44ac117fe6e72c6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:47.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 1 -- 10.20.193.16:0/1222734659 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad403c070 con 0x7f3ae015fa10 2026-02-17T05:58:47.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:47.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:47.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 10 monclient: dump: 2026-02-17T05:58:47.085 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:47.085 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:47.085 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:47.085 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:47.085 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:47.085 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:47.085 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:47.086 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:47.086 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:47.086 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:47.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 10 monclient: _finish_auth 0 2026-02-17T05:58:47.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:47.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:47.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 1 -- 10.20.193.16:0/1222734659 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad402fe10 con 0x7f3ae015fa10 2026-02-17T05:58:47.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 5 monclient: authenticate success, global_id 4710 2026-02-17T05:58:47.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:47.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:47.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 1 -- 10.20.193.16:0/1222734659 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4037660 con 0x7f3ae015fa10 2026-02-17T05:58:47.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:47.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 4 set_mon_vals no callback set 2026-02-17T05:58:47.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:47.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:47.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3adeffd640 10 monclient: dump: 2026-02-17T05:58:47.087 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:47.087 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:47.087 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:47.087 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:47.088 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:47.088 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:47.088 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:47.088 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:47.088 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:47.088 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:47.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 10 monclient: shutdown 2026-02-17T05:58:47.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:47.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 1 -- 10.20.193.16:0/1222734659 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae015fa10 msgr2=0x7f3ae00ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:47.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 1 --2- 10.20.193.16:0/1222734659 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae015fa10 0x7f3ae00ca700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009d90 tx=0x7f3ad402f190 comp rx=0 tx=0).stop 2026-02-17T05:58:47.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 1 -- 10.20.193.16:0/1222734659 shutdown_connections 2026-02-17T05:58:47.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 1 --2- 10.20.193.16:0/1222734659 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae015fa10 0x7f3ae00ca700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:47.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 1 -- 10.20.193.16:0/1222734659 >> 10.20.193.16:0/1222734659 conn(0x7f3ae00ce6c0 msgr2=0x7f3ae00cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:47.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 1 -- 10.20.193.16:0/1222734659 shutdown_connections 2026-02-17T05:58:47.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.075+0000 7f3ae62f1640 1 -- 10.20.193.16:0/1222734659 wait complete. 2026-02-17T05:58:47.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient: build_initial_monmap 2026-02-17T05:58:47.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient: monmap: 2026-02-17T05:58:47.089 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:47.089 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:47.089 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 librados: starting msgr at 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 librados: starting objecter 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 Processor -- start 2026-02-17T05:58:47.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 -- start start 2026-02-17T05:58:47.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 librados: setting wanted keys 2026-02-17T05:58:47.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 librados: calling monclient init 2026-02-17T05:58:47.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient: init 2026-02-17T05:58:47.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:47.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:58:47.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae0155e00 0x7f3ae014f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:47.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient(hunting): picked mon.noname-a con 0x7f3ae0155e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:47.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ae01561e0 0x7f3ae015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:47.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient(hunting): picked mon.noname-b con 0x7f3ae01561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:47.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3ae015fa10 0x7f3ae01f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:47.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient(hunting): picked mon.noname-c con 0x7f3ae015fa10 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:47.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:47.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae00c9360 con 0x7f3ae0155e00 2026-02-17T05:58:47.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:47.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae00ca1e0 con 0x7f3ae01561e0 2026-02-17T05:58:47.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:47.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0074dc0 con 0x7f3ae015fa10 2026-02-17T05:58:47.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:47.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.076+0000 7f3ae62f1640 10 monclient(hunting): authenticate will time out at 1179.645983s 2026-02-17T05:58:47.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adffff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae0155e00 0x7f3ae014f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adffff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae0155e00 0x7f3ae014f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:49490/0 (socket says 10.20.193.16:49490) 2026-02-17T05:58:47.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adffff640 1 -- 10.20.193.16:0/3138586960 learned_addr learned my addr 10.20.193.16:0/3138586960 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:47.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adffff640 10 monclient(hunting): get_auth_request con 0x7f3ae0155e00 auth_method 0 2026-02-17T05:58:47.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:47.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:47.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:47.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adf7fe640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ae01561e0 0x7f3ae015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3ae4867640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3ae015fa10 0x7f3ae01f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:47.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:47.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:47.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adf7fe640 10 monclient(hunting): get_auth_request con 0x7f3ae01561e0 auth_method 0 2026-02-17T05:58:47.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:47.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:47.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:47.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3ae4867640 10 monclient(hunting): get_auth_request con 0x7f3ae015fa10 auth_method 0 2026-02-17T05:58:47.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3ae4867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:47.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3ae4867640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:47.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3ae4867640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:47.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:47.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:47.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.077+0000 7f3adf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:47.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.078+0000 7f3ae4867640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:47.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.078+0000 7f3ae4867640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:47.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.078+0000 7f3ae4867640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:47.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.078+0000 7f3adffff640 10 monclient(hunting): handle_auth_done global_id 4713 payload 824 2026-02-17T05:58:47.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.079+0000 7f3adffff640 1 -- 10.20.193.16:0/3138586960 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3ae015fa10 msgr2=0x7f3ae01f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:47.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.079+0000 7f3adffff640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3ae015fa10 0x7f3ae01f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:47.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.079+0000 7f3adffff640 1 -- 10.20.193.16:0/3138586960 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ae01561e0 msgr2=0x7f3ae015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:47.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.079+0000 7f3adffff640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ae01561e0 0x7f3ae015d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.079+0000 7f3adffff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.079+0000 7f3adffff640 1 monclient: found mon.noname-a 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.079+0000 7f3adffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.079+0000 7f3adffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.079+0000 7f3adffff640 1 -- 10.20.193.16:0/3138586960 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae0148170 con 0x7f3ae0155e00 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.080+0000 7f3adffff640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae0155e00 0x7f3ae014f0b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00b570 tx=0x7f3acc00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.080+0000 7f3adcff9640 10 client.?.objecter ms_handle_connect 0x7f3ae0155e00 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.080+0000 7f3adcff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 1 -- 10.20.193.16:0/3138586960 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc013020 con 0x7f3ae0155e00 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:47.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:47.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 10 monclient: dump: 2026-02-17T05:58:47.097 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:47.097 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:47.097 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:47.097 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:47.097 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:47.097 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:47.097 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:47.097 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:47.097 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:47.098 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:47.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 10 monclient: _finish_auth 0 2026-02-17T05:58:47.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 10 monclient: _check_auth_tickets 2026-02-17T05:58:47.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:47.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 1 -- 10.20.193.16:0/3138586960 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3acc0044a0 con 0x7f3ae0155e00 2026-02-17T05:58:47.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3ae62f1640 5 monclient: authenticate success, global_id 4713 2026-02-17T05:58:47.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3ae62f1640 10 monclient: _renew_subs 2026-02-17T05:58:47.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3ae62f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:47.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3ae62f1640 1 -- 10.20.193.16:0/3138586960 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae015d7f0 con 0x7f3ae0155e00 2026-02-17T05:58:47.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:47.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 1 -- 10.20.193.16:0/3138586960 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0029c0 con 0x7f3ae0155e00 2026-02-17T05:58:47.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:47.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:47.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3adcff9640 10 monclient: dump: 2026-02-17T05:58:47.099 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:47.099 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:47.099 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:47.099 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:47.099 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3ade7fc640 4 set_mon_vals no callback set 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3ae62f1640 10 client.4713.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3ae62f1640 10 monclient: _renew_subs 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3ae62f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3ae62f1640 1 -- 10.20.193.16:0/3138586960 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae015ddb0 con 0x7f3ae0155e00 2026-02-17T05:58:47.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.081+0000 7f3ae62f1640 1 librados: init done 2026-02-17T05:58:47.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.082+0000 7f3ae62f1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:47.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.082+0000 7f3ae62f1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:47.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.082+0000 7f3ae62f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:47.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.082+0000 7f3ae62f1640 1 -- 10.20.193.16:0/3138586960 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae0154540 con 0x7f3ae0155e00 2026-02-17T05:58:47.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.190+0000 7f3adcff9640 1 -- 10.20.193.16:0/3138586960 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f3acc002b60 con 0x7f3ae0155e00 2026-02-17T05:58:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.191+0000 7f3adcff9640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3ab0079190 0x7f3ab007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.191+0000 7f3adf7fe640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3ab0079190 0x7f3ab007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.191+0000 7f3adf7fe640 10 monclient: get_auth_request con 0x7f3ab0079190 auth_method 0 2026-02-17T05:58:47.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.191+0000 7f3adf7fe640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3ab0079190 0x7f3ab007b650 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0007e70 tx=0x7f3ad0007e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:47.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.191+0000 7f3adcff9640 10 client.4713.objecter ms_handle_connect 0x7f3ab0079190 2026-02-17T05:58:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.199+0000 7f3adcff9640 1 -- 10.20.193.16:0/3138586960 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f3acc05e260 con 0x7f3ae0155e00 2026-02-17T05:58:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.199+0000 7f3adcff9640 10 client.4713.objecter ms_dispatch2 0x7f3ae0000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.199+0000 7f3adcff9640 3 client.4713.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.199+0000 7f3adcff9640 3 client.4713.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.199+0000 7f3adcff9640 20 client.4713.objecter dump_active .. 0 homeless 2026-02-17T05:58:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.199+0000 7f3adcff9640 1 -- 10.20.193.16:0/3138586960 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3ae0154540 con 0x7f3ae0155e00 2026-02-17T05:58:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.199+0000 7f3adcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:47.200 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.199+0000 7f3adcff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:47.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.297+0000 7f3ae62f1640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:47.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.297+0000 7f3ae62f1640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:47.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.297+0000 7f3ae62f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:47.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.297+0000 7f3ae62f1640 1 -- 10.20.193.16:0/3138586960 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3ae00ca700 con 0x7f3ae0155e00 2026-02-17T05:58:47.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.300+0000 7f3adcff9640 1 -- 10.20.193.16:0/3138586960 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3acc01d020 con 0x7f3ae0155e00 2026-02-17T05:58:47.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.300+0000 7f3adcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:47.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.300+0000 7f3adcff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:47.300 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:47.300 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:47.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 10 librados: watch_flush enter 2026-02-17T05:58:47.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 10 librados: watch_flush exit 2026-02-17T05:58:47.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 20 client.4713.objecter shutdown clearing up homeless session... 2026-02-17T05:58:47.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 10 client.4713.objecter successfully canceled tick 2026-02-17T05:58:47.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 -- 10.20.193.16:0/3138586960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3ab0079190 msgr2=0x7f3ab007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:47.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3ab0079190 0x7f3ab007b650 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0007e70 tx=0x7f3ad0007e00 comp rx=0 tx=0).stop 2026-02-17T05:58:47.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 10 monclient: shutdown 2026-02-17T05:58:47.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 -- 10.20.193.16:0/3138586960 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae0155e00 msgr2=0x7f3ae014f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae0155e00 0x7f3ae014f0b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00b570 tx=0x7f3acc00ba40 comp rx=0 tx=0).stop 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 -- 10.20.193.16:0/3138586960 shutdown_connections 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3ab0079190 0x7f3ab007b650 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ae01561e0 0x7f3ae015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3ae015fa10 0x7f3ae01f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 --2- 10.20.193.16:0/3138586960 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ae0155e00 0x7f3ae014f0b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 -- 10.20.193.16:0/3138586960 >> 10.20.193.16:0/3138586960 conn(0x7f3ae00ce6c0 msgr2=0x7f3ae0153820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 -- 10.20.193.16:0/3138586960 shutdown_connections 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 -- 10.20.193.16:0/3138586960 wait complete. 2026-02-17T05:58:47.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:47.301+0000 7f3ae62f1640 1 librados: shutdown 2026-02-17T05:58:50.310 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:50.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.379+0000 7f9dbfb0a640 10 monclient: get_monmap_and_config 2026-02-17T05:58:50.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.379+0000 7f9dbfb0a640 10 monclient: build_initial_monmap 2026-02-17T05:58:50.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.379+0000 7f9dbfb0a640 10 monclient: monmap: 2026-02-17T05:58:50.384 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:50.384 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:50.384 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:50.380329+0000 2026-02-17T05:58:50.384 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:50.380329+0000 2026-02-17T05:58:50.384 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:50.384 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:50.384 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:50.384 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:50.384 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:50.385 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:50.385 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:50.385 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:50.385 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:50.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.379+0000 7f9dbfb0a640 1 Processor -- start 2026-02-17T05:58:50.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.379+0000 7f9dbfb0a640 1 -- start start 2026-02-17T05:58:50.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.379+0000 7f9dbfb0a640 10 monclient: init 2026-02-17T05:58:50.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:50.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 10 monclient: _add_conns ranks=[4,3,5,1,0,2] 2026-02-17T05:58:50.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f9db80700f0 0x7f9db80704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:50.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 10 monclient(hunting): picked mon.noname-b con 0x7f9db80700f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:50.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f9db80c7fe0 0x7f9db80c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:50.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 10 monclient(hunting): picked mon.noname-a con 0x7f9db80c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T05:58:50.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f9db81536f0 0x7f9db8153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:50.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 10 monclient(hunting): picked mon.noname-c con 0x7f9db81536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:50.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:50.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9db80c8970 con 0x7f9db80c7fe0 2026-02-17T05:58:50.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:50.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f9db80c4820 con 0x7f9db80700f0 2026-02-17T05:58:50.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:50.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f9db80c4b00 con 0x7f9db81536f0 2026-02-17T05:58:50.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:50.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbfb0a640 10 monclient(hunting): authenticate will time out at 1182.949290s 2026-02-17T05:58:50.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbd07e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f9db80c7fe0 0x7f9db80c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:50.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbd07e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f9db80c7fe0 0x7f9db80c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:40180/0 (socket says 10.20.193.16:40180) 2026-02-17T05:58:50.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbd07e640 1 -- 10.20.193.16:0/3107301284 learned_addr learned my addr 10.20.193.16:0/3107301284 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:50.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbd07e640 10 monclient(hunting): get_auth_request con 0x7f9db80c7fe0 auth_method 0 2026-02-17T05:58:50.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbd07e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbd07e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbd07e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbe080640 1 --2- 10.20.193.16:0/3107301284 >> v2:10.20.193.148:3300/0 conn(0x7f9db81536f0 0x7f9db8153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.380+0000 7f9dbd87f640 1 --2- 10.20.193.16:0/3107301284 >> v2:10.20.193.128:3300/0 conn(0x7f9db80700f0 0x7f9db80704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbd07e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbd07e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbd07e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbe080640 10 monclient(hunting): get_auth_request con 0x7f9db81536f0 auth_method 0 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbe080640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbe080640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:50.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbe080640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbd87f640 10 monclient(hunting): get_auth_request con 0x7f9db80700f0 auth_method 0 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbd87f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbd87f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbd87f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbd87f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbd87f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbd87f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbe080640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbe080640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.381+0000 7f9dbe080640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 10 monclient(hunting): handle_auth_done global_id 4716 payload 274 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 1 -- 10.20.193.16:0/3107301284 >> v2:10.20.193.148:3300/0 conn(0x7f9db81536f0 msgr2=0x7f9db8153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:50.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 1 --2- 10.20.193.16:0/3107301284 >> v2:10.20.193.148:3300/0 conn(0x7f9db81536f0 0x7f9db8153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 1 -- 10.20.193.16:0/3107301284 >> v2:10.20.193.128:3300/0 conn(0x7f9db80700f0 msgr2=0x7f9db80704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:50.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 1 --2- 10.20.193.16:0/3107301284 >> v2:10.20.193.128:3300/0 conn(0x7f9db80700f0 0x7f9db80704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 10 monclient: _finish_hunting 0 2026-02-17T05:58:50.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 1 monclient: found mon.noname-a 2026-02-17T05:58:50.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:50.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:50.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 1 -- 10.20.193.16:0/3107301284 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db80c4de0 con 0x7f9db80c7fe0 2026-02-17T05:58:50.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.382+0000 7f9dbd07e640 1 --2- 10.20.193.16:0/3107301284 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db80c8430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9db8070b90 tx=0x7f9dac02edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe369f3bef49923d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:50.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 1 -- 10.20.193.16:0/3107301284 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac03c070 con 0x7f9db80c7fe0 2026-02-17T05:58:50.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:50.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:50.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 10 monclient: dump: 2026-02-17T05:58:50.397 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 10 monclient: _finish_auth 0 2026-02-17T05:58:50.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 10 monclient: _check_auth_tickets 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 1 -- 10.20.193.16:0/3107301284 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dac02fab0 con 0x7f9db80c7fe0 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 1 -- 10.20.193.16:0/3107301284 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac037440 con 0x7f9db80c7fe0 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 5 monclient: authenticate success, global_id 4716 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 4 set_mon_vals no callback set 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbc87d640 10 monclient: dump: 2026-02-17T05:58:50.399 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 10 monclient: shutdown 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:50.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/3107301284 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 msgr2=0x7f9db80c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 1 --2- 10.20.193.16:0/3107301284 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db80c8430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9db8070b90 tx=0x7f9dac02edf0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/3107301284 shutdown_connections 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 1 --2- 10.20.193.16:0/3107301284 >> v2:10.20.193.148:3300/0 conn(0x7f9db81536f0 0x7f9db8153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 1 --2- 10.20.193.16:0/3107301284 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db80c8430 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 1 --2- 10.20.193.16:0/3107301284 >> v2:10.20.193.128:3300/0 conn(0x7f9db80700f0 0x7f9db80704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/3107301284 >> 10.20.193.16:0/3107301284 conn(0x7f9db80cd770 msgr2=0x7f9db80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/3107301284 shutdown_connections 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/3107301284 wait complete. 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 10 monclient: build_initial_monmap 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.385+0000 7f9dbfb0a640 10 monclient: monmap: 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:50.401 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 librados: starting msgr at 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 librados: starting objecter 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 Processor -- start 2026-02-17T05:58:50.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 -- start start 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 librados: setting wanted keys 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 librados: calling monclient init 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient: init 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9db80700f0 0x7f9db8139900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient(hunting): picked mon.noname-c con 0x7f9db80700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient(hunting): picked mon.noname-a con 0x7f9db80c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db81536f0 0x7f9db8141c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient(hunting): picked mon.noname-b con 0x7f9db81536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:50.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9db80c5680 con 0x7f9db80c7fe0 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f9db80c53a0 con 0x7f9db81536f0 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9db8154040 con 0x7f9db80700f0 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbfb0a640 10 monclient(hunting): authenticate will time out at 1182.955555s 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbd07e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbd07e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db806ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:40190/0 (socket says 10.20.193.16:40190) 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbd07e640 1 -- 10.20.193.16:0/2919913034 learned_addr learned my addr 10.20.193.16:0/2919913034 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbd07e640 10 monclient(hunting): get_auth_request con 0x7f9db80c7fe0 auth_method 0 2026-02-17T05:58:50.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbd07e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbd07e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbd07e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbd87f640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9db80700f0 0x7f9db8139900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.386+0000 7f9dbe080640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db81536f0 0x7f9db8141c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.387+0000 7f9dbd07e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.387+0000 7f9dbd07e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.387+0000 7f9dbd07e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.387+0000 7f9dbe080640 10 monclient(hunting): get_auth_request con 0x7f9db81536f0 auth_method 0 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.387+0000 7f9dbe080640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.387+0000 7f9dbe080640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.387+0000 7f9dbe080640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:50.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.387+0000 7f9dbe080640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:50.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.387+0000 7f9dbe080640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:50.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.387+0000 7f9dbe080640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:50.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd87f640 10 monclient(hunting): get_auth_request con 0x7f9db80700f0 auth_method 0 2026-02-17T05:58:50.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd87f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:50.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd87f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:50.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd87f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:50.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd87f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:50.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd87f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:50.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd87f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:50.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd07e640 10 monclient(hunting): handle_auth_done global_id 4719 payload 824 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd07e640 1 -- 10.20.193.16:0/2919913034 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9db80700f0 msgr2=0x7f9db8139900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd07e640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9db80700f0 0x7f9db8139900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd07e640 1 -- 10.20.193.16:0/2919913034 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db81536f0 msgr2=0x7f9db8141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd07e640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db81536f0 0x7f9db8141c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd07e640 10 monclient: _finish_hunting 0 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd07e640 1 monclient: found mon.noname-a 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd07e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd07e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.388+0000 7f9dbd07e640 1 -- 10.20.193.16:0/2919913034 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db8142810 con 0x7f9db80c7fe0 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.389+0000 7f9dbd07e640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db806ebc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9dac02f350 tx=0x7f9dac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.389+0000 7f9da67fc640 10 client.?.objecter ms_handle_connect 0x7f9db80c7fe0 2026-02-17T05:58:50.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.389+0000 7f9da67fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 1 -- 10.20.193.16:0/2919913034 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac03c040 con 0x7f9db80c7fe0 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 10 monclient: dump: 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:50.408 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 1 -- 10.20.193.16:0/2919913034 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dac042430 con 0x7f9db80c7fe0 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 1 -- 10.20.193.16:0/2919913034 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac042800 con 0x7f9db80c7fe0 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da7fff640 4 set_mon_vals no callback set 2026-02-17T05:58:50.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9da67fc640 10 monclient: dump: 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 5 monclient: authenticate success, global_id 4719 2026-02-17T05:58:50.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 10 monclient: _renew_subs 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/2919913034 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db8142af0 con 0x7f9db80c7fe0 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 10 client.4719.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 10 monclient: _renew_subs 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/2919913034 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db813e520 con 0x7f9db80c7fe0 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 1 librados: init done 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:50.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.391+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/2919913034 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db8074c00 con 0x7f9db80c7fe0 2026-02-17T05:58:50.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.403+0000 7f9da67fc640 1 -- 10.20.193.16:0/2919913034 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f9dac0429a0 con 0x7f9db80c7fe0 2026-02-17T05:58:50.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.404+0000 7f9da67fc640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079140 0x7f9d8807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:50.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.404+0000 7f9dbd87f640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079140 0x7f9d8807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:50.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.404+0000 7f9dbd87f640 10 monclient: get_auth_request con 0x7f9d88079140 auth_method 0 2026-02-17T05:58:50.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.404+0000 7f9dbd87f640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079140 0x7f9d8807b600 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009ba0 tx=0x7f9da8009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:50.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.404+0000 7f9da67fc640 10 client.4719.objecter ms_handle_connect 0x7f9d88079140 2026-02-17T05:58:50.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.406+0000 7f9da67fc640 1 -- 10.20.193.16:0/2919913034 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f9dac0838d0 con 0x7f9db80c7fe0 2026-02-17T05:58:50.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.406+0000 7f9da67fc640 10 client.4719.objecter ms_dispatch2 0x7f9db8000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:50.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.406+0000 7f9da67fc640 3 client.4719.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:50.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.406+0000 7f9da67fc640 3 client.4719.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:50.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.406+0000 7f9da67fc640 20 client.4719.objecter dump_active .. 0 homeless 2026-02-17T05:58:50.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.509+0000 7f9da67fc640 1 -- 10.20.193.16:0/2919913034 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9dac087890 con 0x7f9db80c7fe0 2026-02-17T05:58:50.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.509+0000 7f9da67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:50.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.509+0000 7f9da67fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:50.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.609+0000 7f9dbfb0a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:50.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.609+0000 7f9dbfb0a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:50.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.609+0000 7f9dbfb0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:50.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.609+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/2919913034 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9db81573f0 con 0x7f9db80c7fe0 2026-02-17T05:58:50.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.611+0000 7f9da67fc640 1 -- 10.20.193.16:0/2919913034 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f9dac08bb90 con 0x7f9db80c7fe0 2026-02-17T05:58:50.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.611+0000 7f9da67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:50.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.611+0000 7f9da67fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:50.612 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:50.613 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:50.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 10 librados: watch_flush enter 2026-02-17T05:58:50.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 10 librados: watch_flush exit 2026-02-17T05:58:50.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 20 client.4719.objecter shutdown clearing up homeless session... 2026-02-17T05:58:50.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 10 client.4719.objecter successfully canceled tick 2026-02-17T05:58:50.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/2919913034 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079140 msgr2=0x7f9d8807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079140 0x7f9d8807b600 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009ba0 tx=0x7f9da8009290 comp rx=0 tx=0).stop 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 10 monclient: shutdown 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/2919913034 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 msgr2=0x7f9db806ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db806ebc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9dac02f350 tx=0x7f9dac0047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/2919913034 shutdown_connections 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079140 0x7f9d8807b600 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db81536f0 0x7f9db8141c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db806ebc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 --2- 10.20.193.16:0/2919913034 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9db80700f0 0x7f9db8139900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/2919913034 >> 10.20.193.16:0/2919913034 conn(0x7f9db80cd770 msgr2=0x7f9db814e580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:50.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/2919913034 shutdown_connections 2026-02-17T05:58:50.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 -- 10.20.193.16:0/2919913034 wait complete. 2026-02-17T05:58:50.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:50.612+0000 7f9dbfb0a640 1 librados: shutdown 2026-02-17T05:58:53.622 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:53.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.691+0000 7f13b8734640 10 monclient: get_monmap_and_config 2026-02-17T05:58:53.697 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.691+0000 7f13b8734640 10 monclient: build_initial_monmap 2026-02-17T05:58:53.697 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.691+0000 7f13b8734640 10 monclient: monmap: 2026-02-17T05:58:53.697 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:53.697 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:53.697 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:53.692671+0000 2026-02-17T05:58:53.697 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:53.692671+0000 2026-02-17T05:58:53.698 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:53.698 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:53.698 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:53.698 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:53.698 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:53.698 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:53.698 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:53.699 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:53.699 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:53.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 1 Processor -- start 2026-02-17T05:58:53.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 1 -- start start 2026-02-17T05:58:53.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 10 monclient: init 2026-02-17T05:58:53.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:53.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 10 monclient: _add_conns ranks=[1,4,5,3,0,2] 2026-02-17T05:58:53.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f13b00700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:58:53.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f13b00c86f0 0x7f13b00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:53.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 10 monclient(hunting): picked mon.noname-b con 0x7f13b00c86f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:53.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f13b01536f0 0x7f13b0153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:53.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 10 monclient(hunting): picked mon.noname-c con 0x7f13b01536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:53.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b00c4820 con 0x7f13b00700f0 2026-02-17T05:58:53.700 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:53.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f13b00c4b00 con 0x7f13b00c86f0 2026-02-17T05:58:53.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:53.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f13b00c4de0 con 0x7f13b01536f0 2026-02-17T05:58:53.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:53.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.692+0000 7f13b8734640 10 monclient(hunting): authenticate will time out at 1186.261629s 2026-02-17T05:58:53.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b5ca8640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f13b00c86f0 0x7f13b00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:53.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b6caa640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f13b01536f0 0x7f13b0153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:53.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b5ca8640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f13b00c86f0 0x7f13b00704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:53812/0 (socket says 10.20.193.16:53812) 2026-02-17T05:58:53.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b5ca8640 1 -- 10.20.193.16:0/619066975 learned_addr learned my addr 10.20.193.16:0/619066975 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:53.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b5ca8640 10 monclient(hunting): get_auth_request con 0x7f13b00c86f0 auth_method 0 2026-02-17T05:58:53.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b5ca8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:53.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b5ca8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:53.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b5ca8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:53.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b6caa640 10 monclient(hunting): get_auth_request con 0x7f13b01536f0 auth_method 0 2026-02-17T05:58:53.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b6caa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:53.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b6caa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:53.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b6caa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:53.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b6caa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:53.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b6caa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:53.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b6caa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:53.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b5ca8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:53.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.693+0000 7f13b5ca8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:53.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.694+0000 7f13b5ca8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:53.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.694+0000 7f13b6caa640 10 monclient(hunting): handle_auth_done global_id 4853 payload 274 2026-02-17T05:58:53.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.694+0000 7f13b6caa640 1 -- 10.20.193.16:0/619066975 >> v2:10.20.193.128:3300/0 conn(0x7f13b00c86f0 msgr2=0x7f13b00704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:53.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.694+0000 7f13b6caa640 1 --2- 10.20.193.16:0/619066975 >> v2:10.20.193.128:3300/0 conn(0x7f13b00c86f0 0x7f13b00704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:53.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.694+0000 7f13b6caa640 1 -- 10.20.193.16:0/619066975 >> v1:10.20.193.128:6789/0 conn(0x7f13b00700f0 legacy=0x7f13b00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:53.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.694+0000 7f13b6caa640 10 monclient: _finish_hunting 0 2026-02-17T05:58:53.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.694+0000 7f13b6caa640 1 monclient: found mon.noname-c 2026-02-17T05:58:53.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.694+0000 7f13b6caa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:53.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.694+0000 7f13b6caa640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:53.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.694+0000 7f13b6caa640 1 -- 10.20.193.16:0/619066975 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b00c50c0 con 0x7f13b01536f0 2026-02-17T05:58:53.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.695+0000 7f13b6caa640 1 --2- 10.20.193.16:0/619066975 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f13b01536f0 0x7f13b0153b20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13b00c9540 tx=0x7f13ac02f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c780848271ab0a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:53.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 1 -- 10.20.193.16:0/619066975 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac03c070 con 0x7f13b01536f0 2026-02-17T05:58:53.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:53.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:53.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 10 monclient: dump: 2026-02-17T05:58:53.716 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:53.716 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:53.716 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:53.717 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:53.717 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:53.717 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:53.717 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:53.717 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:53.717 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:53.717 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 10 monclient: _finish_auth 0 2026-02-17T05:58:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 10 monclient: _check_auth_tickets 2026-02-17T05:58:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 1 -- 10.20.193.16:0/619066975 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ac02fe40 con 0x7f13b01536f0 2026-02-17T05:58:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 1 -- 10.20.193.16:0/619066975 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac037660 con 0x7f13b01536f0 2026-02-17T05:58:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 5 monclient: authenticate success, global_id 4853 2026-02-17T05:58:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b54a7640 10 monclient: dump: 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 4 set_mon_vals no callback set 2026-02-17T05:58:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 10 monclient: shutdown 2026-02-17T05:58:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 1 -- 10.20.193.16:0/619066975 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f13b01536f0 msgr2=0x7f13b0153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 1 --2- 10.20.193.16:0/619066975 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f13b01536f0 0x7f13b0153b20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13b00c9540 tx=0x7f13ac02f130 comp rx=0 tx=0).stop 2026-02-17T05:58:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 1 -- 10.20.193.16:0/619066975 shutdown_connections 2026-02-17T05:58:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 1 --2- 10.20.193.16:0/619066975 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f13b01536f0 0x7f13b0153b20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 1 --2- 10.20.193.16:0/619066975 >> v2:10.20.193.128:3300/0 conn(0x7f13b00c86f0 0x7f13b00704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 1 -- 10.20.193.16:0/619066975 >> 10.20.193.16:0/619066975 conn(0x7f13b00cd770 msgr2=0x7f13b00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 1 -- 10.20.193.16:0/619066975 shutdown_connections 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.697+0000 7f13b8734640 1 -- 10.20.193.16:0/619066975 wait complete. 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient: build_initial_monmap 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient: monmap: 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:53.721 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:53.722 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 librados: starting msgr at 2026-02-17T05:58:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 librados: starting objecter 2026-02-17T05:58:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 Processor -- start 2026-02-17T05:58:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 -- start start 2026-02-17T05:58:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 librados: setting wanted keys 2026-02-17T05:58:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 librados: calling monclient init 2026-02-17T05:58:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient: init 2026-02-17T05:58:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:58:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f13b00700f0 0x7f13b01f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient(hunting): picked mon.noname-b con 0x7f13b00700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f13b00c86f0 0x7f13b006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient(hunting): picked mon.noname-a con 0x7f13b00c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f13b01536f0 0x7f13b01f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient(hunting): picked mon.noname-c con 0x7f13b01536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13b0154060 con 0x7f13b00c86f0 2026-02-17T05:58:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f13b00c5680 con 0x7f13b00700f0 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13b0074c00 con 0x7f13b01536f0 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.698+0000 7f13b8734640 10 monclient(hunting): authenticate will time out at 1186.267820s 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b5ca8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f13b00c86f0 0x7f13b006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b6caa640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f13b01536f0 0x7f13b01f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b64a9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f13b00700f0 0x7f13b01f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b5ca8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f13b00c86f0 0x7f13b006db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:40204/0 (socket says 10.20.193.16:40204) 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b5ca8640 1 -- 10.20.193.16:0/1227389236 learned_addr learned my addr 10.20.193.16:0/1227389236 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b5ca8640 10 monclient(hunting): get_auth_request con 0x7f13b00c86f0 auth_method 0 2026-02-17T05:58:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b5ca8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b5ca8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b5ca8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b6caa640 10 monclient(hunting): get_auth_request con 0x7f13b01536f0 auth_method 0 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b6caa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b6caa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b6caa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b64a9640 10 monclient(hunting): get_auth_request con 0x7f13b00700f0 auth_method 0 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b64a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b64a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b64a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b6caa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b6caa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b6caa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b5ca8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b5ca8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b5ca8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b64a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b64a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.699+0000 7f13b64a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.700+0000 7f13b6caa640 10 monclient(hunting): handle_auth_done global_id 4856 payload 824 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.700+0000 7f13b6caa640 1 -- 10.20.193.16:0/1227389236 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f13b00700f0 msgr2=0x7f13b01f6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.700+0000 7f13b6caa640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f13b00700f0 0x7f13b01f6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.700+0000 7f13b6caa640 1 -- 10.20.193.16:0/1227389236 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f13b00c86f0 msgr2=0x7f13b006db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.700+0000 7f13b6caa640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f13b00c86f0 0x7f13b006db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.700+0000 7f13b6caa640 10 monclient: _finish_hunting 0 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.700+0000 7f13b6caa640 1 monclient: found mon.noname-c 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.700+0000 7f13b6caa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.700+0000 7f13b6caa640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.700+0000 7f13b6caa640 1 -- 10.20.193.16:0/1227389236 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b01f7840 con 0x7f13b01536f0 2026-02-17T05:58:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.701+0000 7f13b6caa640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f13b01536f0 0x7f13b01f6020 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f13ac036000 tx=0x7f13ac004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.701+0000 7f1396ffd640 10 client.?.objecter ms_handle_connect 0x7f13b01536f0 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.701+0000 7f1396ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f1396ffd640 1 -- 10.20.193.16:0/1227389236 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac047020 con 0x7f13b01536f0 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f1396ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f1396ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f1396ffd640 10 monclient: dump: 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f1396ffd640 10 monclient: _finish_auth 0 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f1396ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f1396ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f13b8734640 5 monclient: authenticate success, global_id 4856 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f13b8734640 10 monclient: _renew_subs 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f13b8734640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f13b8734640 1 -- 10.20.193.16:0/1227389236 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b01f6fa0 con 0x7f13b01536f0 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f13b8734640 10 client.4856.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f13b8734640 10 monclient: _renew_subs 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f13b8734640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f13b8734640 1 -- 10.20.193.16:0/1227389236 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b01f7560 con 0x7f13b01536f0 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.702+0000 7f13b8734640 1 librados: init done 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f13b8734640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f13b8734640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f13b8734640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f13b8734640 1 -- 10.20.193.16:0/1227389236 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b013e7e0 con 0x7f13b01536f0 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f1396ffd640 1 -- 10.20.193.16:0/1227389236 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ac042470 con 0x7f13b01536f0 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f1396ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f13b4ca6640 4 set_mon_vals no callback set 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f1396ffd640 1 -- 10.20.193.16:0/1227389236 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac03c040 con 0x7f13b01536f0 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f1396ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f1396ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.703+0000 7f1396ffd640 10 monclient: dump: 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:53.729 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:53.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.811+0000 7f1396ffd640 1 -- 10.20.193.16:0/1227389236 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f13ac042860 con 0x7f13b01536f0 2026-02-17T05:58:53.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.811+0000 7f1396ffd640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1380079140 0x7f138007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:53.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.811+0000 7f13b64a9640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1380079140 0x7f138007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:53.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.811+0000 7f13b64a9640 10 monclient: get_auth_request con 0x7f1380079140 auth_method 0 2026-02-17T05:58:53.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.811+0000 7f13b64a9640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1380079140 0x7f138007b600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f13a0002770 tx=0x7f13a0007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:53.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.811+0000 7f1396ffd640 10 client.4856.objecter ms_handle_connect 0x7f1380079140 2026-02-17T05:58:53.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.813+0000 7f1396ffd640 1 -- 10.20.193.16:0/1227389236 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f13ac0bd9e0 con 0x7f13b01536f0 2026-02-17T05:58:53.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.813+0000 7f1396ffd640 10 client.4856.objecter ms_dispatch2 0x7f13b0000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:53.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.813+0000 7f1396ffd640 3 client.4856.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:53.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.813+0000 7f1396ffd640 3 client.4856.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:53.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.813+0000 7f1396ffd640 20 client.4856.objecter dump_active .. 0 homeless 2026-02-17T05:58:53.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.819+0000 7f1396ffd640 1 -- 10.20.193.16:0/1227389236 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f13b013e7e0 con 0x7f13b01536f0 2026-02-17T05:58:53.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.819+0000 7f1396ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:53.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.819+0000 7f1396ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:53.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.918+0000 7f13b8734640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:53.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.918+0000 7f13b8734640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:53.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.918+0000 7f13b8734640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:58:53.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.918+0000 7f13b8734640 1 -- 10.20.193.16:0/1227389236 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f13b00704d0 con 0x7f13b01536f0 2026-02-17T05:58:53.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.921+0000 7f1396ffd640 1 -- 10.20.193.16:0/1227389236 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f13ac051070 con 0x7f13b01536f0 2026-02-17T05:58:53.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.921+0000 7f1396ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:53.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.921+0000 7f1396ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:53.922 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:53.922 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:53.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 10 librados: watch_flush enter 2026-02-17T05:58:53.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 10 librados: watch_flush exit 2026-02-17T05:58:53.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 20 client.4856.objecter shutdown clearing up homeless session... 2026-02-17T05:58:53.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 10 client.4856.objecter successfully canceled tick 2026-02-17T05:58:53.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 -- 10.20.193.16:0/1227389236 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1380079140 msgr2=0x7f138007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:53.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1380079140 0x7f138007b600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f13a0002770 tx=0x7f13a0007ef0 comp rx=0 tx=0).stop 2026-02-17T05:58:53.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 10 monclient: shutdown 2026-02-17T05:58:53.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:53.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 -- 10.20.193.16:0/1227389236 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f13b01536f0 msgr2=0x7f13b01f6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:53.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f13b01536f0 0x7f13b01f6020 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f13ac036000 tx=0x7f13ac004b40 comp rx=0 tx=0).stop 2026-02-17T05:58:53.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 -- 10.20.193.16:0/1227389236 shutdown_connections 2026-02-17T05:58:53.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1380079140 0x7f138007b600 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:53.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f13b01536f0 0x7f13b01f6020 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:53.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f13b00c86f0 0x7f13b006db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:53.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 --2- 10.20.193.16:0/1227389236 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f13b00700f0 0x7f13b01f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:53.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 -- 10.20.193.16:0/1227389236 >> 10.20.193.16:0/1227389236 conn(0x7f13b00cd770 msgr2=0x7f13b014e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:53.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 -- 10.20.193.16:0/1227389236 shutdown_connections 2026-02-17T05:58:53.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 -- 10.20.193.16:0/1227389236 wait complete. 2026-02-17T05:58:53.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:53.922+0000 7f13b8734640 1 librados: shutdown 2026-02-17T05:58:56.932 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.001+0000 7f51fc65b640 10 monclient: get_monmap_and_config 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.001+0000 7f51fc65b640 10 monclient: build_initial_monmap 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.001+0000 7f51fc65b640 10 monclient: monmap: 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:58:57.002761+0000 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:58:57.002761+0000 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 1 Processor -- start 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 1 -- start start 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 10 monclient: init 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 10 monclient: _add_conns ranks=[1,3,4,2,5,0] 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f51f40700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 0x7f51f40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 10 monclient(hunting): picked mon.noname-a con 0x7f51f40c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f51f41536f0 0x7f51f4153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 10 monclient(hunting): picked mon.noname-b con 0x7f51f41536f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f40c4820 con 0x7f51f40700f0 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f51f40c4b00 con 0x7f51f40c86f0 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f51f40c4de0 con 0x7f51f41536f0 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.002+0000 7f51fc65b640 10 monclient(hunting): authenticate will time out at 1189.571863s 2026-02-17T05:58:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51f9bcf640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 0x7f51f40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51fabd1640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f51f41536f0 0x7f51f4153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51f9bcf640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 0x7f51f40704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:40216/0 (socket says 10.20.193.16:40216) 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51f9bcf640 1 -- 10.20.193.16:0/4223699682 learned_addr learned my addr 10.20.193.16:0/4223699682 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51f9bcf640 10 monclient(hunting): get_auth_request con 0x7f51f40c86f0 auth_method 0 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51f9bcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51f9bcf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51f9bcf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51fabd1640 10 monclient(hunting): get_auth_request con 0x7f51f41536f0 auth_method 0 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51fabd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51fabd1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51fabd1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51f9bcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51f9bcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.003+0000 7f51f9bcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51fabd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51fabd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51fabd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51f9bcf640 10 monclient(hunting): handle_auth_done global_id 4725 payload 274 2026-02-17T05:58:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51f9bcf640 1 -- 10.20.193.16:0/4223699682 >> v2:10.20.193.128:3300/0 conn(0x7f51f41536f0 msgr2=0x7f51f4153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51f9bcf640 1 --2- 10.20.193.16:0/4223699682 >> v2:10.20.193.128:3300/0 conn(0x7f51f41536f0 0x7f51f4153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51f9bcf640 1 -- 10.20.193.16:0/4223699682 >> v1:10.20.193.128:6789/0 conn(0x7f51f40700f0 legacy=0x7f51f40c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51f9bcf640 10 monclient: _finish_hunting 0 2026-02-17T05:58:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51f9bcf640 1 monclient: found mon.noname-a 2026-02-17T05:58:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51f9bcf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51f9bcf640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.004+0000 7f51f9bcf640 1 -- 10.20.193.16:0/4223699682 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f40c50c0 con 0x7f51f40c86f0 2026-02-17T05:58:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.005+0000 7f51f9bcf640 1 -- 10.20.193.16:0/4223699682 >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 msgr2=0x7f51f40704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.005+0000 7f51f9bcf640 1 -- 10.20.193.16:0/4223699682 >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 msgr2=0x7f51f40704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.005+0000 7f51f9bcf640 1 --2- 10.20.193.16:0/4223699682 >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 0x7f51f40704d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f51f40c9540 tx=0x7f51e802ee60 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.005+0000 7f51f9bcf640 1 --2- 10.20.193.16:0/4223699682 >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 0x7f51f40704d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:57.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f9bcf640 1 --2- 10.20.193.16:0/4223699682 >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 0x7f51f40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f9bcf640 10 monclient: get_auth_request con 0x7f51f40c86f0 auth_method 0 2026-02-17T05:58:57.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f9bcf640 0 --2- 10.20.193.16:0/4223699682 >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 0x7f51f40704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-17T05:58:57.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f9bcf640 1 --2- 10.20.193.16:0/4223699682 >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 0x7f51f40704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 10 monclient: ms_handle_reset current mon v2:10.20.193.16:3300/0 2026-02-17T05:58:57.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:57.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 msgr2=0x7f51f40704d0 unknown :-1 s=STATE_CLOSED l=0).mark_down 2026-02-17T05:58:57.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 1 --2- 10.20.193.16:0/4223699682 >> v2:10.20.193.16:3300/0 conn(0x7f51f40c86f0 0x7f51f40704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 10 monclient: _add_conns ranks=[5,2,0] 2026-02-17T05:58:57.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 1 --2- 10.20.193.16:0/4223699682 >> v2:10.20.193.148:3300/0 conn(0x7f51dc003630 0x7f51dc005a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:57.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 10 monclient(hunting): picked mon.noname-c con 0x7f51dc003630 addr v2:10.20.193.148:3300/0 2026-02-17T05:58:57.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f51dc0063b0 addr v1:10.20.193.148:6789/0 2026-02-17T05:58:57.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f51dc00a380 addr v1:10.20.193.16:6789/0 2026-02-17T05:58:57.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51dc00e330 con 0x7f51dc00a380 2026-02-17T05:58:57.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51dc00f530 con 0x7f51dc0063b0 2026-02-17T05:58:57.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:57.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f51dc010780 con 0x7f51dc003630 2026-02-17T05:58:57.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:57.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.206+0000 7f51f93ce640 10 monclient(hunting): ms_handle_reset stray mon v2:10.20.193.16:3300/0 2026-02-17T05:58:57.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.208+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2883070767 0 0) 0x7f51dc00e330 con 0x7f51dc00a380 2026-02-17T05:58:57.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.208+0000 7f51f93ce640 10 monclient(hunting): _negotiate 2026-02-17T05:58:57.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.208+0000 7f51f93ce640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.208+0000 7f51f93ce640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:57.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.208+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 --> v1:10.20.193.16:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f51dc00e500 con 0x7f51dc00a380 2026-02-17T05:58:57.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.209+0000 7f51fa3d0640 1 --2- 10.20.193.16:0/4223699682 >> v2:10.20.193.148:3300/0 conn(0x7f51dc003630 0x7f51dc005a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.209+0000 7f51fa3d0640 10 monclient(hunting): get_auth_request con 0x7f51dc003630 auth_method 0 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.209+0000 7f51fa3d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.209+0000 7f51fa3d0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.209+0000 7f51fa3d0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.210+0000 7f51fa3d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.210+0000 7f51fa3d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.210+0000 7f51fa3d0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 <== mon.0 v1:10.20.193.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 3574921325 0 0) 0x7f51dc00e500 con 0x7f51dc00a380 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 >> v2:10.20.193.148:3300/0 conn(0x7f51dc003630 msgr2=0x7f51dc005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 --2- 10.20.193.16:0/4223699682 >> v2:10.20.193.148:3300/0 conn(0x7f51dc003630 0x7f51dc005a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 >> v1:10.20.193.148:6789/0 conn(0x7f51dc0063b0 legacy=0x7f51dc008870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient: _finish_hunting 0 2026-02-17T05:58:57.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51fabd1640 1 -- 10.20.193.16:0/4223699682 reap_dead start 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.16:6789/0 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 --> v1:10.20.193.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc010a40 con 0x7f51dc00a380 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient: _finish_auth 0 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient: _check_auth_tickets 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51fc65b640 5 monclient: authenticate success, global_id 4725 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51fc65b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 <== mon.0 v1:10.20.193.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f51f0005e10 con 0x7f51dc00a380 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:57.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient: dump: 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:57.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] but i'm connected to v1:10.20.193.16:6789/0, reconnecting 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 >> v1:10.20.193.16:6789/0 conn(0x7f51dc00a380 legacy=0x7f51dc00c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00ec80 0x7f51dc005a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient(hunting): picked mon.b con 0x7f51dc00ec80 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f51dc00f060 0x7f51dc0087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient(hunting): picked mon.c con 0x7f51dc00f060 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00f700 0x7f51dc00fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient(hunting): picked mon.a con 0x7f51dc00f700 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc010780 con 0x7f51dc00f700 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:57.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc010240 con 0x7f51dc00ec80 2026-02-17T05:58:57.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:57.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc010410 con 0x7f51dc00f060 2026-02-17T05:58:57.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:57.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51f93ce640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:58:57.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.211+0000 7f51fc65b640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:57.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.212+0000 7f51fa3d0640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00ec80 0x7f51dc005a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.212+0000 7f51fa3d0640 10 monclient(hunting): get_auth_request con 0x7f51dc00ec80 auth_method 0 2026-02-17T05:58:57.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.212+0000 7f51fa3d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:57.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.212+0000 7f51fa3d0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.212+0000 7f51fa3d0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:57.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.212+0000 7f51fa3d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:57.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.212+0000 7f51fa3d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:57.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.212+0000 7f51fa3d0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:57.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fabd1640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00f700 0x7f51dc00fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fabd1640 10 monclient(hunting): get_auth_request con 0x7f51dc00f700 auth_method 0 2026-02-17T05:58:57.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fabd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:57.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fabd1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fabd1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:57.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51f9bcf640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f51dc00f060 0x7f51dc0087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fabd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:57.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fabd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:57.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fabd1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:57.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 10 monclient(hunting): handle_auth_done global_id 4725 payload 293 2026-02-17T05:58:57.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 1 -- 10.20.193.16:0/4223699682 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f51dc00f060 msgr2=0x7f51dc0087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:57.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f51dc00f060 0x7f51dc0087f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 1 -- 10.20.193.16:0/4223699682 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00f700 msgr2=0x7f51dc00fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:57.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00f700 0x7f51dc00fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 10 monclient: _finish_hunting 0 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 1 monclient: found mon.b 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 1 -- 10.20.193.16:0/4223699682 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc00e500 con 0x7f51dc00ec80 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 10 monclient: _finish_auth 0 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 10 monclient: _check_auth_tickets 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.213+0000 7f51fa3d0640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.214+0000 7f51fa3d0640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00ec80 0x7f51dc005a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f51e4005130 tx=0x7f51dc00e740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c84e93361c1745a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4003d10 con 0x7f51dc00ec80 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51f93ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:57.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51f93ce640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51f93ce640 10 monclient: dump: 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:57.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e4018070 con 0x7f51dc00ec80 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51fc65b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51f93ce640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51f93ce640 1 -- 10.20.193.16:0/4223699682 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4002e40 con 0x7f51dc00ec80 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51fc65b640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51fc65b640 4 set_mon_vals no callback set 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51f93ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51f93ce640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51f93ce640 10 monclient: dump: 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:57.231 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51fc65b640 10 monclient: shutdown 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51fc65b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.216+0000 7f51fc65b640 1 -- 10.20.193.16:0/4223699682 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00ec80 msgr2=0x7f51dc005a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00ec80 0x7f51dc005a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f51e4005130 tx=0x7f51dc00e740 comp rx=0 tx=0).stop 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 -- 10.20.193.16:0/4223699682 shutdown_connections 2026-02-17T05:58:57.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00f700 0x7f51dc00fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f51dc00f060 0x7f51dc0087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 --2- 10.20.193.16:0/4223699682 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00ec80 0x7f51dc005a20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 -- 10.20.193.16:0/4223699682 >> 10.20.193.16:0/4223699682 conn(0x7f51f40cd770 msgr2=0x7f51f40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 -- 10.20.193.16:0/4223699682 shutdown_connections 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 -- 10.20.193.16:0/4223699682 wait complete. 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 10 monclient: build_initial_monmap 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 10 monclient: monmap: 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:58:57.233 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 librados: starting msgr at 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 librados: starting objecter 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 Processor -- start 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 -- start start 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 librados: setting wanted keys 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 1 librados: calling monclient init 2026-02-17T05:58:57.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.217+0000 7f51fc65b640 10 monclient: init 2026-02-17T05:58:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:58:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00a380 0x7f51f4154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 10 monclient(hunting): picked mon.noname-b con 0x7f51dc00a380 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:58:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00ec80 0x7f51f41e7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 10 monclient(hunting): picked mon.noname-a con 0x7f51dc00ec80 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:58:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f51dc00f060 0x7f51f4155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 10 monclient(hunting): picked mon.noname-c con 0x7f51dc00f060 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:58:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51f4074c00 con 0x7f51dc00ec80 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f51f40c5680 con 0x7f51dc00a380 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51f4074d80 con 0x7f51dc00f060 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fc65b640 10 monclient(hunting): authenticate will time out at 1189.787390s 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51f9bcf640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00ec80 0x7f51f41e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fa3d0640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00a380 0x7f51f4154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51f9bcf640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00ec80 0x7f51f41e7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:40228/0 (socket says 10.20.193.16:40228) 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51f9bcf640 1 -- 10.20.193.16:0/2159091922 learned_addr learned my addr 10.20.193.16:0/2159091922 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51f9bcf640 10 monclient(hunting): get_auth_request con 0x7f51dc00ec80 auth_method 0 2026-02-17T05:58:57.236 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51f9bcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51f9bcf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fabd1640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f51dc00f060 0x7f51f4155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51f9bcf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fa3d0640 10 monclient(hunting): get_auth_request con 0x7f51dc00a380 auth_method 0 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fa3d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fa3d0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fa3d0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fabd1640 10 monclient(hunting): get_auth_request con 0x7f51dc00f060 auth_method 0 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fabd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fabd1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51fabd1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51f9bcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:57.237 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.218+0000 7f51f9bcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.219+0000 7f51f9bcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.219+0000 7f51fa3d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.219+0000 7f51fa3d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.219+0000 7f51fa3d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.219+0000 7f51fabd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.219+0000 7f51fabd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.219+0000 7f51fabd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.220+0000 7f51f9bcf640 10 monclient(hunting): handle_auth_done global_id 4728 payload 824 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.220+0000 7f51f9bcf640 1 -- 10.20.193.16:0/2159091922 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f51dc00f060 msgr2=0x7f51f4155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.220+0000 7f51f9bcf640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f51dc00f060 0x7f51f4155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.220+0000 7f51f9bcf640 1 -- 10.20.193.16:0/2159091922 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00a380 msgr2=0x7f51f4154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:57.238 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.220+0000 7f51f9bcf640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00a380 0x7f51f4154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.220+0000 7f51f9bcf640 10 monclient: _finish_hunting 0 2026-02-17T05:58:57.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.220+0000 7f51f9bcf640 1 monclient: found mon.noname-a 2026-02-17T05:58:57.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.220+0000 7f51f9bcf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:57.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.220+0000 7f51f9bcf640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:58:57.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.220+0000 7f51f9bcf640 1 -- 10.20.193.16:0/2159091922 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f41f2380 con 0x7f51dc00ec80 2026-02-17T05:58:57.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.221+0000 7f51f9bcf640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00ec80 0x7f51f41e7650 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f51e802eea0 tx=0x7f51e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:57.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.221+0000 7f51daffd640 10 client.?.objecter ms_handle_connect 0x7f51dc00ec80 2026-02-17T05:58:57.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.221+0000 7f51daffd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:58:57.239 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51daffd640 1 -- 10.20.193.16:0/2159091922 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8046070 con 0x7f51dc00ec80 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51daffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51daffd640 10 monclient: dump: 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:57.240 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51daffd640 10 monclient: _finish_auth 0 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51daffd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51daffd640 1 -- 10.20.193.16:0/2159091922 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e8003920 con 0x7f51dc00ec80 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51daffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51daffd640 1 -- 10.20.193.16:0/2159091922 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8003cd0 con 0x7f51dc00ec80 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51fc65b640 5 monclient: authenticate success, global_id 4728 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51fc65b640 10 monclient: _renew_subs 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51f8bcd640 4 set_mon_vals no callback set 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51fc65b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:57.241 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.222+0000 7f51fc65b640 1 -- 10.20.193.16:0/2159091922 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f40724a0 con 0x7f51dc00ec80 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51daffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51daffd640 10 monclient: dump: 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:58:57.242 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:58:57.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51fc65b640 10 client.4728.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:58:57.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51fc65b640 10 monclient: _renew_subs 2026-02-17T05:58:57.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51fc65b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:57.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51fc65b640 1 -- 10.20.193.16:0/2159091922 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f4072a60 con 0x7f51dc00ec80 2026-02-17T05:58:57.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51fc65b640 1 librados: init done 2026-02-17T05:58:57.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51fc65b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:57.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51fc65b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:57.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51fc65b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:57.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.223+0000 7f51fc65b640 1 -- 10.20.193.16:0/2159091922 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f4076e00 con 0x7f51dc00ec80 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.234+0000 7f51daffd640 1 -- 10.20.193.16:0/2159091922 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f51e8053080 con 0x7f51dc00ec80 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.235+0000 7f51daffd640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f51c4079140 0x7f51c407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.235+0000 7f51fa3d0640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f51c4079140 0x7f51c407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.235+0000 7f51fa3d0640 10 monclient: get_auth_request con 0x7f51c4079140 auth_method 0 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.235+0000 7f51fa3d0640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f51c4079140 0x7f51c407b600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51e4015040 tx=0x7f51e4003f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.235+0000 7f51daffd640 10 client.4728.objecter ms_handle_connect 0x7f51c4079140 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.237+0000 7f51daffd640 1 -- 10.20.193.16:0/2159091922 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f51e8082890 con 0x7f51dc00ec80 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.237+0000 7f51daffd640 10 client.4728.objecter ms_dispatch2 0x7f51f4000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.237+0000 7f51daffd640 3 client.4728.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.237+0000 7f51daffd640 3 client.4728.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:58:57.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.237+0000 7f51daffd640 20 client.4728.objecter dump_active .. 0 homeless 2026-02-17T05:58:57.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.342+0000 7f51daffd640 1 -- 10.20.193.16:0/2159091922 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f51e8086850 con 0x7f51dc00ec80 2026-02-17T05:58:57.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.342+0000 7f51daffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:57.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.342+0000 7f51daffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:57.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.445+0000 7f51fc65b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:58:57.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.445+0000 7f51fc65b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:57.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.445+0000 7f51fc65b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:58:57.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.445+0000 7f51fc65b640 1 -- 10.20.193.16:0/2159091922 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f51f41573f0 con 0x7f51dc00ec80 2026-02-17T05:58:57.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.448+0000 7f51daffd640 1 -- 10.20.193.16:0/2159091922 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f51e808ab50 con 0x7f51dc00ec80 2026-02-17T05:58:57.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.448+0000 7f51daffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:58:57.449 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.448+0000 7f51daffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:57.449 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:58:57.449 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:58:57.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 10 librados: watch_flush enter 2026-02-17T05:58:57.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 10 librados: watch_flush exit 2026-02-17T05:58:57.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 20 client.4728.objecter shutdown clearing up homeless session... 2026-02-17T05:58:57.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 10 client.4728.objecter successfully canceled tick 2026-02-17T05:58:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 -- 10.20.193.16:0/2159091922 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f51c4079140 msgr2=0x7f51c407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f51c4079140 0x7f51c407b600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51e4015040 tx=0x7f51e4003f90 comp rx=0 tx=0).stop 2026-02-17T05:58:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 10 monclient: shutdown 2026-02-17T05:58:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 -- 10.20.193.16:0/2159091922 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00ec80 msgr2=0x7f51f41e7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00ec80 0x7f51f41e7650 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f51e802eea0 tx=0x7f51e80047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 -- 10.20.193.16:0/2159091922 shutdown_connections 2026-02-17T05:58:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f51c4079140 0x7f51c407b600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f51dc00f060 0x7f51f4155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f51dc00ec80 0x7f51f41e7650 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 --2- 10.20.193.16:0/2159091922 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f51dc00a380 0x7f51f4154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:57.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 -- 10.20.193.16:0/2159091922 >> 10.20.193.16:0/2159091922 conn(0x7f51f40cd770 msgr2=0x7f51f40cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:57.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 -- 10.20.193.16:0/2159091922 shutdown_connections 2026-02-17T05:58:57.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 -- 10.20.193.16:0/2159091922 wait complete. 2026-02-17T05:58:57.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:58:57.450+0000 7f51fc65b640 1 librados: shutdown 2026-02-17T05:59:00.461 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:00.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.527+0000 7fcb3a349640 10 monclient: get_monmap_and_config 2026-02-17T05:59:00.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.527+0000 7fcb3a349640 10 monclient: build_initial_monmap 2026-02-17T05:59:00.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.527+0000 7fcb3a349640 10 monclient: monmap: 2026-02-17T05:59:00.529 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:00.529 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:00.529 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:00.528557+0000 2026-02-17T05:59:00.529 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:00.528557+0000 2026-02-17T05:59:00.529 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:00.529 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:00.529 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:00.529 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:00.530 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:00.530 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:00.530 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:00.530 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:00.530 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:00.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.527+0000 7fcb3a349640 1 Processor -- start 2026-02-17T05:59:00.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 1 -- start start 2026-02-17T05:59:00.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 10 monclient: init 2026-02-17T05:59:00.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:00.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 10 monclient: _add_conns ranks=[1,3,5,0,2,4] 2026-02-17T05:59:00.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fcb340700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:59:00.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fcb340c86f0 0x7fcb340704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:00.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 10 monclient(hunting): picked mon.noname-a con 0x7fcb340c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:00.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fcb341536f0 0x7fcb34153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 10 monclient(hunting): picked mon.noname-c con 0x7fcb341536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb340c4820 con 0x7fcb340700f0 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcb340c4b00 con 0x7fcb340c86f0 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fcb340c4de0 con 0x7fcb341536f0 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb3a349640 10 monclient(hunting): authenticate will time out at 1193.097576s 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.528+0000 7fcb337fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fcb340c86f0 0x7fcb340704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb33fff640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fcb340700f0 0x7fcb340c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:40644/0 (socket says 10.20.193.16:40644) 2026-02-17T05:59:00.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb33fff640 1 -- 10.20.193.16:0/3905609622 learned_addr learned my addr 10.20.193.16:0/3905609622 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb337fe640 10 monclient(hunting): get_auth_request con 0x7fcb340c86f0 auth_method 0 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb337fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb388bf640 1 --2- 10.20.193.16:0/3905609622 >> v2:10.20.193.148:3300/0 conn(0x7fcb341536f0 0x7fcb34153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb388bf640 10 monclient(hunting): get_auth_request con 0x7fcb341536f0 auth_method 0 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb388bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:00.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb388bf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb388bf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb388bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb388bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.529+0000 7fcb388bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb32ffd640 1 -- 10.20.193.16:0/3905609622 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3617182356 0 0) 0x7fcb340c4820 con 0x7fcb340700f0 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb32ffd640 10 monclient(hunting): _negotiate 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb32ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb32ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb32ffd640 10 monclient(hunting): my global_id is 4813 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb32ffd640 1 -- 10.20.193.16:0/3905609622 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb18003620 con 0x7fcb340700f0 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb337fe640 10 monclient(hunting): handle_auth_done global_id 4731 payload 274 2026-02-17T05:59:00.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb337fe640 1 -- 10.20.193.16:0/3905609622 >> v2:10.20.193.148:3300/0 conn(0x7fcb341536f0 msgr2=0x7fcb34153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb337fe640 1 --2- 10.20.193.16:0/3905609622 >> v2:10.20.193.148:3300/0 conn(0x7fcb341536f0 0x7fcb34153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb337fe640 1 -- 10.20.193.16:0/3905609622 >> v1:10.20.193.128:6789/0 conn(0x7fcb340700f0 legacy=0x7fcb340c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb337fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb337fe640 1 monclient: found mon.noname-a 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb337fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.530+0000 7fcb337fe640 1 -- 10.20.193.16:0/3905609622 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb340c50c0 con 0x7fcb340c86f0 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.531+0000 7fcb337fe640 1 --2- 10.20.193.16:0/3905609622 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 0x7fcb340704d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb340c9540 tx=0x7fcb2402ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d31513b0cd5086d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 1 -- 10.20.193.16:0/3905609622 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2403c070 con 0x7fcb340c86f0 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:00.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 10 monclient: dump: 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 10 monclient: _finish_auth 0 2026-02-17T05:59:00.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 1 -- 10.20.193.16:0/3905609622 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb2402fa60 con 0x7fcb340c86f0 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 1 -- 10.20.193.16:0/3905609622 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2402fe30 con 0x7fcb340c86f0 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 5 monclient: authenticate success, global_id 4731 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 4 set_mon_vals no callback set 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb32ffd640 10 monclient: dump: 2026-02-17T05:59:00.547 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 10 monclient: shutdown 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:00.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 1 -- 10.20.193.16:0/3905609622 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 msgr2=0x7fcb340704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 1 --2- 10.20.193.16:0/3905609622 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 0x7fcb340704d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb340c9540 tx=0x7fcb2402ed50 comp rx=0 tx=0).stop 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 1 -- 10.20.193.16:0/3905609622 shutdown_connections 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 1 --2- 10.20.193.16:0/3905609622 >> v2:10.20.193.148:3300/0 conn(0x7fcb341536f0 0x7fcb34153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 1 --2- 10.20.193.16:0/3905609622 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 0x7fcb340704d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 1 -- 10.20.193.16:0/3905609622 >> 10.20.193.16:0/3905609622 conn(0x7fcb340cd770 msgr2=0x7fcb340cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 1 -- 10.20.193.16:0/3905609622 shutdown_connections 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.533+0000 7fcb3a349640 1 -- 10.20.193.16:0/3905609622 wait complete. 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.534+0000 7fcb3a349640 10 monclient: build_initial_monmap 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.534+0000 7fcb3a349640 10 monclient: monmap: 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:00.549 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.534+0000 7fcb3a349640 1 librados: starting msgr at 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.534+0000 7fcb3a349640 1 librados: starting objecter 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.534+0000 7fcb3a349640 1 Processor -- start 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.534+0000 7fcb3a349640 1 -- start start 2026-02-17T05:59:00.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.534+0000 7fcb3a349640 1 librados: setting wanted keys 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.534+0000 7fcb3a349640 1 librados: calling monclient init 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.534+0000 7fcb3a349640 10 monclient: init 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb340700f0 0x7fcb3415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 10 monclient(hunting): picked mon.noname-c con 0x7fcb340700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 0x7fcb34152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 10 monclient(hunting): picked mon.noname-a con 0x7fcb340c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb341536f0 0x7fcb3406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 10 monclient(hunting): picked mon.noname-b con 0x7fcb341536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:00.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34154060 con 0x7fcb340c86f0 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fcb340c5680 con 0x7fcb341536f0 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34074c00 con 0x7fcb340700f0 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb3a349640 10 monclient(hunting): authenticate will time out at 1193.104185s 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb337fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 0x7fcb34152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb337fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 0x7fcb34152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:50138/0 (socket says 10.20.193.16:50138) 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb337fe640 1 -- 10.20.193.16:0/2856410177 learned_addr learned my addr 10.20.193.16:0/2856410177 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb337fe640 10 monclient(hunting): get_auth_request con 0x7fcb340c86f0 auth_method 0 2026-02-17T05:59:00.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb337fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb33fff640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb340700f0 0x7fcb3415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb388bf640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb341536f0 0x7fcb3406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb33fff640 10 monclient(hunting): get_auth_request con 0x7fcb340700f0 auth_method 0 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb33fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb388bf640 10 monclient(hunting): get_auth_request con 0x7fcb341536f0 auth_method 0 2026-02-17T05:59:00.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb388bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb388bf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb388bf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.535+0000 7fcb33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.536+0000 7fcb33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.536+0000 7fcb388bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.536+0000 7fcb388bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.536+0000 7fcb388bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.536+0000 7fcb337fe640 10 monclient(hunting): handle_auth_done global_id 4734 payload 824 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb337fe640 1 -- 10.20.193.16:0/2856410177 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb340700f0 msgr2=0x7fcb3415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb337fe640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb340700f0 0x7fcb3415d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb337fe640 1 -- 10.20.193.16:0/2856410177 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb341536f0 msgr2=0x7fcb3406d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb337fe640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb341536f0 0x7fcb3406d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb337fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb337fe640 1 monclient: found mon.noname-a 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb337fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb337fe640 1 -- 10.20.193.16:0/2856410177 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3415b7d0 con 0x7fcb340c86f0 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb337fe640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 0x7fcb34152440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009130 tx=0x7fcb240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb30ff9640 10 client.?.objecter ms_handle_connect 0x7fcb340c86f0 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.537+0000 7fcb30ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 1 -- 10.20.193.16:0/2856410177 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24046020 con 0x7fcb340c86f0 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:00.555 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 10 monclient: dump: 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 10 monclient: _finish_auth 0 2026-02-17T05:59:00.556 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 1 -- 10.20.193.16:0/2856410177 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb24042630 con 0x7fcb340c86f0 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb3a349640 5 monclient: authenticate success, global_id 4734 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb3a349640 10 monclient: _renew_subs 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb3a349640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb3a349640 1 -- 10.20.193.16:0/2856410177 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb341f79d0 con 0x7fcb340c86f0 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 1 -- 10.20.193.16:0/2856410177 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2403c040 con 0x7fcb340c86f0 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb327fc640 4 set_mon_vals no callback set 2026-02-17T05:59:00.557 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb30ff9640 10 monclient: dump: 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb3a349640 10 client.4734.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb3a349640 10 monclient: _renew_subs 2026-02-17T05:59:00.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb3a349640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:00.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb3a349640 1 -- 10.20.193.16:0/2856410177 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb341f3cd0 con 0x7fcb340c86f0 2026-02-17T05:59:00.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.539+0000 7fcb3a349640 1 librados: init done 2026-02-17T05:59:00.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.540+0000 7fcb3a349640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:00.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.540+0000 7fcb3a349640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:00.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.540+0000 7fcb3a349640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:00.559 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.540+0000 7fcb3a349640 1 -- 10.20.193.16:0/2856410177 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb340704d0 con 0x7fcb340c86f0 2026-02-17T05:59:00.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.552+0000 7fcb30ff9640 1 -- 10.20.193.16:0/2856410177 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fcb24053070 con 0x7fcb340c86f0 2026-02-17T05:59:00.560 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.553+0000 7fcb30ff9640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb04079190 0x7fcb0407b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:00.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.553+0000 7fcb33fff640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb04079190 0x7fcb0407b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:00.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.553+0000 7fcb33fff640 10 monclient: get_auth_request con 0x7fcb04079190 auth_method 0 2026-02-17T05:59:00.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.553+0000 7fcb33fff640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb04079190 0x7fcb0407b650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb20007b60 tx=0x7fcb20007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:00.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.553+0000 7fcb30ff9640 10 client.4734.objecter ms_handle_connect 0x7fcb04079190 2026-02-17T05:59:00.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.554+0000 7fcb30ff9640 1 -- 10.20.193.16:0/2856410177 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fcb2402f920 con 0x7fcb340c86f0 2026-02-17T05:59:00.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.554+0000 7fcb30ff9640 10 client.4734.objecter ms_dispatch2 0x7fcb34000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:00.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.554+0000 7fcb30ff9640 3 client.4734.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:00.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.554+0000 7fcb30ff9640 3 client.4734.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:00.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.554+0000 7fcb30ff9640 20 client.4734.objecter dump_active .. 0 homeless 2026-02-17T05:59:00.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.657+0000 7fcb30ff9640 1 -- 10.20.193.16:0/2856410177 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcb24050020 con 0x7fcb340c86f0 2026-02-17T05:59:00.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.657+0000 7fcb30ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:00.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.657+0000 7fcb30ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:00.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.759+0000 7fcb3a349640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:00.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.759+0000 7fcb3a349640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:00.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.759+0000 7fcb3a349640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:00.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.759+0000 7fcb3a349640 1 -- 10.20.193.16:0/2856410177 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcb341573f0 con 0x7fcb340c86f0 2026-02-17T05:59:00.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.761+0000 7fcb30ff9640 1 -- 10.20.193.16:0/2856410177 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fcb240877d0 con 0x7fcb340c86f0 2026-02-17T05:59:00.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.761+0000 7fcb30ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:00.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.761+0000 7fcb30ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:00.762 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:00.762 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:00.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 10 librados: watch_flush enter 2026-02-17T05:59:00.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 10 librados: watch_flush exit 2026-02-17T05:59:00.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 20 client.4734.objecter shutdown clearing up homeless session... 2026-02-17T05:59:00.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 10 client.4734.objecter successfully canceled tick 2026-02-17T05:59:00.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 -- 10.20.193.16:0/2856410177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb04079190 msgr2=0x7fcb0407b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:00.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb04079190 0x7fcb0407b650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb20007b60 tx=0x7fcb20007ab0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 10 monclient: shutdown 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 -- 10.20.193.16:0/2856410177 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 msgr2=0x7fcb34152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 0x7fcb34152440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009130 tx=0x7fcb240047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 -- 10.20.193.16:0/2856410177 shutdown_connections 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb04079190 0x7fcb0407b650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb341536f0 0x7fcb3406d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb340c86f0 0x7fcb34152440 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 --2- 10.20.193.16:0/2856410177 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb340700f0 0x7fcb3415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 -- 10.20.193.16:0/2856410177 >> 10.20.193.16:0/2856410177 conn(0x7fcb340cd770 msgr2=0x7fcb340cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 -- 10.20.193.16:0/2856410177 shutdown_connections 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 -- 10.20.193.16:0/2856410177 wait complete. 2026-02-17T05:59:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:00.763+0000 7fcb3a349640 1 librados: shutdown 2026-02-17T05:59:03.773 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:03.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.841+0000 7f97c0722640 10 monclient: get_monmap_and_config 2026-02-17T05:59:03.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.841+0000 7f97c0722640 10 monclient: build_initial_monmap 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.841+0000 7f97c0722640 10 monclient: monmap: 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:03.842265+0000 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:03.842265+0000 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:03.842 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:03.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.841+0000 7f97c0722640 1 Processor -- start 2026-02-17T05:59:03.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.841+0000 7f97c0722640 1 -- start start 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.841+0000 7f97c0722640 10 monclient: init 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 10 monclient: _add_conns ranks=[5,4,0,2,1,3] 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f97b80700f0 0x7f97b80704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 10 monclient(hunting): picked mon.noname-c con 0x7f97b80700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f97b80c7fe0 0x7f97b80c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 10 monclient(hunting): picked mon.noname-b con 0x7f97b80c7fe0 addr v2:10.20.193.128:3300/0 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f97b81536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97b80c8970 con 0x7f97b81536f0 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f97b80c4820 con 0x7f97b80c7fe0 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f97b80c4b00 con 0x7f97b80700f0 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97c0722640 10 monclient(hunting): authenticate will time out at 1196.411346s 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97bec98640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f97b81536f0 0x7f97b8153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:33100/0 (socket says 10.20.193.16:33100) 2026-02-17T05:59:03.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97bec98640 1 -- 10.20.193.16:0/1996884268 learned_addr learned my addr 10.20.193.16:0/1996884268 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97be497640 1 --2- 10.20.193.16:0/1996884268 >> v2:10.20.193.148:3300/0 conn(0x7f97b80700f0 0x7f97b80704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97bdc96640 1 --2- 10.20.193.16:0/1996884268 >> v2:10.20.193.128:3300/0 conn(0x7f97b80c7fe0 0x7f97b80c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97be497640 10 monclient(hunting): get_auth_request con 0x7f97b80700f0 auth_method 0 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97be497640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97be497640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.842+0000 7f97be497640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bdc96640 10 monclient(hunting): get_auth_request con 0x7f97b80c7fe0 auth_method 0 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bdc96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bdc96640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bdc96640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97be497640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97be497640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97be497640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bdc96640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bdc96640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bdc96640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bd495640 1 -- 10.20.193.16:0/1996884268 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069020692 0 0) 0x7f97b80c8970 con 0x7f97b81536f0 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bd495640 10 monclient(hunting): _negotiate 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bd495640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:03.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bd495640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bd495640 10 monclient(hunting): my global_id is 4737 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.843+0000 7f97bd495640 1 -- 10.20.193.16:0/1996884268 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97a0003620 con 0x7f97b81536f0 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.844+0000 7f97be497640 10 monclient(hunting): handle_auth_done global_id 4868 payload 274 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.844+0000 7f97be497640 1 -- 10.20.193.16:0/1996884268 >> v2:10.20.193.128:3300/0 conn(0x7f97b80c7fe0 msgr2=0x7f97b80c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.844+0000 7f97be497640 1 --2- 10.20.193.16:0/1996884268 >> v2:10.20.193.128:3300/0 conn(0x7f97b80c7fe0 0x7f97b80c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.844+0000 7f97be497640 1 -- 10.20.193.16:0/1996884268 >> v1:10.20.193.16:6789/0 conn(0x7f97b81536f0 legacy=0x7f97b8153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.844+0000 7f97be497640 10 monclient: _finish_hunting 0 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.844+0000 7f97be497640 1 monclient: found mon.noname-c 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.844+0000 7f97be497640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.844+0000 7f97be497640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:59:03.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.844+0000 7f97be497640 1 -- 10.20.193.16:0/1996884268 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b80c4de0 con 0x7f97b80700f0 2026-02-17T05:59:03.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.845+0000 7f97be497640 1 --2- 10.20.193.16:0/1996884268 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b80700f0 0x7f97b80704d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f97b8070b90 tx=0x7f97a802ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de30f1114591257 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:03.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 1 -- 10.20.193.16:0/1996884268 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a803c070 con 0x7f97b80700f0 2026-02-17T05:59:03.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:03.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:03.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 10 monclient: dump: 2026-02-17T05:59:03.850 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:03.850 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:03.850 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:03.850 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 10 monclient: _finish_auth 0 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 10 monclient: _check_auth_tickets 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97c0722640 5 monclient: authenticate success, global_id 4868 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97c0722640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 1 -- 10.20.193.16:0/1996884268 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a802fa60 con 0x7f97b80700f0 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 1 -- 10.20.193.16:0/1996884268 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a802fe10 con 0x7f97b80700f0 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97c0722640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97c0722640 4 set_mon_vals no callback set 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.846+0000 7f97bd495640 10 monclient: dump: 2026-02-17T05:59:03.851 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 10 monclient: shutdown 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 -- 10.20.193.16:0/1996884268 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b80700f0 msgr2=0x7f97b80704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 --2- 10.20.193.16:0/1996884268 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b80700f0 0x7f97b80704d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f97b8070b90 tx=0x7f97a802ed50 comp rx=0 tx=0).stop 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 -- 10.20.193.16:0/1996884268 shutdown_connections 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 --2- 10.20.193.16:0/1996884268 >> v2:10.20.193.128:3300/0 conn(0x7f97b80c7fe0 0x7f97b80c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 --2- 10.20.193.16:0/1996884268 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b80700f0 0x7f97b80704d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 -- 10.20.193.16:0/1996884268 >> 10.20.193.16:0/1996884268 conn(0x7f97b80cd770 msgr2=0x7f97b80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 -- 10.20.193.16:0/1996884268 shutdown_connections 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 -- 10.20.193.16:0/1996884268 wait complete. 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 10 monclient: build_initial_monmap 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 10 monclient: monmap: 2026-02-17T05:59:03.852 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 librados: starting msgr at 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 librados: starting objecter 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 Processor -- start 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 -- start start 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 librados: setting wanted keys 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 1 librados: calling monclient init 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.847+0000 7f97c0722640 10 monclient: init 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b80700f0 0x7f97b8073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:03.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 10 monclient(hunting): picked mon.noname-a con 0x7f97b80700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b80c7fe0 0x7f97b8073a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 10 monclient(hunting): picked mon.noname-c con 0x7f97b80c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b81536f0 0x7f97b81f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 10 monclient(hunting): picked mon.noname-b con 0x7f97b81536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97b80c5680 con 0x7f97b80700f0 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f97b80c53a0 con 0x7f97b81536f0 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97b8074c00 con 0x7f97b80c7fe0 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97c0722640 10 monclient(hunting): authenticate will time out at 1196.417316s 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97be497640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b80700f0 0x7f97b8073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97bdc96640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b80c7fe0 0x7f97b8073a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97be497640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b80700f0 0x7f97b8073540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:50144/0 (socket says 10.20.193.16:50144) 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97be497640 1 -- 10.20.193.16:0/278869158 learned_addr learned my addr 10.20.193.16:0/278869158 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97be497640 10 monclient(hunting): get_auth_request con 0x7f97b80700f0 auth_method 0 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97be497640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97be497640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97be497640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97bec98640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b81536f0 0x7f97b81f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97bdc96640 10 monclient(hunting): get_auth_request con 0x7f97b80c7fe0 auth_method 0 2026-02-17T05:59:03.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97bdc96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97bdc96640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97bdc96640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97bec98640 10 monclient(hunting): get_auth_request con 0x7f97b81536f0 auth_method 0 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97bec98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97bec98640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97bec98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97be497640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97be497640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.848+0000 7f97be497640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.849+0000 7f97bdc96640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.849+0000 7f97bdc96640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.849+0000 7f97bdc96640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.849+0000 7f97bec98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.849+0000 7f97bec98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.849+0000 7f97bec98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.850+0000 7f97be497640 10 monclient(hunting): handle_auth_done global_id 4740 payload 824 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.850+0000 7f97be497640 1 -- 10.20.193.16:0/278869158 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b80c7fe0 msgr2=0x7f97b8073a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.850+0000 7f97be497640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b80c7fe0 0x7f97b8073a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.850+0000 7f97be497640 1 -- 10.20.193.16:0/278869158 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b81536f0 msgr2=0x7f97b81f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.850+0000 7f97be497640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b81536f0 0x7f97b81f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.850+0000 7f97be497640 10 monclient: _finish_hunting 0 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.850+0000 7f97be497640 1 monclient: found mon.noname-a 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.850+0000 7f97be497640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.850+0000 7f97be497640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:03.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.850+0000 7f97be497640 1 -- 10.20.193.16:0/278869158 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b80724a0 con 0x7f97b80700f0 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.851+0000 7f97be497640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b80700f0 0x7f97b8073540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f97a803a040 tx=0x7f97a80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.851+0000 7f979effd640 10 client.?.objecter ms_handle_connect 0x7f97b80700f0 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.851+0000 7f979effd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 1 -- 10.20.193.16:0/278869158 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a803c040 con 0x7f97b80700f0 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 10 monclient: dump: 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 10 monclient: _finish_auth 0 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 1 -- 10.20.193.16:0/278869158 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a8003910 con 0x7f97b80700f0 2026-02-17T05:59:03.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 1 -- 10.20.193.16:0/278869158 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8003d00 con 0x7f97b80700f0 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97bcc94640 4 set_mon_vals no callback set 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 5 monclient: authenticate success, global_id 4740 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 10 monclient: _renew_subs 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 1 -- 10.20.193.16:0/278869158 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b8072780 con 0x7f97b80700f0 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f979effd640 10 monclient: dump: 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 10 client.4740.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 10 monclient: _renew_subs 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 1 -- 10.20.193.16:0/278869158 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b8072d40 con 0x7f97b80700f0 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 1 librados: init done 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:03.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:03.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:03.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.853+0000 7f97c0722640 1 -- 10.20.193.16:0/278869158 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b806f500 con 0x7f97b80700f0 2026-02-17T05:59:03.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.865+0000 7f979effd640 1 -- 10.20.193.16:0/278869158 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f97a8042430 con 0x7f97b80700f0 2026-02-17T05:59:03.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.865+0000 7f979effd640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9788079190 0x7f978807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:03.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.865+0000 7f97bdc96640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9788079190 0x7f978807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:03.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.865+0000 7f97bdc96640 10 monclient: get_auth_request con 0x7f9788079190 auth_method 0 2026-02-17T05:59:03.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.865+0000 7f97bdc96640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9788079190 0x7f978807b650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f97b8074120 tx=0x7f97ac009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:03.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.865+0000 7f979effd640 10 client.4740.objecter ms_handle_connect 0x7f9788079190 2026-02-17T05:59:03.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.969+0000 7f979effd640 1 -- 10.20.193.16:0/278869158 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f97a8083110 con 0x7f97b80700f0 2026-02-17T05:59:03.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.969+0000 7f979effd640 10 client.4740.objecter ms_dispatch2 0x7f97b8000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:03.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.969+0000 7f979effd640 3 client.4740.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:03.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.969+0000 7f979effd640 3 client.4740.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:03.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.969+0000 7f979effd640 20 client.4740.objecter dump_active .. 0 homeless 2026-02-17T05:59:03.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.969+0000 7f979effd640 1 -- 10.20.193.16:0/278869158 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f97b806f500 con 0x7f97b80700f0 2026-02-17T05:59:03.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.969+0000 7f979effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:03.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:03.969+0000 7f979effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:04.023 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:04:42.851 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:04.027 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:04:42.854 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:04.037 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:04:42.864 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:04.054 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:04:42.879 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:04.054 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:04:42.880 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:04.055 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:04:42.843 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:04.055 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:04:42.844 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:04.064 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:04:42.890 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:04.066 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:04:42.855 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:04.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.074+0000 7f97c0722640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:04.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.074+0000 7f97c0722640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:04.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.074+0000 7f97c0722640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:04.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.074+0000 7f97c0722640 1 -- 10.20.193.16:0/278869158 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f97b80704d0 con 0x7f97b80700f0 2026-02-17T05:59:04.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.078+0000 7f979effd640 1 -- 10.20.193.16:0/278869158 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f97a80870d0 con 0x7f97b80700f0 2026-02-17T05:59:04.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.078+0000 7f979effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:04.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.078+0000 7f979effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:04.078 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:04.078 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:04.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 10 librados: watch_flush enter 2026-02-17T05:59:04.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 10 librados: watch_flush exit 2026-02-17T05:59:04.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 20 client.4740.objecter shutdown clearing up homeless session... 2026-02-17T05:59:04.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 10 client.4740.objecter successfully canceled tick 2026-02-17T05:59:04.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 -- 10.20.193.16:0/278869158 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9788079190 msgr2=0x7f978807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:04.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9788079190 0x7f978807b650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f97b8074120 tx=0x7f97ac009290 comp rx=0 tx=0).stop 2026-02-17T05:59:04.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 10 monclient: shutdown 2026-02-17T05:59:04.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:04.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 -- 10.20.193.16:0/278869158 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b80700f0 msgr2=0x7f97b8073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:04.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b80700f0 0x7f97b8073540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f97a803a040 tx=0x7f97a80047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 -- 10.20.193.16:0/278869158 shutdown_connections 2026-02-17T05:59:04.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9788079190 0x7f978807b650 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f97b81536f0 0x7f97b81f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f97b80c7fe0 0x7f97b8073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 --2- 10.20.193.16:0/278869158 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f97b80700f0 0x7f97b8073540 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 -- 10.20.193.16:0/278869158 >> 10.20.193.16:0/278869158 conn(0x7f97b80cd770 msgr2=0x7f97b814e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:04.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 -- 10.20.193.16:0/278869158 shutdown_connections 2026-02-17T05:59:04.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 -- 10.20.193.16:0/278869158 wait complete. 2026-02-17T05:59:04.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:04.080+0000 7f97c0722640 1 librados: shutdown 2026-02-17T05:59:07.089 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:07.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.157+0000 7efe85ff8640 10 monclient: get_monmap_and_config 2026-02-17T05:59:07.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.157+0000 7efe85ff8640 10 monclient: build_initial_monmap 2026-02-17T05:59:07.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.157+0000 7efe85ff8640 10 monclient: monmap: 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:07.158514+0000 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:07.158514+0000 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:07.159 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:07.160 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:07.160 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:07.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.157+0000 7efe85ff8640 1 Processor -- start 2026-02-17T05:59:07.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.157+0000 7efe85ff8640 1 -- start start 2026-02-17T05:59:07.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.157+0000 7efe85ff8640 10 monclient: init 2026-02-17T05:59:07.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:07.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 10 monclient: _add_conns ranks=[5,4,3,0,1,2] 2026-02-17T05:59:07.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7efe800700f0 0x7efe800704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 10 monclient(hunting): picked mon.noname-c con 0x7efe800700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:07.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7efe800c7fe0 0x7efe800c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.163 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 10 monclient(hunting): picked mon.noname-b con 0x7efe800c7fe0 addr v2:10.20.193.128:3300/0 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7efe801536f0 0x7efe80153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 10 monclient(hunting): picked mon.noname-a con 0x7efe801536f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7efe800c8970 con 0x7efe801536f0 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7efe800c4820 con 0x7efe800c7fe0 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7efe800c4b00 con 0x7efe800700f0 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe85ff8640 10 monclient(hunting): authenticate will time out at 1199.727471s 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe7ffff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7efe801536f0 0x7efe80153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe7effd640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7efe800c7fe0 0x7efe800c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.158+0000 7efe7f7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7efe800700f0 0x7efe800704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7ffff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7efe801536f0 0x7efe80153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:50150/0 (socket says 10.20.193.16:50150) 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7ffff640 1 -- 10.20.193.16:0/153114336 learned_addr learned my addr 10.20.193.16:0/153114336 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7ffff640 10 monclient(hunting): get_auth_request con 0x7efe801536f0 auth_method 0 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7f7fe640 10 monclient(hunting): get_auth_request con 0x7efe800700f0 auth_method 0 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7effd640 10 monclient(hunting): get_auth_request con 0x7efe800c7fe0 auth_method 0 2026-02-17T05:59:07.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.159+0000 7efe7effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.160+0000 7efe7ffff640 10 monclient(hunting): handle_auth_done global_id 4743 payload 274 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.160+0000 7efe7ffff640 1 -- 10.20.193.16:0/153114336 >> v2:10.20.193.148:3300/0 conn(0x7efe800700f0 msgr2=0x7efe800704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.160+0000 7efe7ffff640 1 --2- 10.20.193.16:0/153114336 >> v2:10.20.193.148:3300/0 conn(0x7efe800700f0 0x7efe800704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.160+0000 7efe7ffff640 1 -- 10.20.193.16:0/153114336 >> v2:10.20.193.128:3300/0 conn(0x7efe800c7fe0 msgr2=0x7efe800c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.160+0000 7efe7ffff640 1 --2- 10.20.193.16:0/153114336 >> v2:10.20.193.128:3300/0 conn(0x7efe800c7fe0 0x7efe800c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.160+0000 7efe7ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.160+0000 7efe7ffff640 1 monclient: found mon.noname-a 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.160+0000 7efe7ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:07.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.160+0000 7efe7ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:07.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.160+0000 7efe7ffff640 1 -- 10.20.193.16:0/153114336 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe800c4de0 con 0x7efe801536f0 2026-02-17T05:59:07.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.161+0000 7efe7ffff640 1 --2- 10.20.193.16:0/153114336 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efe801536f0 0x7efe80153b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe800c5b80 tx=0x7efe7402f200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d64604c9c68f9d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 1 -- 10.20.193.16:0/153114336 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7402fc70 con 0x7efe801536f0 2026-02-17T05:59:07.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:07.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 10 monclient: dump: 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:07.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 1 -- 10.20.193.16:0/153114336 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7402fe10 con 0x7efe801536f0 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 5 monclient: authenticate success, global_id 4743 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 1 -- 10.20.193.16:0/153114336 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74037610 con 0x7efe801536f0 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 4 set_mon_vals no callback set 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:07.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe7e7fc640 10 monclient: dump: 2026-02-17T05:59:07.177 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:07.177 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:07.177 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:07.177 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:07.177 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.177 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:07.177 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:07.177 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:07.177 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:07.177 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 10 monclient: shutdown 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 1 -- 10.20.193.16:0/153114336 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efe801536f0 msgr2=0x7efe80153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 1 --2- 10.20.193.16:0/153114336 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efe801536f0 0x7efe80153b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe800c5b80 tx=0x7efe7402f200 comp rx=0 tx=0).stop 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 1 -- 10.20.193.16:0/153114336 shutdown_connections 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 1 --2- 10.20.193.16:0/153114336 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efe801536f0 0x7efe80153b00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 1 --2- 10.20.193.16:0/153114336 >> v2:10.20.193.128:3300/0 conn(0x7efe800c7fe0 0x7efe800c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 1 --2- 10.20.193.16:0/153114336 >> v2:10.20.193.148:3300/0 conn(0x7efe800700f0 0x7efe800704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 1 -- 10.20.193.16:0/153114336 >> 10.20.193.16:0/153114336 conn(0x7efe800cd770 msgr2=0x7efe800cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 1 -- 10.20.193.16:0/153114336 shutdown_connections 2026-02-17T05:59:07.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.163+0000 7efe85ff8640 1 -- 10.20.193.16:0/153114336 wait complete. 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.164+0000 7efe85ff8640 10 monclient: build_initial_monmap 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.164+0000 7efe85ff8640 10 monclient: monmap: 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:07.179 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.164+0000 7efe85ff8640 1 librados: starting msgr at 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.164+0000 7efe85ff8640 1 librados: starting objecter 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.164+0000 7efe85ff8640 1 Processor -- start 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.164+0000 7efe85ff8640 1 -- start start 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.164+0000 7efe85ff8640 1 librados: setting wanted keys 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.164+0000 7efe85ff8640 1 librados: calling monclient init 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.164+0000 7efe85ff8640 10 monclient: init 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efe800700f0 0x7efe801448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 10 monclient(hunting): picked mon.noname-c con 0x7efe800700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efe800c7fe0 0x7efe8014e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 10 monclient(hunting): picked mon.noname-a con 0x7efe800c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efe801536f0 0x7efe801e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 10 monclient(hunting): picked mon.noname-b con 0x7efe801536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe800c5680 con 0x7efe800c7fe0 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7efe800c53a0 con 0x7efe801536f0 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe80154040 con 0x7efe800700f0 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe85ff8640 10 monclient(hunting): authenticate will time out at 1199.734190s 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7effd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efe800c7fe0 0x7efe8014e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7effd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efe800c7fe0 0x7efe8014e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:50166/0 (socket says 10.20.193.16:50166) 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7ffff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efe801536f0 0x7efe801e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7effd640 1 -- 10.20.193.16:0/3159477327 learned_addr learned my addr 10.20.193.16:0/3159477327 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7effd640 10 monclient(hunting): get_auth_request con 0x7efe800c7fe0 auth_method 0 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7f7fe640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efe800700f0 0x7efe801448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7ffff640 10 monclient(hunting): get_auth_request con 0x7efe801536f0 auth_method 0 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.182 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7f7fe640 10 monclient(hunting): get_auth_request con 0x7efe800700f0 auth_method 0 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.165+0000 7efe7f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.166+0000 7efe7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.166+0000 7efe7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.166+0000 7efe7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.167+0000 7efe7ffff640 10 monclient(hunting): handle_auth_done global_id 4828 payload 824 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.167+0000 7efe7ffff640 1 -- 10.20.193.16:0/3159477327 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efe800700f0 msgr2=0x7efe801448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.167+0000 7efe7ffff640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efe800700f0 0x7efe801448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.167+0000 7efe7ffff640 1 -- 10.20.193.16:0/3159477327 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efe800c7fe0 msgr2=0x7efe8014e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.167+0000 7efe7ffff640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efe800c7fe0 0x7efe8014e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.167+0000 7efe7ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.167+0000 7efe7ffff640 1 monclient: found mon.noname-b 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.167+0000 7efe7ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.167+0000 7efe7ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.167+0000 7efe7ffff640 1 -- 10.20.193.16:0/3159477327 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe801f7ad0 con 0x7efe801536f0 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.169+0000 7efe7ffff640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efe801536f0 0x7efe801e4f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe74035000 tx=0x7efe74004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.169+0000 7efe5bfff640 10 client.?.objecter ms_handle_connect 0x7efe801536f0 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.169+0000 7efe5bfff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe5bfff640 1 -- 10.20.193.16:0/3159477327 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74046070 con 0x7efe801536f0 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe5bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe5bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe5bfff640 10 monclient: dump: 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:07.184 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe5bfff640 10 monclient: _finish_auth 0 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe5bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe5bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 5 monclient: authenticate success, global_id 4828 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 10 monclient: _renew_subs 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 1 -- 10.20.193.16:0/3159477327 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe801f3af0 con 0x7efe801536f0 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 10 client.4828.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:07.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 10 monclient: _renew_subs 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 1 -- 10.20.193.16:0/3159477327 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe801f40b0 con 0x7efe801536f0 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 1 librados: init done 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.170+0000 7efe85ff8640 1 -- 10.20.193.16:0/3159477327 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe801f47b0 con 0x7efe801536f0 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.171+0000 7efe5bfff640 1 -- 10.20.193.16:0/3159477327 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe74042640 con 0x7efe801536f0 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.171+0000 7efe5bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.171+0000 7efe5bfff640 1 -- 10.20.193.16:0/3159477327 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7404f060 con 0x7efe801536f0 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.171+0000 7efe5bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.171+0000 7efe5bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.171+0000 7efe7dffb640 4 set_mon_vals no callback set 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.171+0000 7efe5bfff640 10 monclient: dump: 2026-02-17T05:59:07.186 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:07.187 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:07.187 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:07.187 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:07.187 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.187 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:07.187 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:07.187 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:07.187 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:07.187 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:07.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.277+0000 7efe5bfff640 1 -- 10.20.193.16:0/3159477327 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7efe74042a30 con 0x7efe801536f0 2026-02-17T05:59:07.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.277+0000 7efe5bfff640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efe50079140 0x7efe5007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.277+0000 7efe7f7fe640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efe50079140 0x7efe5007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.277+0000 7efe7f7fe640 10 monclient: get_auth_request con 0x7efe50079140 auth_method 0 2026-02-17T05:59:07.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.278+0000 7efe5bfff640 10 client.4828.objecter ms_handle_connect 0x7efe50079140 2026-02-17T05:59:07.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.278+0000 7efe7f7fe640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efe50079140 0x7efe5007b600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe80157960 tx=0x7efe6c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.278+0000 7efe5bfff640 1 -- 10.20.193.16:0/3159477327 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7efe74083a20 con 0x7efe801536f0 2026-02-17T05:59:07.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.278+0000 7efe5bfff640 10 client.4828.objecter ms_dispatch2 0x7efe80000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:07.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.278+0000 7efe5bfff640 3 client.4828.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:07.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.278+0000 7efe5bfff640 3 client.4828.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:07.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.278+0000 7efe5bfff640 20 client.4828.objecter dump_active .. 0 homeless 2026-02-17T05:59:07.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.284+0000 7efe5bfff640 1 -- 10.20.193.16:0/3159477327 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efe801f47b0 con 0x7efe801536f0 2026-02-17T05:59:07.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.284+0000 7efe5bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:07.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.284+0000 7efe5bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:07.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.383+0000 7efe85ff8640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:07.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.383+0000 7efe85ff8640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:07.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.383+0000 7efe85ff8640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:59:07.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.383+0000 7efe85ff8640 1 -- 10.20.193.16:0/3159477327 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efe80074c00 con 0x7efe801536f0 2026-02-17T05:59:07.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.386+0000 7efe5bfff640 1 -- 10.20.193.16:0/3159477327 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7efe740879e0 con 0x7efe801536f0 2026-02-17T05:59:07.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.386+0000 7efe5bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:07.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.386+0000 7efe5bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:07.386 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:07.386 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:07.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.386+0000 7efe85ff8640 10 librados: watch_flush enter 2026-02-17T05:59:07.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.386+0000 7efe85ff8640 10 librados: watch_flush exit 2026-02-17T05:59:07.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.386+0000 7efe85ff8640 20 client.4828.objecter shutdown clearing up homeless session... 2026-02-17T05:59:07.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.386+0000 7efe85ff8640 10 client.4828.objecter successfully canceled tick 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 -- 10.20.193.16:0/3159477327 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efe50079140 msgr2=0x7efe5007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efe50079140 0x7efe5007b600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe80157960 tx=0x7efe6c009290 comp rx=0 tx=0).stop 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 10 monclient: shutdown 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 -- 10.20.193.16:0/3159477327 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efe801536f0 msgr2=0x7efe801e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efe801536f0 0x7efe801e4f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe74035000 tx=0x7efe74004b40 comp rx=0 tx=0).stop 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 -- 10.20.193.16:0/3159477327 shutdown_connections 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efe50079140 0x7efe5007b600 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efe801536f0 0x7efe801e4f70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efe800c7fe0 0x7efe8014e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 --2- 10.20.193.16:0/3159477327 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efe800700f0 0x7efe801448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 -- 10.20.193.16:0/3159477327 >> 10.20.193.16:0/3159477327 conn(0x7efe800cd770 msgr2=0x7efe800c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 -- 10.20.193.16:0/3159477327 shutdown_connections 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 -- 10.20.193.16:0/3159477327 wait complete. 2026-02-17T05:59:07.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:07.387+0000 7efe85ff8640 1 librados: shutdown 2026-02-17T05:59:10.396 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:10.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.464+0000 7f010f6c6640 10 monclient: get_monmap_and_config 2026-02-17T05:59:10.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.464+0000 7f010f6c6640 10 monclient: build_initial_monmap 2026-02-17T05:59:10.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.464+0000 7f010f6c6640 10 monclient: monmap: 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:10.465451+0000 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:10.465451+0000 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:10.466 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:10.467 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:10.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.464+0000 7f010f6c6640 1 Processor -- start 2026-02-17T05:59:10.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.464+0000 7f010f6c6640 1 -- start start 2026-02-17T05:59:10.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.464+0000 7f010f6c6640 10 monclient: init 2026-02-17T05:59:10.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:10.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 10 monclient: _add_conns ranks=[5,0,4,2,3,1] 2026-02-17T05:59:10.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f01080700f0 0x7f01080704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:10.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 10 monclient(hunting): picked mon.noname-c con 0x7f01080700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:10.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f01080c7fe0 addr v1:10.20.193.16:6789/0 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f01081536f0 0x7f0108153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 10 monclient(hunting): picked mon.noname-b con 0x7f01081536f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01080c4820 con 0x7f01080c7fe0 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f01080c4b00 con 0x7f01081536f0 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f01080c4de0 con 0x7f01080700f0 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010f6c6640 10 monclient(hunting): authenticate will time out at 1203.034513s 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010cc3a640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f01080c7fe0 0x7f01080c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:44334/0 (socket says 10.20.193.16:44334) 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.465+0000 7f010cc3a640 1 -- 10.20.193.16:0/2618502530 learned_addr learned my addr 10.20.193.16:0/2618502530 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:10.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010d43b640 1 --2- 10.20.193.16:0/2618502530 >> v2:10.20.193.148:3300/0 conn(0x7f01080700f0 0x7f01080704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010dc3c640 1 --2- 10.20.193.16:0/2618502530 >> v2:10.20.193.128:3300/0 conn(0x7f01081536f0 0x7f0108153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010dc3c640 10 monclient(hunting): get_auth_request con 0x7f01081536f0 auth_method 0 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010dc3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010dc3c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010dc3c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010d43b640 10 monclient(hunting): get_auth_request con 0x7f01080700f0 auth_method 0 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010d43b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010d43b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010d43b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010dc3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010dc3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:10.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010dc3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010d43b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010d43b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.466+0000 7f010d43b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f00f7fff640 1 -- 10.20.193.16:0/2618502530 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 486015646 0 0) 0x7f01080c4820 con 0x7f01080c7fe0 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f00f7fff640 10 monclient(hunting): _negotiate 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f00f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f00f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f00f7fff640 10 monclient(hunting): my global_id is 4749 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f00f7fff640 1 -- 10.20.193.16:0/2618502530 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00f0003620 con 0x7f01080c7fe0 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f010dc3c640 10 monclient(hunting): handle_auth_done global_id 4831 payload 274 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f010dc3c640 1 -- 10.20.193.16:0/2618502530 >> v2:10.20.193.148:3300/0 conn(0x7f01080700f0 msgr2=0x7f01080704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f010dc3c640 1 --2- 10.20.193.16:0/2618502530 >> v2:10.20.193.148:3300/0 conn(0x7f01080700f0 0x7f01080704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f010dc3c640 1 -- 10.20.193.16:0/2618502530 >> v1:10.20.193.16:6789/0 conn(0x7f01080c7fe0 legacy=0x7f01080c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f010dc3c640 10 monclient: _finish_hunting 0 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f010dc3c640 1 monclient: found mon.noname-b 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f010dc3c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f010dc3c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.467+0000 7f010dc3c640 1 -- 10.20.193.16:0/2618502530 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01080c50c0 con 0x7f01081536f0 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.468+0000 7f010dc3c640 1 --2- 10.20.193.16:0/2618502530 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01081536f0 0x7f0108153b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0108070b90 tx=0x7f010402f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c05eaeb28ec72ce9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.469+0000 7f00f7fff640 1 -- 10.20.193.16:0/2618502530 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010403c070 con 0x7f01081536f0 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.469+0000 7f00f7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.469+0000 7f00f7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.469+0000 7f00f7fff640 10 monclient: dump: 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:10.474 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.469+0000 7f00f7fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.469+0000 7f00f7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.469+0000 7f00f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.469+0000 7f010f6c6640 5 monclient: authenticate success, global_id 4831 2026-02-17T05:59:10.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.469+0000 7f010f6c6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:10.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.470+0000 7f00f7fff640 1 -- 10.20.193.16:0/2618502530 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f010402fe40 con 0x7f01081536f0 2026-02-17T05:59:10.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.470+0000 7f00f7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:10.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.470+0000 7f00f7fff640 1 -- 10.20.193.16:0/2618502530 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104037640 con 0x7f01081536f0 2026-02-17T05:59:10.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.470+0000 7f00f7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:10.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.470+0000 7f00f7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.470+0000 7f00f7fff640 10 monclient: dump: 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:10.483 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.470+0000 7f010f6c6640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.470+0000 7f010f6c6640 4 set_mon_vals no callback set 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 10 monclient: shutdown 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 -- 10.20.193.16:0/2618502530 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01081536f0 msgr2=0x7f0108153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 --2- 10.20.193.16:0/2618502530 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01081536f0 0x7f0108153b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0108070b90 tx=0x7f010402f130 comp rx=0 tx=0).stop 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 -- 10.20.193.16:0/2618502530 shutdown_connections 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 --2- 10.20.193.16:0/2618502530 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01081536f0 0x7f0108153b20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 --2- 10.20.193.16:0/2618502530 >> v2:10.20.193.148:3300/0 conn(0x7f01080700f0 0x7f01080704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 -- 10.20.193.16:0/2618502530 >> 10.20.193.16:0/2618502530 conn(0x7f01080cd770 msgr2=0x7f01080cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 -- 10.20.193.16:0/2618502530 shutdown_connections 2026-02-17T05:59:10.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 -- 10.20.193.16:0/2618502530 wait complete. 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 10 monclient: build_initial_monmap 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 10 monclient: monmap: 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:10.485 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 librados: starting msgr at 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 librados: starting objecter 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 Processor -- start 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 -- start start 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 librados: setting wanted keys 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 1 librados: calling monclient init 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.471+0000 7f010f6c6640 10 monclient: init 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f01080700f0 0x7f0108143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 10 monclient(hunting): picked mon.noname-c con 0x7f01080700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:10.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01080c7fe0 0x7f0108141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 10 monclient(hunting): picked mon.noname-a con 0x7f01080c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01081536f0 0x7f01081f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 10 monclient(hunting): picked mon.noname-b con 0x7f01081536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0108154060 con 0x7f01080c7fe0 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f01080c5680 con 0x7f01081536f0 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0108074c00 con 0x7f01080700f0 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010f6c6640 10 monclient(hunting): authenticate will time out at 1203.041219s 2026-02-17T05:59:10.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010cc3a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01080c7fe0 0x7f0108141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010cc3a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01080c7fe0 0x7f0108141440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:38968/0 (socket says 10.20.193.16:38968) 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010cc3a640 1 -- 10.20.193.16:0/3449147438 learned_addr learned my addr 10.20.193.16:0/3449147438 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010cc3a640 10 monclient(hunting): get_auth_request con 0x7f01080c7fe0 auth_method 0 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010cc3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010cc3a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010cc3a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010d43b640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f01080700f0 0x7f0108143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010dc3c640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01081536f0 0x7f01081f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010d43b640 10 monclient(hunting): get_auth_request con 0x7f01080700f0 auth_method 0 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010d43b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010d43b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010d43b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:10.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010cc3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010cc3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010cc3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010dc3c640 10 monclient(hunting): get_auth_request con 0x7f01081536f0 auth_method 0 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010dc3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010dc3c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.472+0000 7f010dc3c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.473+0000 7f010d43b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.473+0000 7f010d43b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.473+0000 7f010d43b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.473+0000 7f010dc3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.473+0000 7f010dc3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:10.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.473+0000 7f010dc3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.474+0000 7f010cc3a640 10 monclient(hunting): handle_auth_done global_id 4752 payload 824 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.474+0000 7f010cc3a640 1 -- 10.20.193.16:0/3449147438 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f01080700f0 msgr2=0x7f0108143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.474+0000 7f010cc3a640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f01080700f0 0x7f0108143390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.474+0000 7f010cc3a640 1 -- 10.20.193.16:0/3449147438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01081536f0 msgr2=0x7f01081f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.474+0000 7f010cc3a640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01081536f0 0x7f01081f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.474+0000 7f010cc3a640 10 monclient: _finish_hunting 0 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.474+0000 7f010cc3a640 1 monclient: found mon.noname-a 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.474+0000 7f010cc3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.474+0000 7f010cc3a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.474+0000 7f010cc3a640 1 -- 10.20.193.16:0/3449147438 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01080724a0 con 0x7f01080c7fe0 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.475+0000 7f010cc3a640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01080c7fe0 0x7f0108141440 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00c520 tx=0x7f00fc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.475+0000 7f00f5ffb640 10 client.?.objecter ms_handle_connect 0x7f01080c7fe0 2026-02-17T05:59:10.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.475+0000 7f00f5ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f00f5ffb640 1 -- 10.20.193.16:0/3449147438 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc017070 con 0x7f01080c7fe0 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f00f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f00f5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f00f5ffb640 10 monclient: dump: 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:10.491 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f00f5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f00f5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f00f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 5 monclient: authenticate success, global_id 4752 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 10 monclient: _renew_subs 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 1 -- 10.20.193.16:0/3449147438 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0108072740 con 0x7f01080c7fe0 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 10 client.4752.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 10 monclient: _renew_subs 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:10.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 1 -- 10.20.193.16:0/3449147438 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0108072d00 con 0x7f01080c7fe0 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 1 librados: init done 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.476+0000 7f010f6c6640 1 -- 10.20.193.16:0/3449147438 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01080704d0 con 0x7f01080c7fe0 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.477+0000 7f00f5ffb640 1 -- 10.20.193.16:0/3449147438 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00fc002a20 con 0x7f01080c7fe0 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.477+0000 7f00f5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.477+0000 7f00f5ffb640 1 -- 10.20.193.16:0/3449147438 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc021060 con 0x7f01080c7fe0 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.477+0000 7f00f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.477+0000 7f00f5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.477+0000 7f00f77fe640 4 set_mon_vals no callback set 2026-02-17T05:59:10.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.477+0000 7f00f5ffb640 10 monclient: dump: 2026-02-17T05:59:10.494 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:10.494 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:10.494 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:10.494 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:10.494 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:10.494 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:10.494 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:10.494 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:10.494 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:10.494 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:10.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.583+0000 7f00f5ffb640 1 -- 10.20.193.16:0/3449147438 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f00fc024080 con 0x7f01080c7fe0 2026-02-17T05:59:10.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.583+0000 7f00f5ffb640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f00d8079140 0x7f00d807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:10.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.583+0000 7f010d43b640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f00d8079140 0x7f00d807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:10.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.583+0000 7f010d43b640 10 monclient: get_auth_request con 0x7f00d8079140 auth_method 0 2026-02-17T05:59:10.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.583+0000 7f010d43b640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f00d8079140 0x7f00d807b600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009ba0 tx=0x7f00f8009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:10.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.583+0000 7f00f5ffb640 10 client.4752.objecter ms_handle_connect 0x7f00d8079140 2026-02-17T05:59:10.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.590+0000 7f00f5ffb640 1 -- 10.20.193.16:0/3449147438 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f00fc0971c0 con 0x7f01080c7fe0 2026-02-17T05:59:10.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.590+0000 7f00f5ffb640 10 client.4752.objecter ms_dispatch2 0x7f0108000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:10.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.590+0000 7f00f5ffb640 3 client.4752.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:10.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.590+0000 7f00f5ffb640 3 client.4752.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:10.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.590+0000 7f00f5ffb640 20 client.4752.objecter dump_active .. 0 homeless 2026-02-17T05:59:10.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.590+0000 7f00f5ffb640 1 -- 10.20.193.16:0/3449147438 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f00fc0616e0 con 0x7f01080c7fe0 2026-02-17T05:59:10.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.590+0000 7f00f5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:10.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.590+0000 7f00f5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:10.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.691+0000 7f010f6c6640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:10.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.691+0000 7f010f6c6640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:10.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.691+0000 7f010f6c6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:10.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.691+0000 7f010f6c6640 1 -- 10.20.193.16:0/3449147438 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0108151970 con 0x7f01080c7fe0 2026-02-17T05:59:10.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.694+0000 7f00f5ffb640 1 -- 10.20.193.16:0/3449147438 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f00fc0659e0 con 0x7f01080c7fe0 2026-02-17T05:59:10.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.695+0000 7f00f5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:10.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.695+0000 7f00f5ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:10.696 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:10.696 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:10.697 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 10 librados: watch_flush enter 2026-02-17T05:59:10.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 10 librados: watch_flush exit 2026-02-17T05:59:10.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 20 client.4752.objecter shutdown clearing up homeless session... 2026-02-17T05:59:10.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 10 client.4752.objecter successfully canceled tick 2026-02-17T05:59:10.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 -- 10.20.193.16:0/3449147438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f00d8079140 msgr2=0x7f00d807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:10.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f00d8079140 0x7f00d807b600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009ba0 tx=0x7f00f8009290 comp rx=0 tx=0).stop 2026-02-17T05:59:10.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 10 monclient: shutdown 2026-02-17T05:59:10.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:10.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 -- 10.20.193.16:0/3449147438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01080c7fe0 msgr2=0x7f0108141440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:10.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01080c7fe0 0x7f0108141440 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00c520 tx=0x7f00fc00c9f0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 -- 10.20.193.16:0/3449147438 shutdown_connections 2026-02-17T05:59:10.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f00d8079140 0x7f00d807b600 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f01081536f0 0x7f01081f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f01080c7fe0 0x7f0108141440 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 --2- 10.20.193.16:0/3449147438 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f01080700f0 0x7f0108143390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 -- 10.20.193.16:0/3449147438 >> 10.20.193.16:0/3449147438 conn(0x7f01080cd770 msgr2=0x7f01081506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:10.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 -- 10.20.193.16:0/3449147438 shutdown_connections 2026-02-17T05:59:10.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 -- 10.20.193.16:0/3449147438 wait complete. 2026-02-17T05:59:10.699 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:10.696+0000 7f010f6c6640 1 librados: shutdown 2026-02-17T05:59:13.706 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:13.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.774+0000 7fc402155640 10 monclient: get_monmap_and_config 2026-02-17T05:59:13.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.774+0000 7fc402155640 10 monclient: build_initial_monmap 2026-02-17T05:59:13.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.774+0000 7fc402155640 10 monclient: monmap: 2026-02-17T05:59:13.778 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:13.779 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:13.779 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:13.775186+0000 2026-02-17T05:59:13.779 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:13.775186+0000 2026-02-17T05:59:13.779 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:13.779 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:13.779 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:13.779 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:13.780 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:13.780 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:13.780 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:13.780 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:13.780 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:13.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.774+0000 7fc402155640 1 Processor -- start 2026-02-17T05:59:13.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.774+0000 7fc402155640 1 -- start start 2026-02-17T05:59:13.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.774+0000 7fc402155640 10 monclient: init 2026-02-17T05:59:13.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:13.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 10 monclient: _add_conns ranks=[1,3,5,4,2,0] 2026-02-17T05:59:13.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc3fc155e00 addr v1:10.20.193.128:6789/0 2026-02-17T05:59:13.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fc3fc1561e0 0x7fc3fc0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 10 monclient(hunting): picked mon.noname-a con 0x7fc3fc1561e0 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:13.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fc3fc074dc0 0x7fc3fc0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 10 monclient(hunting): picked mon.noname-c con 0x7fc3fc074dc0 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:13.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3fc0c96d0 con 0x7fc3fc155e00 2026-02-17T05:59:13.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:13.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc3fc074860 con 0x7fc3fc1561e0 2026-02-17T05:59:13.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:13.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fc3fc0c9940 con 0x7fc3fc074dc0 2026-02-17T05:59:13.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:13.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc402155640 10 monclient(hunting): authenticate will time out at 1206.344280s 2026-02-17T05:59:13.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3faffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fc3fc1561e0 0x7fc3fc0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:13.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3fbfff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fc3fc074dc0 0x7fc3fc0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:13.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3fb7fe640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fc3fc155e00 0x7fc3fc074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:53996/0 (socket says 10.20.193.16:53996) 2026-02-17T05:59:13.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3fb7fe640 1 -- 10.20.193.16:0/157208769 learned_addr learned my addr 10.20.193.16:0/157208769 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:13.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3faffd640 10 monclient(hunting): get_auth_request con 0x7fc3fc1561e0 auth_method 0 2026-02-17T05:59:13.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:13.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:13.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:13.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3fbfff640 10 monclient(hunting): get_auth_request con 0x7fc3fc074dc0 auth_method 0 2026-02-17T05:59:13.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:13.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3fbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:13.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.775+0000 7fc3fbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:13.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.776+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:13.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.776+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:13.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.776+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:13.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.776+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:13.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.776+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:13.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.776+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:13.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.777+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_done global_id 4886 payload 274 2026-02-17T05:59:13.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.777+0000 7fc3fbfff640 1 -- 10.20.193.16:0/157208769 >> v2:10.20.193.16:3300/0 conn(0x7fc3fc1561e0 msgr2=0x7fc3fc0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:13.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.777+0000 7fc3fbfff640 1 --2- 10.20.193.16:0/157208769 >> v2:10.20.193.16:3300/0 conn(0x7fc3fc1561e0 0x7fc3fc0ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.777+0000 7fc3fbfff640 1 -- 10.20.193.16:0/157208769 >> v1:10.20.193.128:6789/0 conn(0x7fc3fc155e00 legacy=0x7fc3fc074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:13.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.777+0000 7fc3fbfff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:13.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.777+0000 7fc3fbfff640 1 monclient: found mon.noname-c 2026-02-17T05:59:13.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.777+0000 7fc3fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:13.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.777+0000 7fc3fbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:59:13.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.777+0000 7fc3fbfff640 1 -- 10.20.193.16:0/157208769 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3fc0c9c20 con 0x7fc3fc074dc0 2026-02-17T05:59:13.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.778+0000 7fc3fbfff640 1 --2- 10.20.193.16:0/157208769 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc3fc074dc0 0x7fc3fc0cc1a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc06cbb0 tx=0x7fc3f002f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c004b975ec4811ea server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:13.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 1 -- 10.20.193.16:0/157208769 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f003c070 con 0x7fc3fc074dc0 2026-02-17T05:59:13.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:13.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:13.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 10 monclient: dump: 2026-02-17T05:59:13.794 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:13.794 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:13.794 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:13.794 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 1 -- 10.20.193.16:0/157208769 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f0037440 con 0x7fc3fc074dc0 2026-02-17T05:59:13.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc402155640 5 monclient: authenticate success, global_id 4886 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc402155640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 1 -- 10.20.193.16:0/157208769 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0037810 con 0x7fc3fc074dc0 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc3fa7fc640 10 monclient: dump: 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:13.796 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc402155640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.779+0000 7fc402155640 4 set_mon_vals no callback set 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 10 monclient: shutdown 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 1 -- 10.20.193.16:0/157208769 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc3fc074dc0 msgr2=0x7fc3fc0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 1 --2- 10.20.193.16:0/157208769 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc3fc074dc0 0x7fc3fc0cc1a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc06cbb0 tx=0x7fc3f002f1c0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 1 -- 10.20.193.16:0/157208769 shutdown_connections 2026-02-17T05:59:13.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 1 --2- 10.20.193.16:0/157208769 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc3fc074dc0 0x7fc3fc0cc1a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 1 --2- 10.20.193.16:0/157208769 >> v2:10.20.193.16:3300/0 conn(0x7fc3fc1561e0 0x7fc3fc0ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 1 -- 10.20.193.16:0/157208769 >> 10.20.193.16:0/157208769 conn(0x7fc3fc0ce6c0 msgr2=0x7fc3fc0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 1 -- 10.20.193.16:0/157208769 shutdown_connections 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 1 -- 10.20.193.16:0/157208769 wait complete. 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 10 monclient: build_initial_monmap 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 10 monclient: monmap: 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:13.798 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 1 librados: starting msgr at 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.780+0000 7fc402155640 1 librados: starting objecter 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 1 Processor -- start 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 1 -- start start 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 1 librados: setting wanted keys 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 1 librados: calling monclient init 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient: init 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:13.799 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc3fc074dc0 0x7fc3fc15c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient(hunting): picked mon.noname-b con 0x7fc3fc074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc3fc155e00 0x7fc3fc0c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient(hunting): picked mon.noname-c con 0x7fc3fc155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc3fc1561e0 0x7fc3fc0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient(hunting): picked mon.noname-a con 0x7fc3fc1561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc0c9360 con 0x7fc3fc1561e0 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc0ca1e0 con 0x7fc3fc074dc0 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:13.800 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc0c8c20 con 0x7fc3fc155e00 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc402155640 10 monclient(hunting): authenticate will time out at 1206.350458s 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3fbfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc3fc1561e0 0x7fc3fc0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3faffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc3fc155e00 0x7fc3fc0c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3fb7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc3fc074dc0 0x7fc3fc15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3fbfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc3fc1561e0 0x7fc3fc0c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:38992/0 (socket says 10.20.193.16:38992) 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3fbfff640 1 -- 10.20.193.16:0/1710522760 learned_addr learned my addr 10.20.193.16:0/1710522760 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3fbfff640 10 monclient(hunting): get_auth_request con 0x7fc3fc1561e0 auth_method 0 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3fbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:13.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3fbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3faffd640 10 monclient(hunting): get_auth_request con 0x7fc3fc155e00 auth_method 0 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.781+0000 7fc3faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3fb7fe640 10 monclient(hunting): get_auth_request con 0x7fc3fc074dc0 auth_method 0 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:13.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.782+0000 7fc3fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.783+0000 7fc3faffd640 10 monclient(hunting): handle_auth_done global_id 4889 payload 824 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.783+0000 7fc3faffd640 1 -- 10.20.193.16:0/1710522760 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc3fc074dc0 msgr2=0x7fc3fc15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.783+0000 7fc3faffd640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc3fc074dc0 0x7fc3fc15c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.783+0000 7fc3faffd640 1 -- 10.20.193.16:0/1710522760 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc3fc1561e0 msgr2=0x7fc3fc0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.783+0000 7fc3faffd640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc3fc1561e0 0x7fc3fc0c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.783+0000 7fc3faffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.783+0000 7fc3faffd640 1 monclient: found mon.noname-c 2026-02-17T05:59:13.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.783+0000 7fc3faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.783+0000 7fc3faffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.783+0000 7fc3faffd640 1 -- 10.20.193.16:0/1710522760 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3fc15ca20 con 0x7fc3fc155e00 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.784+0000 7fc3faffd640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc3fc155e00 0x7fc3fc0c4790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00e8a0 tx=0x7fc3ec00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.784+0000 7fc3d7fff640 10 client.?.objecter ms_handle_connect 0x7fc3fc155e00 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.784+0000 7fc3d7fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc3d7fff640 1 -- 10.20.193.16:0/1710522760 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec017070 con 0x7fc3fc155e00 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc3d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc3d7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc3d7fff640 10 monclient: dump: 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:13.804 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc3d7fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc3d7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc3d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc402155640 5 monclient: authenticate success, global_id 4889 2026-02-17T05:59:13.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc402155640 10 monclient: _renew_subs 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc402155640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc402155640 1 -- 10.20.193.16:0/1710522760 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3fc15a630 con 0x7fc3fc155e00 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc402155640 10 client.4889.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc402155640 10 monclient: _renew_subs 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc402155640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc402155640 1 -- 10.20.193.16:0/1710522760 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3fc15abf0 con 0x7fc3fc155e00 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.785+0000 7fc402155640 1 librados: init done 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc3d7fff640 1 -- 10.20.193.16:0/1710522760 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ec005a90 con 0x7fc3fc155e00 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc3d7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc3f9ffb640 4 set_mon_vals no callback set 2026-02-17T05:59:13.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc3d7fff640 1 -- 10.20.193.16:0/1710522760 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec021060 con 0x7fc3fc155e00 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc3d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc3d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc3d7fff640 10 monclient: dump: 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:13.807 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:13.808 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:13.808 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:13.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc402155640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:13.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc402155640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:13.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc402155640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:13.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.786+0000 7fc402155640 1 -- 10.20.193.16:0/1710522760 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3fc050a10 con 0x7fc3fc155e00 2026-02-17T05:59:13.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.797+0000 7fc3d7fff640 1 -- 10.20.193.16:0/1710522760 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fc3ec024060 con 0x7fc3fc155e00 2026-02-17T05:59:13.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.798+0000 7fc3d7fff640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc3cc079170 0x7fc3cc07b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.798+0000 7fc3fb7fe640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc3cc079170 0x7fc3cc07b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:13.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.798+0000 7fc3fb7fe640 10 monclient: get_auth_request con 0x7fc3cc079170 auth_method 0 2026-02-17T05:59:13.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.798+0000 7fc3d7fff640 10 client.4889.objecter ms_handle_connect 0x7fc3cc079170 2026-02-17T05:59:13.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.798+0000 7fc3fb7fe640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc3cc079170 0x7fc3cc07b630 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8002770 tx=0x7fc3e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:13.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.799+0000 7fc3d7fff640 1 -- 10.20.193.16:0/1710522760 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fc3ec097a70 con 0x7fc3fc155e00 2026-02-17T05:59:13.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.799+0000 7fc3d7fff640 10 client.4889.objecter ms_dispatch2 0x7fc3fc000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:13.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.799+0000 7fc3d7fff640 3 client.4889.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:13.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.799+0000 7fc3d7fff640 3 client.4889.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:13.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.799+0000 7fc3d7fff640 20 client.4889.objecter dump_active .. 0 homeless 2026-02-17T05:59:13.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.901+0000 7fc3d7fff640 1 -- 10.20.193.16:0/1710522760 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc3fc050a10 con 0x7fc3fc155e00 2026-02-17T05:59:13.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.901+0000 7fc3d7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:13.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:13.901+0000 7fc3d7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:14.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.001+0000 7fc402155640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:14.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.001+0000 7fc402155640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:14.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.001+0000 7fc402155640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:14.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.001+0000 7fc402155640 1 -- 10.20.193.16:0/1710522760 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc3fc0c4ff0 con 0x7fc3fc155e00 2026-02-17T05:59:14.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.005+0000 7fc3d7fff640 1 -- 10.20.193.16:0/1710522760 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc3ec061f00 con 0x7fc3fc155e00 2026-02-17T05:59:14.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.005+0000 7fc3d7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:14.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.005+0000 7fc3d7fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:14.006 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:14.006 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:14.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 10 librados: watch_flush enter 2026-02-17T05:59:14.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 10 librados: watch_flush exit 2026-02-17T05:59:14.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 20 client.4889.objecter shutdown clearing up homeless session... 2026-02-17T05:59:14.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 10 client.4889.objecter successfully canceled tick 2026-02-17T05:59:14.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 -- 10.20.193.16:0/1710522760 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc3cc079170 msgr2=0x7fc3cc07b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:14.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc3cc079170 0x7fc3cc07b630 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8002770 tx=0x7fc3e8009040 comp rx=0 tx=0).stop 2026-02-17T05:59:14.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 10 monclient: shutdown 2026-02-17T05:59:14.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:14.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 -- 10.20.193.16:0/1710522760 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc3fc155e00 msgr2=0x7fc3fc0c4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:14.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc3fc155e00 0x7fc3fc0c4790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00e8a0 tx=0x7fc3ec00ed70 comp rx=0 tx=0).stop 2026-02-17T05:59:14.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 -- 10.20.193.16:0/1710522760 shutdown_connections 2026-02-17T05:59:14.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fc3cc079170 0x7fc3cc07b630 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:14.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fc3fc1561e0 0x7fc3fc0c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:14.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fc3fc155e00 0x7fc3fc0c4790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:14.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 --2- 10.20.193.16:0/1710522760 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fc3fc074dc0 0x7fc3fc15c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:14.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 -- 10.20.193.16:0/1710522760 >> 10.20.193.16:0/1710522760 conn(0x7fc3fc0ce6c0 msgr2=0x7fc3fc152a10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:14.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 -- 10.20.193.16:0/1710522760 shutdown_connections 2026-02-17T05:59:14.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 -- 10.20.193.16:0/1710522760 wait complete. 2026-02-17T05:59:14.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:14.006+0000 7fc402155640 1 librados: shutdown 2026-02-17T05:59:17.016 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:17.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.083+0000 7f0e7cd22640 10 monclient: get_monmap_and_config 2026-02-17T05:59:17.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.083+0000 7f0e7cd22640 10 monclient: build_initial_monmap 2026-02-17T05:59:17.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.083+0000 7f0e7cd22640 10 monclient: monmap: 2026-02-17T05:59:17.087 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:17.087 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:17.087 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:17.084840+0000 2026-02-17T05:59:17.087 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:17.084840+0000 2026-02-17T05:59:17.087 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:17.087 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:17.087 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:17.087 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:17.088 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:17.088 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:17.088 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:17.088 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:17.088 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:17.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 1 Processor -- start 2026-02-17T05:59:17.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 1 -- start start 2026-02-17T05:59:17.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 10 monclient: init 2026-02-17T05:59:17.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:17.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 10 monclient: _add_conns ranks=[3,1,4,0,2,5] 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f0e780700f0 0x7f0e780704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 10 monclient(hunting): picked mon.noname-a con 0x7f0e780700f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0e780c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f0e781536f0 0x7f0e78153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 10 monclient(hunting): picked mon.noname-b con 0x7f0e781536f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e780c4820 con 0x7f0e780c7fe0 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0e780c4b00 con 0x7f0e780700f0 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f0e780c4de0 con 0x7f0e781536f0 2026-02-17T05:59:17.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.084+0000 7f0e7cd22640 10 monclient(hunting): authenticate will time out at 1209.653780s 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76575640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f0e780700f0 0x7f0e780704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76575640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f0e780700f0 0x7f0e780704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:38996/0 (socket says 10.20.193.16:38996) 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76d76640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f0e781536f0 0x7f0e78153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76575640 1 -- 10.20.193.16:0/2448124312 learned_addr learned my addr 10.20.193.16:0/2448124312 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76575640 10 monclient(hunting): get_auth_request con 0x7f0e780700f0 auth_method 0 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:17.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:17.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:17.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76d76640 10 monclient(hunting): get_auth_request con 0x7f0e781536f0 auth_method 0 2026-02-17T05:59:17.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:17.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:17.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.085+0000 7f0e76d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:17.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:17.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:17.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:17.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76575640 10 monclient(hunting): handle_auth_done global_id 4761 payload 274 2026-02-17T05:59:17.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76575640 1 -- 10.20.193.16:0/2448124312 >> v2:10.20.193.128:3300/0 conn(0x7f0e781536f0 msgr2=0x7f0e78153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:17.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76575640 1 --2- 10.20.193.16:0/2448124312 >> v2:10.20.193.128:3300/0 conn(0x7f0e781536f0 0x7f0e78153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76575640 1 -- 10.20.193.16:0/2448124312 >> v1:10.20.193.128:6789/0 conn(0x7f0e780c7fe0 legacy=0x7f0e780c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:17.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76575640 10 monclient: _finish_hunting 0 2026-02-17T05:59:17.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76575640 1 monclient: found mon.noname-a 2026-02-17T05:59:17.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:17.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:17.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.086+0000 7f0e76575640 1 -- 10.20.193.16:0/2448124312 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e780c50c0 con 0x7f0e780700f0 2026-02-17T05:59:17.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.087+0000 7f0e76575640 1 --2- 10.20.193.16:0/2448124312 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e780700f0 0x7f0e780704d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0e780c9540 tx=0x7f0e6c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac65b993e3663fd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:17.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 1 -- 10.20.193.16:0/2448124312 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c03c070 con 0x7f0e780700f0 2026-02-17T05:59:17.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:17.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:17.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 10 monclient: dump: 2026-02-17T05:59:17.099 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 10 monclient: _finish_auth 0 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 10 monclient: _check_auth_tickets 2026-02-17T05:59:17.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:17.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 1 -- 10.20.193.16:0/2448124312 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6c02fb30 con 0x7f0e780700f0 2026-02-17T05:59:17.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 5 monclient: authenticate success, global_id 4761 2026-02-17T05:59:17.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:17.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 1 -- 10.20.193.16:0/2448124312 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c037440 con 0x7f0e780700f0 2026-02-17T05:59:17.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:17.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 4 set_mon_vals no callback set 2026-02-17T05:59:17.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:17.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:17.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e75573640 10 monclient: dump: 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 10 monclient: shutdown 2026-02-17T05:59:17.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2448124312 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e780700f0 msgr2=0x7f0e780704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 1 --2- 10.20.193.16:0/2448124312 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e780700f0 0x7f0e780704d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0e780c9540 tx=0x7f0e6c02ee60 comp rx=0 tx=0).stop 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2448124312 shutdown_connections 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 1 --2- 10.20.193.16:0/2448124312 >> v2:10.20.193.128:3300/0 conn(0x7f0e781536f0 0x7f0e78153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 1 --2- 10.20.193.16:0/2448124312 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e780700f0 0x7f0e780704d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2448124312 >> 10.20.193.16:0/2448124312 conn(0x7f0e780cd770 msgr2=0x7f0e780cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2448124312 shutdown_connections 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.089+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2448124312 wait complete. 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.090+0000 7f0e7cd22640 10 monclient: build_initial_monmap 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.090+0000 7f0e7cd22640 10 monclient: monmap: 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:17.103 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.090+0000 7f0e7cd22640 1 librados: starting msgr at 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.090+0000 7f0e7cd22640 1 librados: starting objecter 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.090+0000 7f0e7cd22640 1 Processor -- start 2026-02-17T05:59:17.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.090+0000 7f0e7cd22640 1 -- start start 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.090+0000 7f0e7cd22640 1 librados: setting wanted keys 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.090+0000 7f0e7cd22640 1 librados: calling monclient init 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.090+0000 7f0e7cd22640 10 monclient: init 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e780700f0 0x7f0e781f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 10 monclient(hunting): picked mon.noname-c con 0x7f0e780700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e780c7fe0 0x7f0e781513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 10 monclient(hunting): picked mon.noname-b con 0x7f0e780c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e781536f0 0x7f0e780cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 10 monclient(hunting): picked mon.noname-a con 0x7f0e781536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:17.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78154060 con 0x7f0e781536f0 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f0e780c5680 con 0x7f0e780c7fe0 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78074c00 con 0x7f0e780700f0 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e7cd22640 10 monclient(hunting): authenticate will time out at 1209.660055s 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76d76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e781536f0 0x7f0e780cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e75d74640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e780c7fe0 0x7f0e781513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76d76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e781536f0 0x7f0e780cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:39008/0 (socket says 10.20.193.16:39008) 2026-02-17T05:59:17.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76d76640 1 -- 10.20.193.16:0/2117850155 learned_addr learned my addr 10.20.193.16:0/2117850155 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76d76640 10 monclient(hunting): get_auth_request con 0x7f0e781536f0 auth_method 0 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e75d74640 10 monclient(hunting): get_auth_request con 0x7f0e780c7fe0 auth_method 0 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e75d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e75d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e75d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:17.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76575640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e780700f0 0x7f0e781f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e75d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e75d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e75d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76575640 10 monclient(hunting): get_auth_request con 0x7f0e780700f0 auth_method 0 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.091+0000 7f0e76575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.092+0000 7f0e76575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.092+0000 7f0e76575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.092+0000 7f0e76575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.092+0000 7f0e76d76640 10 monclient(hunting): handle_auth_done global_id 4764 payload 824 2026-02-17T05:59:17.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.092+0000 7f0e76d76640 1 -- 10.20.193.16:0/2117850155 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e780700f0 msgr2=0x7f0e781f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.092+0000 7f0e76d76640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e780700f0 0x7f0e781f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.093+0000 7f0e76d76640 1 -- 10.20.193.16:0/2117850155 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e780c7fe0 msgr2=0x7f0e781513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.093+0000 7f0e76d76640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e780c7fe0 0x7f0e781513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.093+0000 7f0e76d76640 10 monclient: _finish_hunting 0 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.093+0000 7f0e76d76640 1 monclient: found mon.noname-a 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.093+0000 7f0e76d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.093+0000 7f0e76d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.093+0000 7f0e76d76640 1 -- 10.20.193.16:0/2117850155 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e781f4660 con 0x7f0e781536f0 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.094+0000 7f0e76d76640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e781536f0 0x7f0e780cb9e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800efc0 tx=0x7f0e6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.094+0000 7f0e5effd640 10 client.?.objecter ms_handle_connect 0x7f0e781536f0 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.094+0000 7f0e5effd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:17.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e5effd640 1 -- 10.20.193.16:0/2117850155 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68018020 con 0x7f0e781536f0 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e5effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e5effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e5effd640 10 monclient: dump: 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:17.110 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e5effd640 10 monclient: _finish_auth 0 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e5effd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e5effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 5 monclient: authenticate success, global_id 4764 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 10 monclient: _renew_subs 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2117850155 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e780cb220 con 0x7f0e781536f0 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 10 client.4764.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 10 monclient: _renew_subs 2026-02-17T05:59:17.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2117850155 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e781f40a0 con 0x7f0e781536f0 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 1 librados: init done 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.095+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2117850155 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e780704d0 con 0x7f0e781536f0 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.096+0000 7f0e5effd640 1 -- 10.20.193.16:0/2117850155 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e68014480 con 0x7f0e781536f0 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.096+0000 7f0e5effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.096+0000 7f0e5effd640 1 -- 10.20.193.16:0/2117850155 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68014830 con 0x7f0e781536f0 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.096+0000 7f0e5effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.096+0000 7f0e5effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:17.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.096+0000 7f0e74d72640 4 set_mon_vals no callback set 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.096+0000 7f0e5effd640 10 monclient: dump: 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:17.113 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:17.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.205+0000 7f0e5effd640 1 -- 10.20.193.16:0/2117850155 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f0e680149d0 con 0x7f0e781536f0 2026-02-17T05:59:17.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.205+0000 7f0e5effd640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0e44079140 0x7f0e4407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.205+0000 7f0e76575640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0e44079140 0x7f0e4407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:17.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.205+0000 7f0e76575640 10 monclient: get_auth_request con 0x7f0e44079140 auth_method 0 2026-02-17T05:59:17.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.205+0000 7f0e76575640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0e44079140 0x7f0e4407b600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e780c5b80 tx=0x7f0e6c039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:17.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.205+0000 7f0e5effd640 10 client.4764.objecter ms_handle_connect 0x7f0e44079140 2026-02-17T05:59:17.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.213+0000 7f0e5effd640 1 -- 10.20.193.16:0/2117850155 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f0e6805f3d0 con 0x7f0e781536f0 2026-02-17T05:59:17.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.213+0000 7f0e5effd640 10 client.4764.objecter ms_dispatch2 0x7f0e78000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:17.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.213+0000 7f0e5effd640 3 client.4764.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:17.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.213+0000 7f0e5effd640 3 client.4764.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:17.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.213+0000 7f0e5effd640 20 client.4764.objecter dump_active .. 0 homeless 2026-02-17T05:59:17.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.213+0000 7f0e5effd640 1 -- 10.20.193.16:0/2117850155 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0e68022070 con 0x7f0e781536f0 2026-02-17T05:59:17.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.213+0000 7f0e5effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:17.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.213+0000 7f0e5effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:17.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.314+0000 7f0e7cd22640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:17.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.314+0000 7f0e7cd22640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:17.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.314+0000 7f0e7cd22640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:17.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.314+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2117850155 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0e781573f0 con 0x7f0e781536f0 2026-02-17T05:59:17.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.316+0000 7f0e5effd640 1 -- 10.20.193.16:0/2117850155 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f0e68010040 con 0x7f0e781536f0 2026-02-17T05:59:17.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.316+0000 7f0e5effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:17.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.316+0000 7f0e5effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:17.317 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:17.318 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:17.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 10 librados: watch_flush enter 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 10 librados: watch_flush exit 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 20 client.4764.objecter shutdown clearing up homeless session... 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 10 client.4764.objecter successfully canceled tick 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2117850155 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0e44079140 msgr2=0x7f0e4407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0e44079140 0x7f0e4407b600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e780c5b80 tx=0x7f0e6c039040 comp rx=0 tx=0).stop 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 10 monclient: shutdown 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2117850155 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e781536f0 msgr2=0x7f0e780cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e781536f0 0x7f0e780cb9e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800efc0 tx=0x7f0e6800c490 comp rx=0 tx=0).stop 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2117850155 shutdown_connections 2026-02-17T05:59:17.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0e44079140 0x7f0e4407b600 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0e780c7fe0 0x7f0e781513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0e781536f0 0x7f0e780cb9e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 --2- 10.20.193.16:0/2117850155 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0e780700f0 0x7f0e781f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2117850155 >> 10.20.193.16:0/2117850155 conn(0x7f0e780cd770 msgr2=0x7f0e78148e20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:17.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2117850155 shutdown_connections 2026-02-17T05:59:17.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 -- 10.20.193.16:0/2117850155 wait complete. 2026-02-17T05:59:17.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:17.318+0000 7f0e7cd22640 1 librados: shutdown 2026-02-17T05:59:20.329 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:20.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.396+0000 7f48f0828640 10 monclient: get_monmap_and_config 2026-02-17T05:59:20.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.396+0000 7f48f0828640 10 monclient: build_initial_monmap 2026-02-17T05:59:20.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.396+0000 7f48f0828640 10 monclient: monmap: 2026-02-17T05:59:20.398 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:20.398 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:20.398 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:20.397979+0000 2026-02-17T05:59:20.398 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:20.397979+0000 2026-02-17T05:59:20.398 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:20.398 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:20.398 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:20.399 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:20.399 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:20.399 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:20.399 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:20.399 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:20.399 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:20.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.397+0000 7f48f0828640 1 Processor -- start 2026-02-17T05:59:20.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.397+0000 7f48f0828640 1 -- start start 2026-02-17T05:59:20.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.397+0000 7f48f0828640 10 monclient: init 2026-02-17T05:59:20.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 10 monclient: _add_conns ranks=[4,5,3,0,1,2] 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f48ec14e7b0 0x7f48ec15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 10 monclient(hunting): picked mon.noname-b con 0x7f48ec14e7b0 addr v2:10.20.193.128:3300/0 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f48ec14eb90 0x7f48ec15a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 10 monclient(hunting): picked mon.noname-c con 0x7f48ec14eb90 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f48ec073e60 0x7f48ec074290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 10 monclient(hunting): picked mon.noname-a con 0x7f48ec073e60 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f48ec159390 con 0x7f48ec073e60 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f48ec0735a0 con 0x7f48ec14e7b0 2026-02-17T05:59:20.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f48ec073880 con 0x7f48ec14eb90 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48f0828640 10 monclient(hunting): authenticate will time out at 1212.967101s 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48eb577640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f48ec073e60 0x7f48ec074290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48ead76640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f48ec14e7b0 0x7f48ec15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48ea575640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f48ec14eb90 0x7f48ec15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48eb577640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f48ec073e60 0x7f48ec074290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:39124/0 (socket says 10.20.193.16:39124) 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48eb577640 1 -- 10.20.193.16:0/3852553308 learned_addr learned my addr 10.20.193.16:0/3852553308 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48eb577640 10 monclient(hunting): get_auth_request con 0x7f48ec073e60 auth_method 0 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48eb577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48eb577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:20.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48eb577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48ead76640 10 monclient(hunting): get_auth_request con 0x7f48ec14e7b0 auth_method 0 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48ead76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48ead76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48ead76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48ea575640 10 monclient(hunting): get_auth_request con 0x7f48ec14eb90 auth_method 0 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48ea575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48ea575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48ea575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48eb577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48eb577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.398+0000 7f48eb577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:20.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ead76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ead76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ead76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 10 monclient(hunting): handle_auth_done global_id 4895 payload 274 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 1 -- 10.20.193.16:0/3852553308 >> v2:10.20.193.128:3300/0 conn(0x7f48ec14e7b0 msgr2=0x7f48ec15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 1 --2- 10.20.193.16:0/3852553308 >> v2:10.20.193.128:3300/0 conn(0x7f48ec14e7b0 0x7f48ec15a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 1 -- 10.20.193.16:0/3852553308 >> v2:10.20.193.16:3300/0 conn(0x7f48ec073e60 msgr2=0x7f48ec074290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 1 --2- 10.20.193.16:0/3852553308 >> v2:10.20.193.16:3300/0 conn(0x7f48ec073e60 0x7f48ec074290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 10 monclient: _finish_hunting 0 2026-02-17T05:59:20.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 1 monclient: found mon.noname-c 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.399+0000 7f48ea575640 1 -- 10.20.193.16:0/3852553308 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec1595e0 con 0x7f48ec14eb90 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.400+0000 7f48ea575640 1 --2- 10.20.193.16:0/3852553308 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f48ec14eb90 0x7f48ec15a600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f48ec076d50 tx=0x7f48d402ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47902506f29e9907 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 1 -- 10.20.193.16:0/3852553308 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d403c070 con 0x7f48ec14eb90 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 10 monclient: dump: 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:20.412 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:20.413 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:20.413 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:20.413 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:20.413 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:20.413 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:20.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 10 monclient: _finish_auth 0 2026-02-17T05:59:20.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 10 monclient: _check_auth_tickets 2026-02-17T05:59:20.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:20.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 1 -- 10.20.193.16:0/3852553308 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d402f9f0 con 0x7f48ec14eb90 2026-02-17T05:59:20.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:20.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 1 -- 10.20.193.16:0/3852553308 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d402fdc0 con 0x7f48ec14eb90 2026-02-17T05:59:20.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48f0828640 5 monclient: authenticate success, global_id 4895 2026-02-17T05:59:20.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48f0828640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:20.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48f0828640 4 set_mon_vals no callback set 2026-02-17T05:59:20.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:20.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:20.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.402+0000 7f48e9d74640 10 monclient: dump: 2026-02-17T05:59:20.415 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:20.415 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:20.415 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:20.415 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:20.415 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:20.416 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:20.416 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:20.416 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:20.416 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:20.416 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:20.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 10 monclient: shutdown 2026-02-17T05:59:20.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:20.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 -- 10.20.193.16:0/3852553308 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f48ec14eb90 msgr2=0x7f48ec15a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:20.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 --2- 10.20.193.16:0/3852553308 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f48ec14eb90 0x7f48ec15a600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f48ec076d50 tx=0x7f48d402ed30 comp rx=0 tx=0).stop 2026-02-17T05:59:20.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 -- 10.20.193.16:0/3852553308 shutdown_connections 2026-02-17T05:59:20.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 --2- 10.20.193.16:0/3852553308 >> v2:10.20.193.16:3300/0 conn(0x7f48ec073e60 0x7f48ec074290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 --2- 10.20.193.16:0/3852553308 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f48ec14eb90 0x7f48ec15a600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 --2- 10.20.193.16:0/3852553308 >> v2:10.20.193.128:3300/0 conn(0x7f48ec14e7b0 0x7f48ec15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 -- 10.20.193.16:0/3852553308 >> 10.20.193.16:0/3852553308 conn(0x7f48ec0c9b60 msgr2=0x7f48ec0c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:20.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 -- 10.20.193.16:0/3852553308 shutdown_connections 2026-02-17T05:59:20.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 -- 10.20.193.16:0/3852553308 wait complete. 2026-02-17T05:59:20.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 10 monclient: build_initial_monmap 2026-02-17T05:59:20.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 10 monclient: monmap: 2026-02-17T05:59:20.419 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:20.419 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:20.419 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:20.419 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:20.420 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:20.420 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:20.420 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:20.420 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:20.420 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:20.420 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:20.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 librados: starting msgr at 2026-02-17T05:59:20.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 librados: starting objecter 2026-02-17T05:59:20.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.403+0000 7f48f0828640 1 Processor -- start 2026-02-17T05:59:20.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 1 -- start start 2026-02-17T05:59:20.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 1 librados: setting wanted keys 2026-02-17T05:59:20.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 1 librados: calling monclient init 2026-02-17T05:59:20.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient: init 2026-02-17T05:59:20.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:20.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:59:20.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f48ec073e60 0x7f48ec0c2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:20.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient(hunting): picked mon.noname-b con 0x7f48ec073e60 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:20.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f48ec14e7b0 0x7f48ec0be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:20.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient(hunting): picked mon.noname-a con 0x7f48ec14e7b0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:20.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f48ec14eb90 0x7f48ec0beeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:20.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient(hunting): picked mon.noname-c con 0x7f48ec14eb90 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:20.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:20.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0747d0 con 0x7f48ec14e7b0 2026-02-17T05:59:20.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:20.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec159ba0 con 0x7f48ec073e60 2026-02-17T05:59:20.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:20.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0c8a70 con 0x7f48ec14eb90 2026-02-17T05:59:20.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:20.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48f0828640 10 monclient(hunting): authenticate will time out at 1212.973411s 2026-02-17T05:59:20.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48ea575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f48ec14e7b0 0x7f48ec0be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:20.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48ead76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f48ec073e60 0x7f48ec0c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:20.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48eb577640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f48ec14eb90 0x7f48ec0beeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:20.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48ea575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f48ec14e7b0 0x7f48ec0be970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:39140/0 (socket says 10.20.193.16:39140) 2026-02-17T05:59:20.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48ea575640 1 -- 10.20.193.16:0/591940618 learned_addr learned my addr 10.20.193.16:0/591940618 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:20.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48ea575640 10 monclient(hunting): get_auth_request con 0x7f48ec14e7b0 auth_method 0 2026-02-17T05:59:20.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48ea575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:20.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48ea575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:20.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48ea575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:20.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48eb577640 10 monclient(hunting): get_auth_request con 0x7f48ec14eb90 auth_method 0 2026-02-17T05:59:20.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48eb577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:20.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48eb577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:20.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48eb577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:20.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.404+0000 7f48ead76640 10 monclient(hunting): get_auth_request con 0x7f48ec073e60 auth_method 0 2026-02-17T05:59:20.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48ead76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:20.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48ead76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:20.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48ead76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:20.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48ea575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:20.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48ea575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48ea575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48eb577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48eb577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48eb577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48ead76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48ead76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.405+0000 7f48ead76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.406+0000 7f48ea575640 10 monclient(hunting): handle_auth_done global_id 4770 payload 824 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.406+0000 7f48ea575640 1 -- 10.20.193.16:0/591940618 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f48ec14eb90 msgr2=0x7f48ec0beeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.406+0000 7f48ea575640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f48ec14eb90 0x7f48ec0beeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.406+0000 7f48ea575640 1 -- 10.20.193.16:0/591940618 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f48ec073e60 msgr2=0x7f48ec0c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:20.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.406+0000 7f48ea575640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f48ec073e60 0x7f48ec0c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.406+0000 7f48ea575640 10 monclient: _finish_hunting 0 2026-02-17T05:59:20.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.406+0000 7f48ea575640 1 monclient: found mon.noname-a 2026-02-17T05:59:20.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.406+0000 7f48ea575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:20.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.406+0000 7f48ea575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:20.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.406+0000 7f48ea575640 1 -- 10.20.193.16:0/591940618 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec1f3050 con 0x7f48ec14e7b0 2026-02-17T05:59:20.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.407+0000 7f48ea575640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f48ec14e7b0 0x7f48ec0be970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48d403a040 tx=0x7f48d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:20.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.407+0000 7f48cb7fe640 10 client.?.objecter ms_handle_connect 0x7f48ec14e7b0 2026-02-17T05:59:20.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.407+0000 7f48cb7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:20.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 1 -- 10.20.193.16:0/591940618 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d403c040 con 0x7f48ec14e7b0 2026-02-17T05:59:20.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:20.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:20.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 10 monclient: dump: 2026-02-17T05:59:20.431 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:20.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 1 -- 10.20.193.16:0/591940618 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d4037ce0 con 0x7f48ec14e7b0 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 5 monclient: authenticate success, global_id 4770 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 1 -- 10.20.193.16:0/591940618 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4042670 con 0x7f48ec14e7b0 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48e9573640 4 set_mon_vals no callback set 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48cb7fe640 10 monclient: dump: 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:20.433 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 10 monclient: _renew_subs 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 1 -- 10.20.193.16:0/591940618 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec1f3330 con 0x7f48ec14e7b0 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 10 client.4770.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:20.434 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 10 monclient: _renew_subs 2026-02-17T05:59:20.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:20.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 1 -- 10.20.193.16:0/591940618 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec1f38f0 con 0x7f48ec14e7b0 2026-02-17T05:59:20.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 1 librados: init done 2026-02-17T05:59:20.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:20.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.409+0000 7f48f0828640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:20.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.410+0000 7f48f0828640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:20.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.410+0000 7f48f0828640 1 -- 10.20.193.16:0/591940618 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec0c8e00 con 0x7f48ec14e7b0 2026-02-17T05:59:20.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.520+0000 7f48cb7fe640 1 -- 10.20.193.16:0/591940618 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f48d4042810 con 0x7f48ec14e7b0 2026-02-17T05:59:20.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.520+0000 7f48cb7fe640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f48b8079190 0x7f48b807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:20.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.520+0000 7f48ead76640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f48b8079190 0x7f48b807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:20.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.520+0000 7f48ead76640 10 monclient: get_auth_request con 0x7f48b8079190 auth_method 0 2026-02-17T05:59:20.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.521+0000 7f48cb7fe640 10 client.4770.objecter ms_handle_connect 0x7f48b8079190 2026-02-17T05:59:20.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.521+0000 7f48ead76640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f48b8079190 0x7f48b807b650 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009bc0 tx=0x7f48e0009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:20.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.528+0000 7f48cb7fe640 1 -- 10.20.193.16:0/591940618 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f48d402f930 con 0x7f48ec14e7b0 2026-02-17T05:59:20.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.528+0000 7f48cb7fe640 10 client.4770.objecter ms_dispatch2 0x7f48ec000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:20.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.528+0000 7f48cb7fe640 3 client.4770.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:20.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.528+0000 7f48cb7fe640 3 client.4770.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:20.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.528+0000 7f48cb7fe640 20 client.4770.objecter dump_active .. 0 homeless 2026-02-17T05:59:20.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.528+0000 7f48cb7fe640 1 -- 10.20.193.16:0/591940618 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f48d40875f0 con 0x7f48ec14e7b0 2026-02-17T05:59:20.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.528+0000 7f48cb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:20.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.528+0000 7f48cb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:20.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.628+0000 7f48f0828640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:20.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.628+0000 7f48f0828640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:20.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.628+0000 7f48f0828640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:20.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.628+0000 7f48f0828640 1 -- 10.20.193.16:0/591940618 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f48ec072fc0 con 0x7f48ec14e7b0 2026-02-17T05:59:20.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.631+0000 7f48cb7fe640 1 -- 10.20.193.16:0/591940618 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f48d408b8f0 con 0x7f48ec14e7b0 2026-02-17T05:59:20.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.631+0000 7f48cb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:20.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.631+0000 7f48cb7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:20.631 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:20.631 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 10 librados: watch_flush enter 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 10 librados: watch_flush exit 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 20 client.4770.objecter shutdown clearing up homeless session... 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 10 client.4770.objecter successfully canceled tick 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 -- 10.20.193.16:0/591940618 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f48b8079190 msgr2=0x7f48b807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f48b8079190 0x7f48b807b650 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009bc0 tx=0x7f48e0009290 comp rx=0 tx=0).stop 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 10 monclient: shutdown 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 -- 10.20.193.16:0/591940618 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f48ec14e7b0 msgr2=0x7f48ec0be970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f48ec14e7b0 0x7f48ec0be970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48d403a040 tx=0x7f48d40047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 -- 10.20.193.16:0/591940618 shutdown_connections 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f48b8079190 0x7f48b807b650 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f48ec14eb90 0x7f48ec0beeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f48ec14e7b0 0x7f48ec0be970 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 --2- 10.20.193.16:0/591940618 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f48ec073e60 0x7f48ec0c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 -- 10.20.193.16:0/591940618 >> 10.20.193.16:0/591940618 conn(0x7f48ec0c9b60 msgr2=0x7f48ec159fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 -- 10.20.193.16:0/591940618 shutdown_connections 2026-02-17T05:59:20.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 -- 10.20.193.16:0/591940618 wait complete. 2026-02-17T05:59:20.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:20.632+0000 7f48f0828640 1 librados: shutdown 2026-02-17T05:59:23.642 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:23.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.710+0000 7fef41d0a640 10 monclient: get_monmap_and_config 2026-02-17T05:59:23.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.710+0000 7fef41d0a640 10 monclient: build_initial_monmap 2026-02-17T05:59:23.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.710+0000 7fef41d0a640 10 monclient: monmap: 2026-02-17T05:59:23.712 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:23.712 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:23.712 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:23.711503+0000 2026-02-17T05:59:23.712 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:23.711503+0000 2026-02-17T05:59:23.712 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:23.712 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:23.712 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:23.712 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:23.712 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:23.713 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:23.713 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:23.713 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:23.713 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:23.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.710+0000 7fef41d0a640 1 Processor -- start 2026-02-17T05:59:23.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 1 -- start start 2026-02-17T05:59:23.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 10 monclient: init 2026-02-17T05:59:23.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:23.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 10 monclient: _add_conns ranks=[1,2,3,5,0,4] 2026-02-17T05:59:23.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fef3c0700f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:59:23.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fef3c0c86f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:59:23.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fef3c153e00 0x7fef3c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:23.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 10 monclient(hunting): picked mon.noname-a con 0x7fef3c153e00 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:23.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef3c0c4820 con 0x7fef3c0700f0 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef3c0c4b00 con 0x7fef3c0c86f0 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fef3c0c4de0 con 0x7fef3c153e00 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef41d0a640 10 monclient(hunting): authenticate will time out at 1216.280585s 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef3bfff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fef3c153e00 0x7fef3c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef3b7fe640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fef3c0700f0 0x7fef3c0c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:42690/0 (socket says 10.20.193.16:42690) 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.711+0000 7fef3b7fe640 1 -- 10.20.193.16:0/952505807 learned_addr learned my addr 10.20.193.16:0/952505807 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.712+0000 7fef3bfff640 10 monclient(hunting): get_auth_request con 0x7fef3c153e00 auth_method 0 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.712+0000 7fef3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:23.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.712+0000 7fef3bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.712+0000 7fef3bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.712+0000 7fef3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.712+0000 7fef3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.712+0000 7fef3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3a7fc640 1 -- 10.20.193.16:0/952505807 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1287387494 0 0) 0x7fef3c0c4820 con 0x7fef3c0700f0 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3a7fc640 10 monclient(hunting): _negotiate 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3a7fc640 10 monclient(hunting): my global_id is 4858 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3a7fc640 1 -- 10.20.193.16:0/952505807 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef20003620 con 0x7fef3c0700f0 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3bfff640 10 monclient(hunting): handle_auth_done global_id 4773 payload 274 2026-02-17T05:59:23.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3bfff640 1 -- 10.20.193.16:0/952505807 >> v1:10.20.193.148:6789/0 conn(0x7fef3c0c86f0 legacy=0x7fef3c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3bfff640 1 -- 10.20.193.16:0/952505807 >> v1:10.20.193.128:6789/0 conn(0x7fef3c0700f0 legacy=0x7fef3c0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3bfff640 1 monclient: found mon.noname-a 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.713+0000 7fef3bfff640 1 -- 10.20.193.16:0/952505807 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3c0c50c0 con 0x7fef3c153e00 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.714+0000 7fef3bfff640 1 --2- 10.20.193.16:0/952505807 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c153e00 0x7fef3c0704d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fef30009d90 tx=0x7fef3002f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f765a9e609579e63 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 1 -- 10.20.193.16:0/952505807 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3003c070 con 0x7fef3c153e00 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 10 monclient: dump: 2026-02-17T05:59:23.724 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:23.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef41d0a640 5 monclient: authenticate success, global_id 4773 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 1 -- 10.20.193.16:0/952505807 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef30037440 con 0x7fef3c153e00 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef41d0a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef41d0a640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 1 -- 10.20.193.16:0/952505807 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef30037810 con 0x7fef3c153e00 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef41d0a640 4 set_mon_vals no callback set 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.715+0000 7fef3a7fc640 10 monclient: dump: 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:23.726 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 10 monclient: shutdown 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 -- 10.20.193.16:0/952505807 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c153e00 msgr2=0x7fef3c0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:23.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 --2- 10.20.193.16:0/952505807 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c153e00 0x7fef3c0704d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fef30009d90 tx=0x7fef3002f1a0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 -- 10.20.193.16:0/952505807 shutdown_connections 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 --2- 10.20.193.16:0/952505807 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c153e00 0x7fef3c0704d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 -- 10.20.193.16:0/952505807 >> 10.20.193.16:0/952505807 conn(0x7fef3c0cd770 msgr2=0x7fef3c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 -- 10.20.193.16:0/952505807 shutdown_connections 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 -- 10.20.193.16:0/952505807 wait complete. 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 10 monclient: build_initial_monmap 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 10 monclient: monmap: 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:23.728 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 librados: starting msgr at 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 librados: starting objecter 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 Processor -- start 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 -- start start 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 librados: setting wanted keys 2026-02-17T05:59:23.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 1 librados: calling monclient init 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.716+0000 7fef41d0a640 10 monclient: init 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fef3c0700f0 0x7fef3c1424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 10 monclient(hunting): picked mon.noname-b con 0x7fef3c0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c0c86f0 0x7fef3c141410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 10 monclient(hunting): picked mon.noname-a con 0x7fef3c0c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fef3c153e00 0x7fef3c13cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 10 monclient(hunting): picked mon.noname-c con 0x7fef3c153e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c074c00 con 0x7fef3c0c86f0 2026-02-17T05:59:23.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c0c5680 con 0x7fef3c0700f0 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c074d80 con 0x7fef3c153e00 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef41d0a640 10 monclient(hunting): authenticate will time out at 1216.286371s 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3affd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c0c86f0 0x7fef3c141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3affd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c0c86f0 0x7fef3c141410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:39160/0 (socket says 10.20.193.16:39160) 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3affd640 1 -- 10.20.193.16:0/3462603989 learned_addr learned my addr 10.20.193.16:0/3462603989 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3affd640 10 monclient(hunting): get_auth_request con 0x7fef3c0c86f0 auth_method 0 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:23.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3bfff640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fef3c153e00 0x7fef3c13cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3b7fe640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fef3c0700f0 0x7fef3c1424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3bfff640 10 monclient(hunting): get_auth_request con 0x7fef3c153e00 auth_method 0 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.717+0000 7fef3bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:23.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.718+0000 7fef3b7fe640 10 monclient(hunting): get_auth_request con 0x7fef3c0700f0 auth_method 0 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.718+0000 7fef3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.718+0000 7fef3b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.718+0000 7fef3b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.718+0000 7fef3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.718+0000 7fef3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.718+0000 7fef3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.718+0000 7fef3b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.718+0000 7fef3b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.718+0000 7fef3b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.719+0000 7fef3affd640 10 monclient(hunting): handle_auth_done global_id 4776 payload 824 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.719+0000 7fef3affd640 1 -- 10.20.193.16:0/3462603989 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fef3c153e00 msgr2=0x7fef3c13cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:23.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.719+0000 7fef3affd640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fef3c153e00 0x7fef3c13cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.719+0000 7fef3affd640 1 -- 10.20.193.16:0/3462603989 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fef3c0700f0 msgr2=0x7fef3c1424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:23.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.719+0000 7fef3affd640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fef3c0700f0 0x7fef3c1424a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.719+0000 7fef3affd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:23.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.719+0000 7fef3affd640 1 monclient: found mon.noname-a 2026-02-17T05:59:23.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.719+0000 7fef3affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:23.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.719+0000 7fef3affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:23.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.719+0000 7fef3affd640 1 -- 10.20.193.16:0/3462603989 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3c0cb2b0 con 0x7fef3c0c86f0 2026-02-17T05:59:23.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.720+0000 7fef3affd640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c0c86f0 0x7fef3c141410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00c700 tx=0x7fef2c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:23.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.720+0000 7fef17fff640 10 client.?.objecter ms_handle_connect 0x7fef3c0c86f0 2026-02-17T05:59:23.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.720+0000 7fef17fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:23.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 1 -- 10.20.193.16:0/3462603989 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c0052d0 con 0x7fef3c0c86f0 2026-02-17T05:59:23.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:23.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:23.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 10 monclient: dump: 2026-02-17T05:59:23.737 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:23.738 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:23.738 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:23.738 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:23.738 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:23.738 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:23.738 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:23.738 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:23.738 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:23.738 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 1 -- 10.20.193.16:0/3462603989 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef2c005470 con 0x7fef3c0c86f0 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 1 -- 10.20.193.16:0/3462603989 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00a860 con 0x7fef3c0c86f0 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef17fff640 10 monclient: dump: 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:23.739 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef39ffb640 4 set_mon_vals no callback set 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef41d0a640 5 monclient: authenticate success, global_id 4776 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef41d0a640 10 monclient: _renew_subs 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef41d0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:23.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef41d0a640 1 -- 10.20.193.16:0/3462603989 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3c0cb590 con 0x7fef3c0c86f0 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef41d0a640 10 client.4776.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef41d0a640 10 monclient: _renew_subs 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef41d0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef41d0a640 1 -- 10.20.193.16:0/3462603989 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef3c0cbb50 con 0x7fef3c0c86f0 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.721+0000 7fef41d0a640 1 librados: init done 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.722+0000 7fef41d0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.722+0000 7fef41d0a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.722+0000 7fef41d0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.722+0000 7fef41d0a640 1 -- 10.20.193.16:0/3462603989 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef3c0704d0 con 0x7fef3c0c86f0 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.733+0000 7fef17fff640 1 -- 10.20.193.16:0/3462603989 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fef2c00aa00 con 0x7fef3c0c86f0 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.734+0000 7fef17fff640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef0c079190 0x7fef0c07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:23.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.734+0000 7fef3b7fe640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef0c079190 0x7fef0c07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:23.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.734+0000 7fef3b7fe640 10 monclient: get_auth_request con 0x7fef0c079190 auth_method 0 2026-02-17T05:59:23.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.734+0000 7fef17fff640 10 client.4776.objecter ms_handle_connect 0x7fef0c079190 2026-02-17T05:59:23.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.734+0000 7fef3b7fe640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef0c079190 0x7fef0c07b650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fef28007b60 tx=0x7fef28007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:23.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.735+0000 7fef17fff640 1 -- 10.20.193.16:0/3462603989 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fef2c00ad20 con 0x7fef3c0c86f0 2026-02-17T05:59:23.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.735+0000 7fef17fff640 10 client.4776.objecter ms_dispatch2 0x7fef3c000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:23.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.735+0000 7fef17fff640 3 client.4776.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:23.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.735+0000 7fef17fff640 3 client.4776.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:23.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.735+0000 7fef17fff640 20 client.4776.objecter dump_active .. 0 homeless 2026-02-17T05:59:23.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.839+0000 7fef17fff640 1 -- 10.20.193.16:0/3462603989 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fef2c062240 con 0x7fef3c0c86f0 2026-02-17T05:59:23.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.839+0000 7fef17fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:23.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.839+0000 7fef17fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:23.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.942+0000 7fef41d0a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:23.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.942+0000 7fef41d0a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:23.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.942+0000 7fef41d0a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:23.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.942+0000 7fef41d0a640 1 -- 10.20.193.16:0/3462603989 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fef3c1573f0 con 0x7fef3c0c86f0 2026-02-17T05:59:23.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.945+0000 7fef17fff640 1 -- 10.20.193.16:0/3462603989 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fef2c066540 con 0x7fef3c0c86f0 2026-02-17T05:59:23.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.945+0000 7fef17fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:23.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.945+0000 7fef17fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:23.946 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:23.946 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:23.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 10 librados: watch_flush enter 2026-02-17T05:59:23.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 10 librados: watch_flush exit 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 20 client.4776.objecter shutdown clearing up homeless session... 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 10 client.4776.objecter successfully canceled tick 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 -- 10.20.193.16:0/3462603989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef0c079190 msgr2=0x7fef0c07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef0c079190 0x7fef0c07b650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fef28007b60 tx=0x7fef28007ab0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 10 monclient: shutdown 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 -- 10.20.193.16:0/3462603989 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c0c86f0 msgr2=0x7fef3c141410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c0c86f0 0x7fef3c141410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00c700 tx=0x7fef2c00cbd0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 -- 10.20.193.16:0/3462603989 shutdown_connections 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fef0c079190 0x7fef0c07b650 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fef3c153e00 0x7fef3c13cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fef3c0c86f0 0x7fef3c141410 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 --2- 10.20.193.16:0/3462603989 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fef3c0700f0 0x7fef3c1424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 -- 10.20.193.16:0/3462603989 >> 10.20.193.16:0/3462603989 conn(0x7fef3c0cd770 msgr2=0x7fef3c14ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:23.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 -- 10.20.193.16:0/3462603989 shutdown_connections 2026-02-17T05:59:23.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 -- 10.20.193.16:0/3462603989 wait complete. 2026-02-17T05:59:23.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:23.946+0000 7fef41d0a640 1 librados: shutdown 2026-02-17T05:59:24.049 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:05:02.876 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:24.051 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:05:02.878 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:24.059 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:05:02.886 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:24.079 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:05:02.868 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:24.079 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:05:02.868 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:24.080 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:05:02.906 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:24.081 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:05:02.907 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:24.088 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:05:02.877 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:24.088 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:05:02.914 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:26.956 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:27.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.025+0000 7f696a1f7640 10 monclient: get_monmap_and_config 2026-02-17T05:59:27.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.025+0000 7f696a1f7640 10 monclient: build_initial_monmap 2026-02-17T05:59:27.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.025+0000 7f696a1f7640 10 monclient: monmap: 2026-02-17T05:59:27.025 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:27.025 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:27.026127+0000 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:27.026127+0000 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:27.026 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:27.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.025+0000 7f696a1f7640 1 Processor -- start 2026-02-17T05:59:27.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.025+0000 7f696a1f7640 1 -- start start 2026-02-17T05:59:27.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.025+0000 7f696a1f7640 10 monclient: init 2026-02-17T05:59:27.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:27.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 10 monclient: _add_conns ranks=[5,3,0,2,1,4] 2026-02-17T05:59:27.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f69640700f0 0x7f69640704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:27.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 10 monclient(hunting): picked mon.noname-c con 0x7f69640700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:27.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f69640c7fe0 0x7f69640c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:27.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 10 monclient(hunting): picked mon.noname-a con 0x7f69640c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:27.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f69641536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:59:27.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69640c8970 con 0x7f69641536f0 2026-02-17T05:59:27.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:27.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f69640c4820 con 0x7f69640c7fe0 2026-02-17T05:59:27.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:27.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f69640c4b00 con 0x7f69640700f0 2026-02-17T05:59:27.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f696a1f7640 10 monclient(hunting): authenticate will time out at 1219.595098s 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f6962ffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f69640c7fe0 0x7f69640c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f69637fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f69640700f0 0x7f69640704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f6962ffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f69640c7fe0 0x7f69640c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:39162/0 (socket says 10.20.193.16:39162) 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f6962ffd640 1 -- 10.20.193.16:0/290944639 learned_addr learned my addr 10.20.193.16:0/290944639 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f6962ffd640 10 monclient(hunting): get_auth_request con 0x7f69640c7fe0 auth_method 0 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f6962ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f6962ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f6962ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f69637fe640 10 monclient(hunting): get_auth_request con 0x7f69640700f0 auth_method 0 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f69637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:27.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f69637fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:27.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f69637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:27.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f6962ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:27.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f6962ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:27.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.026+0000 7f6962ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:27.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f69637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:27.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f69637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:27.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f69637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:27.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f6962ffd640 10 monclient(hunting): handle_auth_done global_id 4779 payload 274 2026-02-17T05:59:27.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f6962ffd640 1 -- 10.20.193.16:0/290944639 >> v2:10.20.193.148:3300/0 conn(0x7f69640700f0 msgr2=0x7f69640704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f6962ffd640 1 --2- 10.20.193.16:0/290944639 >> v2:10.20.193.148:3300/0 conn(0x7f69640700f0 0x7f69640704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f6962ffd640 1 -- 10.20.193.16:0/290944639 >> v1:10.20.193.16:6789/0 conn(0x7f69641536f0 legacy=0x7f6964153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f6962ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f6962ffd640 1 monclient: found mon.noname-a 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f6962ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f6962ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.027+0000 7f6962ffd640 1 -- 10.20.193.16:0/290944639 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69640c4de0 con 0x7f69640c7fe0 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.028+0000 7f6962ffd640 1 --2- 10.20.193.16:0/290944639 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 0x7f69640c8430 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6964070b90 tx=0x7f695402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c4062ce3d53633 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.030+0000 7f69627fc640 1 -- 10.20.193.16:0/290944639 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695403c070 con 0x7f69640c7fe0 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.030+0000 7f69627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:27.037 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 10 monclient: dump: 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:27.038 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 1 -- 10.20.193.16:0/290944639 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f695402fa90 con 0x7f69640c7fe0 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 5 monclient: authenticate success, global_id 4779 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 1 -- 10.20.193.16:0/290944639 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695402fe60 con 0x7f69640c7fe0 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f69627fc640 10 monclient: dump: 2026-02-17T05:59:27.039 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 4 set_mon_vals no callback set 2026-02-17T05:59:27.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 10 monclient: shutdown 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 1 -- 10.20.193.16:0/290944639 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 msgr2=0x7f69640c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 1 --2- 10.20.193.16:0/290944639 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 0x7f69640c8430 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6964070b90 tx=0x7f695402ee60 comp rx=0 tx=0).stop 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 1 -- 10.20.193.16:0/290944639 shutdown_connections 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 1 --2- 10.20.193.16:0/290944639 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 0x7f69640c8430 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 1 --2- 10.20.193.16:0/290944639 >> v2:10.20.193.148:3300/0 conn(0x7f69640700f0 0x7f69640704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 1 -- 10.20.193.16:0/290944639 >> 10.20.193.16:0/290944639 conn(0x7f69640cd770 msgr2=0x7f69640cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 1 -- 10.20.193.16:0/290944639 shutdown_connections 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.031+0000 7f696a1f7640 1 -- 10.20.193.16:0/290944639 wait complete. 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient: build_initial_monmap 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient: monmap: 2026-02-17T05:59:27.041 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 librados: starting msgr at 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 librados: starting objecter 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 Processor -- start 2026-02-17T05:59:27.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 -- start start 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 librados: setting wanted keys 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 librados: calling monclient init 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient: init 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69640700f0 0x7f69641529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient(hunting): picked mon.noname-c con 0x7f69640700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 0x7f6964155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient(hunting): picked mon.noname-a con 0x7f69640c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f69641536f0 0x7f6964155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient(hunting): picked mon.noname-b con 0x7f69641536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:27.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69640c5680 con 0x7f69640c7fe0 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f69640c53a0 con 0x7f69641536f0 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6964074c00 con 0x7f69640700f0 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f696a1f7640 10 monclient(hunting): authenticate will time out at 1219.601752s 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.032+0000 7f6962ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 0x7f6964155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f69637fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69640700f0 0x7f69641529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6962ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 0x7f6964155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:39172/0 (socket says 10.20.193.16:39172) 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6962ffd640 1 -- 10.20.193.16:0/1915694434 learned_addr learned my addr 10.20.193.16:0/1915694434 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:27.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6962ffd640 10 monclient(hunting): get_auth_request con 0x7f69640c7fe0 auth_method 0 2026-02-17T05:59:27.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6962ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:27.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6962ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:27.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6962ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:27.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6963fff640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f69641536f0 0x7f6964155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:27.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6962ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:27.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6962ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:27.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6962ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:27.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f69637fe640 10 monclient(hunting): get_auth_request con 0x7f69640700f0 auth_method 0 2026-02-17T05:59:27.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f69637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:27.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f69637fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:27.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f69637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:27.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6963fff640 10 monclient(hunting): get_auth_request con 0x7f69641536f0 auth_method 0 2026-02-17T05:59:27.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6963fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:27.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6963fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:27.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6963fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f69637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f69637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f69637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6963fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6963fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.033+0000 7f6963fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.034+0000 7f6962ffd640 10 monclient(hunting): handle_auth_done global_id 4782 payload 824 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.034+0000 7f6962ffd640 1 -- 10.20.193.16:0/1915694434 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69640700f0 msgr2=0x7f69641529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.034+0000 7f6962ffd640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69640700f0 0x7f69641529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.034+0000 7f6962ffd640 1 -- 10.20.193.16:0/1915694434 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f69641536f0 msgr2=0x7f6964155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:27.047 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.034+0000 7f6962ffd640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f69641536f0 0x7f6964155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:27.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.034+0000 7f6962ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:27.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.034+0000 7f6962ffd640 1 monclient: found mon.noname-a 2026-02-17T05:59:27.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.034+0000 7f6962ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:27.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.034+0000 7f6962ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:27.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.034+0000 7f6962ffd640 1 -- 10.20.193.16:0/1915694434 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69641f1780 con 0x7f69640c7fe0 2026-02-17T05:59:27.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.035+0000 7f6962ffd640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 0x7f6964155970 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f695402f370 tx=0x7f69540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:27.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.035+0000 7f693ffff640 10 client.?.objecter ms_handle_connect 0x7f69640c7fe0 2026-02-17T05:59:27.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.035+0000 7f693ffff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:27.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 1 -- 10.20.193.16:0/1915694434 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695403c040 con 0x7f69640c7fe0 2026-02-17T05:59:27.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:27.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:27.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 10 monclient: dump: 2026-02-17T05:59:27.051 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:27.051 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:27.051 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 10 monclient: _finish_auth 0 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:27.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 1 -- 10.20.193.16:0/1915694434 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6954042410 con 0x7f69640c7fe0 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 1 -- 10.20.193.16:0/1915694434 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954042800 con 0x7f69640c7fe0 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f693ffff640 10 monclient: dump: 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:27.053 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f6961ffb640 4 set_mon_vals no callback set 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f696a1f7640 5 monclient: authenticate success, global_id 4782 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f696a1f7640 10 monclient: _renew_subs 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f696a1f7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f696a1f7640 1 -- 10.20.193.16:0/1915694434 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69641f1a60 con 0x7f69640c7fe0 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f696a1f7640 10 client.4782.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f696a1f7640 10 monclient: _renew_subs 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f696a1f7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:27.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f696a1f7640 1 -- 10.20.193.16:0/1915694434 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69641f2020 con 0x7f69640c7fe0 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.037+0000 7f696a1f7640 1 librados: init done 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.038+0000 7f696a1f7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.038+0000 7f696a1f7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.038+0000 7f696a1f7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.038+0000 7f696a1f7640 1 -- 10.20.193.16:0/1915694434 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69640704d0 con 0x7f69640c7fe0 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.049+0000 7f693ffff640 1 -- 10.20.193.16:0/1915694434 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f69540429a0 con 0x7f69640c7fe0 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.049+0000 7f693ffff640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6934079170 0x7f693407b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.049+0000 7f69637fe640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6934079170 0x7f693407b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.049+0000 7f69637fe640 10 monclient: get_auth_request con 0x7f6934079170 auth_method 0 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.050+0000 7f69637fe640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6934079170 0x7f693407b630 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6950009ba0 tx=0x7f6950009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:27.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.050+0000 7f693ffff640 10 client.4782.objecter ms_handle_connect 0x7f6934079170 2026-02-17T05:59:27.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.052+0000 7f693ffff640 1 -- 10.20.193.16:0/1915694434 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f6954083970 con 0x7f69640c7fe0 2026-02-17T05:59:27.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.052+0000 7f693ffff640 10 client.4782.objecter ms_dispatch2 0x7f6964000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:27.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.052+0000 7f693ffff640 3 client.4782.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:27.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.052+0000 7f693ffff640 3 client.4782.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:27.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.052+0000 7f693ffff640 20 client.4782.objecter dump_active .. 0 homeless 2026-02-17T05:59:27.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.153+0000 7f693ffff640 1 -- 10.20.193.16:0/1915694434 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6954087930 con 0x7f69640c7fe0 2026-02-17T05:59:27.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.153+0000 7f693ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:27.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.153+0000 7f693ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:27.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.253+0000 7f696a1f7640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:27.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.253+0000 7f696a1f7640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:27.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.253+0000 7f696a1f7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:27.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.253+0000 7f696a1f7640 1 -- 10.20.193.16:0/1915694434 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f69641573f0 con 0x7f69640c7fe0 2026-02-17T05:59:27.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.256+0000 7f693ffff640 1 -- 10.20.193.16:0/1915694434 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f695408bc30 con 0x7f69640c7fe0 2026-02-17T05:59:27.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.256+0000 7f693ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:27.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.256+0000 7f693ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:27.257 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:27.258 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:27.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 10 librados: watch_flush enter 2026-02-17T05:59:27.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 10 librados: watch_flush exit 2026-02-17T05:59:27.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 20 client.4782.objecter shutdown clearing up homeless session... 2026-02-17T05:59:27.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 10 client.4782.objecter successfully canceled tick 2026-02-17T05:59:27.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 -- 10.20.193.16:0/1915694434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6934079170 msgr2=0x7f693407b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6934079170 0x7f693407b630 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6950009ba0 tx=0x7f6950009290 comp rx=0 tx=0).stop 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 10 monclient: shutdown 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 -- 10.20.193.16:0/1915694434 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 msgr2=0x7f6964155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 0x7f6964155970 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f695402f370 tx=0x7f69540047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 -- 10.20.193.16:0/1915694434 shutdown_connections 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6934079170 0x7f693407b630 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f69641536f0 0x7f6964155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f69640c7fe0 0x7f6964155970 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 --2- 10.20.193.16:0/1915694434 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f69640700f0 0x7f69641529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 -- 10.20.193.16:0/1915694434 >> 10.20.193.16:0/1915694434 conn(0x7f69640cd770 msgr2=0x7f696414eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 -- 10.20.193.16:0/1915694434 shutdown_connections 2026-02-17T05:59:27.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 -- 10.20.193.16:0/1915694434 wait complete. 2026-02-17T05:59:27.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:27.257+0000 7f696a1f7640 1 librados: shutdown 2026-02-17T05:59:30.267 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.335+0000 7f34a595e640 10 monclient: get_monmap_and_config 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.335+0000 7f34a595e640 10 monclient: build_initial_monmap 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.335+0000 7f34a595e640 10 monclient: monmap: 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:30.336969+0000 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:30.336969+0000 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:30.336 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 1 Processor -- start 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 1 -- start start 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 10 monclient: init 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 10 monclient: _add_conns ranks=[2,4,3,1,5,0] 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f34a00700f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f34a00c86f0 0x7f34a00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 10 monclient(hunting): picked mon.noname-b con 0x7f34a00c86f0 addr v2:10.20.193.128:3300/0 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f34a01536f0 0x7f34a0153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 10 monclient(hunting): picked mon.noname-a con 0x7f34a01536f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a00c4820 con 0x7f34a00700f0 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f34a00c4b00 con 0x7f34a01536f0 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f34a00c4de0 con 0x7f34a00c86f0 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:30.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.336+0000 7f34a595e640 10 monclient(hunting): authenticate will time out at 1222.905923s 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349f7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f34a01536f0 0x7f34a0153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349e7fc640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f34a00c86f0 0x7f34a00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349effd640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f34a00700f0 0x7f34a00c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:40332/0 (socket says 10.20.193.16:40332) 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349effd640 1 -- 10.20.193.16:0/822492240 learned_addr learned my addr 10.20.193.16:0/822492240 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349f7fe640 10 monclient(hunting): get_auth_request con 0x7f34a01536f0 auth_method 0 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349e7fc640 10 monclient(hunting): get_auth_request con 0x7f34a00c86f0 auth_method 0 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.337+0000 7f349f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349f7fe640 10 monclient(hunting): handle_auth_done global_id 4785 payload 274 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349f7fe640 1 -- 10.20.193.16:0/822492240 >> v2:10.20.193.128:3300/0 conn(0x7f34a00c86f0 msgr2=0x7f34a00704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:30.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349f7fe640 1 --2- 10.20.193.16:0/822492240 >> v2:10.20.193.128:3300/0 conn(0x7f34a00c86f0 0x7f34a00704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:30.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349f7fe640 1 -- 10.20.193.16:0/822492240 >> v1:10.20.193.148:6789/0 conn(0x7f34a00700f0 legacy=0x7f34a00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:30.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:30.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349f7fe640 1 monclient: found mon.noname-a 2026-02-17T05:59:30.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:30.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:30.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.338+0000 7f349f7fe640 1 -- 10.20.193.16:0/822492240 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a00c50c0 con 0x7f34a01536f0 2026-02-17T05:59:30.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.339+0000 7f349f7fe640 1 --2- 10.20.193.16:0/822492240 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a01536f0 0x7f34a0153b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f34a00c5b80 tx=0x7f349002f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be97561888f3e27 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:30.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 1 -- 10.20.193.16:0/822492240 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349003c070 con 0x7f34a01536f0 2026-02-17T05:59:30.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:30.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:30.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 10 monclient: dump: 2026-02-17T05:59:30.341 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:30.341 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:30.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 1 -- 10.20.193.16:0/822492240 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349002fe40 con 0x7f34a01536f0 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 5 monclient: authenticate success, global_id 4785 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 1 -- 10.20.193.16:0/822492240 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490037660 con 0x7f34a01536f0 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 4 set_mon_vals no callback set 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f349dffb640 10 monclient: dump: 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:30.343 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:30.344 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:30.344 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:30.344 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:30.344 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:30.344 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:30.344 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:30.344 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:30.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 10 monclient: shutdown 2026-02-17T05:59:30.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:30.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 1 -- 10.20.193.16:0/822492240 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a01536f0 msgr2=0x7f34a0153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 1 --2- 10.20.193.16:0/822492240 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a01536f0 0x7f34a0153b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f34a00c5b80 tx=0x7f349002f130 comp rx=0 tx=0).stop 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 1 -- 10.20.193.16:0/822492240 shutdown_connections 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 1 --2- 10.20.193.16:0/822492240 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a01536f0 0x7f34a0153b20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 1 --2- 10.20.193.16:0/822492240 >> v2:10.20.193.128:3300/0 conn(0x7f34a00c86f0 0x7f34a00704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 1 -- 10.20.193.16:0/822492240 >> 10.20.193.16:0/822492240 conn(0x7f34a00cd770 msgr2=0x7f34a00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 1 -- 10.20.193.16:0/822492240 shutdown_connections 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.341+0000 7f34a595e640 1 -- 10.20.193.16:0/822492240 wait complete. 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 10 monclient: build_initial_monmap 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 10 monclient: monmap: 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:30.349 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 1 librados: starting msgr at 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 1 librados: starting objecter 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 1 Processor -- start 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 1 -- start start 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 1 librados: setting wanted keys 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 1 librados: calling monclient init 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 10 monclient: init 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.342+0000 7f34a595e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f34a00700f0 0x7f34a01f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 10 monclient(hunting): picked mon.noname-b con 0x7f34a00700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a00c86f0 0x7f34a006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:30.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 10 monclient(hunting): picked mon.noname-a con 0x7f34a00c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34a01536f0 0x7f34a00c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 10 monclient(hunting): picked mon.noname-c con 0x7f34a01536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0154060 con 0x7f34a00c86f0 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f34a00c5680 con 0x7f34a00700f0 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0074c00 con 0x7f34a01536f0 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f34a595e640 10 monclient(hunting): authenticate will time out at 1222.912034s 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349e7fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a00c86f0 0x7f34a006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349f7fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34a01536f0 0x7f34a00c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349e7fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a00c86f0 0x7f34a006ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:37836/0 (socket says 10.20.193.16:37836) 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349effd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f34a00700f0 0x7f34a01f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349e7fc640 1 -- 10.20.193.16:0/3166427409 learned_addr learned my addr 10.20.193.16:0/3166427409 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349e7fc640 10 monclient(hunting): get_auth_request con 0x7f34a00c86f0 auth_method 0 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:30.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349f7fe640 10 monclient(hunting): get_auth_request con 0x7f34a01536f0 auth_method 0 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349effd640 10 monclient(hunting): get_auth_request con 0x7f34a00700f0 auth_method 0 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:30.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.343+0000 7f349effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.344+0000 7f349e7fc640 10 monclient(hunting): handle_auth_done global_id 4788 payload 824 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.344+0000 7f349e7fc640 1 -- 10.20.193.16:0/3166427409 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34a01536f0 msgr2=0x7f34a00c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.344+0000 7f349e7fc640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34a01536f0 0x7f34a00c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.344+0000 7f349e7fc640 1 -- 10.20.193.16:0/3166427409 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f34a00700f0 msgr2=0x7f34a01f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.344+0000 7f349e7fc640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f34a00700f0 0x7f34a01f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.344+0000 7f349e7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.344+0000 7f349e7fc640 1 monclient: found mon.noname-a 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.344+0000 7f349e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.344+0000 7f349e7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.344+0000 7f349e7fc640 1 -- 10.20.193.16:0/3166427409 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a006e210 con 0x7f34a00c86f0 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.345+0000 7f349e7fc640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a00c86f0 0x7f34a006ecb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f348800e8c0 tx=0x7f348800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.345+0000 7f3477fff640 10 client.?.objecter ms_handle_connect 0x7f34a00c86f0 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.345+0000 7f3477fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 1 -- 10.20.193.16:0/3166427409 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488017070 con 0x7f34a00c86f0 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 10 monclient: dump: 2026-02-17T05:59:30.353 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 1 -- 10.20.193.16:0/3166427409 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348800c940 con 0x7f34a00c86f0 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 1 -- 10.20.193.16:0/3166427409 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348800cd30 con 0x7f34a00c86f0 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f349d7fa640 4 set_mon_vals no callback set 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f3477fff640 10 monclient: dump: 2026-02-17T05:59:30.354 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 5 monclient: authenticate success, global_id 4788 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 10 monclient: _renew_subs 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 1 -- 10.20.193.16:0/3166427409 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a01f3d30 con 0x7f34a00c86f0 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 10 client.4788.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 10 monclient: _renew_subs 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 1 -- 10.20.193.16:0/3166427409 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01f42e0 con 0x7f34a00c86f0 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 1 librados: init done 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:30.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:30.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:30.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.347+0000 7f34a595e640 1 -- 10.20.193.16:0/3166427409 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a00704d0 con 0x7f34a00c86f0 2026-02-17T05:59:30.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.416+0000 7f3477fff640 1 -- 10.20.193.16:0/3166427409 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f3488024060 con 0x7f34a00c86f0 2026-02-17T05:59:30.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.416+0000 7f3477fff640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3470079140 0x7f347007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:30.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.416+0000 7f349effd640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3470079140 0x7f347007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:30.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.416+0000 7f349effd640 10 monclient: get_auth_request con 0x7f3470079140 auth_method 0 2026-02-17T05:59:30.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.416+0000 7f3477fff640 10 client.4788.objecter ms_handle_connect 0x7f3470079140 2026-02-17T05:59:30.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.416+0000 7f349effd640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3470079140 0x7f347007b600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3494002770 tx=0x7f3494009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:30.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.417+0000 7f3477fff640 1 -- 10.20.193.16:0/3166427409 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f348805e3d0 con 0x7f34a00c86f0 2026-02-17T05:59:30.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.417+0000 7f3477fff640 10 client.4788.objecter ms_dispatch2 0x7f34a0000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:30.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.417+0000 7f3477fff640 3 client.4788.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:30.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.417+0000 7f3477fff640 3 client.4788.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:30.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.417+0000 7f3477fff640 20 client.4788.objecter dump_active .. 0 homeless 2026-02-17T05:59:30.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.462+0000 7f3477fff640 1 -- 10.20.193.16:0/3166427409 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3488062390 con 0x7f34a00c86f0 2026-02-17T05:59:30.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.462+0000 7f3477fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:30.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.462+0000 7f3477fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:30.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.562+0000 7f34a595e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:30.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.562+0000 7f34a595e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:30.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.562+0000 7f34a595e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:30.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.562+0000 7f34a595e640 1 -- 10.20.193.16:0/3166427409 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f34a0153b00 con 0x7f34a00c86f0 2026-02-17T05:59:30.565 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.565+0000 7f3477fff640 1 -- 10.20.193.16:0/3166427409 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3488066690 con 0x7f34a00c86f0 2026-02-17T05:59:30.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.565+0000 7f3477fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:30.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.565+0000 7f3477fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:30.566 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:30.566 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:30.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 10 librados: watch_flush enter 2026-02-17T05:59:30.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 10 librados: watch_flush exit 2026-02-17T05:59:30.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 20 client.4788.objecter shutdown clearing up homeless session... 2026-02-17T05:59:30.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 10 client.4788.objecter successfully canceled tick 2026-02-17T05:59:30.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 -- 10.20.193.16:0/3166427409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3470079140 msgr2=0x7f347007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:30.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3470079140 0x7f347007b600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3494002770 tx=0x7f3494009040 comp rx=0 tx=0).stop 2026-02-17T05:59:30.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 10 monclient: shutdown 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 -- 10.20.193.16:0/3166427409 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a00c86f0 msgr2=0x7f34a006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a00c86f0 0x7f34a006ecb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f348800e8c0 tx=0x7f348800ed90 comp rx=0 tx=0).stop 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 -- 10.20.193.16:0/3166427409 shutdown_connections 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3470079140 0x7f347007b600 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f34a01536f0 0x7f34a00c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f34a00c86f0 0x7f34a006ecb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 --2- 10.20.193.16:0/3166427409 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f34a00700f0 0x7f34a01f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 -- 10.20.193.16:0/3166427409 >> 10.20.193.16:0/3166427409 conn(0x7f34a00cd770 msgr2=0x7f34a0152bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 -- 10.20.193.16:0/3166427409 shutdown_connections 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 -- 10.20.193.16:0/3166427409 wait complete. 2026-02-17T05:59:30.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:30.566+0000 7f34a595e640 1 librados: shutdown 2026-02-17T05:59:33.576 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:33.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.644+0000 7f5603c43640 10 monclient: get_monmap_and_config 2026-02-17T05:59:33.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.644+0000 7f5603c43640 10 monclient: build_initial_monmap 2026-02-17T05:59:33.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.644+0000 7f5603c43640 10 monclient: monmap: 2026-02-17T05:59:33.646 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:33.646 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:33.646 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:33.645073+0000 2026-02-17T05:59:33.646 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:33.645073+0000 2026-02-17T05:59:33.646 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:33.646 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:33.646 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.644+0000 7f5603c43640 1 Processor -- start 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.644+0000 7f5603c43640 1 -- start start 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.644+0000 7f5603c43640 10 monclient: init 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.644+0000 7f5603c43640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.644+0000 7f5603c43640 10 monclient: _add_conns ranks=[0,3,5,1,2,4] 2026-02-17T05:59:33.647 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f55fc0700f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f55fc0c86f0 0x7f55fc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 10 monclient(hunting): picked mon.noname-a con 0x7f55fc0c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f55fc1536f0 0x7f55fc153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 10 monclient(hunting): picked mon.noname-c con 0x7f55fc1536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55fc0c4820 con 0x7f55fc0700f0 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f55fc0c4b00 con 0x7f55fc0c86f0 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f55fc0c4de0 con 0x7f55fc1536f0 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f5603c43640 10 monclient(hunting): authenticate will time out at 1226.214042s 2026-02-17T05:59:33.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56021b9640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f55fc1536f0 0x7f55fc153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:33.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56011b7640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f55fc0c86f0 0x7f55fc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:33.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56021b9640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f55fc1536f0 0x7f55fc153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:56054/0 (socket says 10.20.193.16:56054) 2026-02-17T05:59:33.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56021b9640 1 -- 10.20.193.16:0/3583450830 learned_addr learned my addr 10.20.193.16:0/3583450830 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:33.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56021b9640 10 monclient(hunting): get_auth_request con 0x7f55fc1536f0 auth_method 0 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56021b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56021b9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56021b9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56011b7640 10 monclient(hunting): get_auth_request con 0x7f55fc0c86f0 auth_method 0 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56011b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56011b7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.645+0000 7f56011b7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.646+0000 7f56021b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.646+0000 7f56021b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.646+0000 7f56021b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.646+0000 7f56011b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.646+0000 7f56011b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:33.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.646+0000 7f56011b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:33.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.647+0000 7f56011b7640 10 monclient(hunting): handle_auth_done global_id 4791 payload 274 2026-02-17T05:59:33.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.647+0000 7f56011b7640 1 -- 10.20.193.16:0/3583450830 >> v2:10.20.193.148:3300/0 conn(0x7f55fc1536f0 msgr2=0x7f55fc153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:33.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.647+0000 7f56011b7640 1 --2- 10.20.193.16:0/3583450830 >> v2:10.20.193.148:3300/0 conn(0x7f55fc1536f0 0x7f55fc153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.647+0000 7f56011b7640 1 -- 10.20.193.16:0/3583450830 >> v1:10.20.193.16:6789/0 conn(0x7f55fc0700f0 legacy=0x7f55fc0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:33.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.647+0000 7f56011b7640 10 monclient: _finish_hunting 0 2026-02-17T05:59:33.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.647+0000 7f56011b7640 1 monclient: found mon.noname-a 2026-02-17T05:59:33.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.647+0000 7f56011b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:33.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.647+0000 7f56011b7640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:33.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.647+0000 7f56011b7640 1 -- 10.20.193.16:0/3583450830 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc0c50c0 con 0x7f55fc0c86f0 2026-02-17T05:59:33.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.648+0000 7f56011b7640 1 --2- 10.20.193.16:0/3583450830 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 0x7f55fc0704d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0c9540 tx=0x7f55f002ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94ba648e5f8cb12 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:33.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 1 -- 10.20.193.16:0/3583450830 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f003c070 con 0x7f55fc0c86f0 2026-02-17T05:59:33.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:33.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:33.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 10 monclient: dump: 2026-02-17T05:59:33.659 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 10 monclient: _finish_auth 0 2026-02-17T05:59:33.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 10 monclient: _check_auth_tickets 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 1 -- 10.20.193.16:0/3583450830 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f002f9c0 con 0x7f55fc0c86f0 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 5 monclient: authenticate success, global_id 4791 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 1 -- 10.20.193.16:0/3583450830 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f002fd90 con 0x7f55fc0c86f0 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f56009b6640 10 monclient: dump: 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:33.661 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 4 set_mon_vals no callback set 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 10 monclient: shutdown 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 1 -- 10.20.193.16:0/3583450830 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 msgr2=0x7f55fc0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:33.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 1 --2- 10.20.193.16:0/3583450830 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 0x7f55fc0704d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0c9540 tx=0x7f55f002ed50 comp rx=0 tx=0).stop 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 1 -- 10.20.193.16:0/3583450830 shutdown_connections 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 1 --2- 10.20.193.16:0/3583450830 >> v2:10.20.193.148:3300/0 conn(0x7f55fc1536f0 0x7f55fc153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 1 --2- 10.20.193.16:0/3583450830 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 0x7f55fc0704d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 1 -- 10.20.193.16:0/3583450830 >> 10.20.193.16:0/3583450830 conn(0x7f55fc0cd770 msgr2=0x7f55fc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 1 -- 10.20.193.16:0/3583450830 shutdown_connections 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.650+0000 7f5603c43640 1 -- 10.20.193.16:0/3583450830 wait complete. 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.651+0000 7f5603c43640 10 monclient: build_initial_monmap 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.651+0000 7f5603c43640 10 monclient: monmap: 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:33.663 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.651+0000 7f5603c43640 1 librados: starting msgr at 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.651+0000 7f5603c43640 1 librados: starting objecter 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.651+0000 7f5603c43640 1 Processor -- start 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.651+0000 7f5603c43640 1 -- start start 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.651+0000 7f5603c43640 1 librados: setting wanted keys 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.651+0000 7f5603c43640 1 librados: calling monclient init 2026-02-17T05:59:33.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.651+0000 7f5603c43640 10 monclient: init 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f55fc0700f0 0x7f55fc1f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 10 monclient(hunting): picked mon.noname-c con 0x7f55fc0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 0x7f55fc13f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 10 monclient(hunting): picked mon.noname-a con 0x7f55fc0c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55fc1536f0 0x7f55fc06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 10 monclient(hunting): picked mon.noname-b con 0x7f55fc1536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc154060 con 0x7f55fc0c86f0 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:33.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc0c5680 con 0x7f55fc1536f0 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc074c00 con 0x7f55fc0700f0 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f5603c43640 10 monclient(hunting): authenticate will time out at 1226.221149s 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56011b7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 0x7f55fc13f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56011b7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 0x7f55fc13f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:37852/0 (socket says 10.20.193.16:37852) 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56011b7640 1 -- 10.20.193.16:0/2224600588 learned_addr learned my addr 10.20.193.16:0/2224600588 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56011b7640 10 monclient(hunting): get_auth_request con 0x7f55fc0c86f0 auth_method 0 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56011b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56011b7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56011b7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:33.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56019b8640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f55fc0700f0 0x7f55fc1f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56011b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56011b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56011b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56019b8640 10 monclient(hunting): get_auth_request con 0x7f55fc0700f0 auth_method 0 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56019b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56019b8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56019b8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56021b9640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55fc1536f0 0x7f55fc06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56021b9640 10 monclient(hunting): get_auth_request con 0x7f55fc1536f0 auth_method 0 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56021b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56021b9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:33.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.652+0000 7f56021b9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.653+0000 7f56019b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.653+0000 7f56019b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.653+0000 7f56019b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.653+0000 7f56021b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.653+0000 7f56021b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.653+0000 7f56021b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.654+0000 7f56011b7640 10 monclient(hunting): handle_auth_done global_id 4794 payload 824 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.654+0000 7f56011b7640 1 -- 10.20.193.16:0/2224600588 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f55fc0700f0 msgr2=0x7f55fc1f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.654+0000 7f56011b7640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f55fc0700f0 0x7f55fc1f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.654+0000 7f56011b7640 1 -- 10.20.193.16:0/2224600588 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55fc1536f0 msgr2=0x7f55fc06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.654+0000 7f56011b7640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55fc1536f0 0x7f55fc06d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.654+0000 7f56011b7640 10 monclient: _finish_hunting 0 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.654+0000 7f56011b7640 1 monclient: found mon.noname-a 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.654+0000 7f56011b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.654+0000 7f56011b7640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.654+0000 7f56011b7640 1 -- 10.20.193.16:0/2224600588 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc1f7340 con 0x7f55fc0c86f0 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.655+0000 7f56011b7640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 0x7f55fc13f6f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f55f003a040 tx=0x7f55f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.655+0000 7f55ea7fc640 10 client.?.objecter ms_handle_connect 0x7f55fc0c86f0 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.655+0000 7f55ea7fc640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 1 -- 10.20.193.16:0/2224600588 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f003c040 con 0x7f55fc0c86f0 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 10 monclient: dump: 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:33.669 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:33.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 1 -- 10.20.193.16:0/2224600588 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f0037d90 con 0x7f55fc0c86f0 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f5603c43640 5 monclient: authenticate success, global_id 4794 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f5603c43640 10 monclient: _renew_subs 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f5603c43640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f5603c43640 1 -- 10.20.193.16:0/2224600588 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55fc1f7620 con 0x7f55fc0c86f0 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 1 -- 10.20.193.16:0/2224600588 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0042670 con 0x7f55fc0c86f0 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ea7fc640 10 monclient: dump: 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:33.671 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f55ebfff640 4 set_mon_vals no callback set 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f5603c43640 10 client.4794.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f5603c43640 10 monclient: _renew_subs 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f5603c43640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f5603c43640 1 -- 10.20.193.16:0/2224600588 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55fc1f7be0 con 0x7f55fc0c86f0 2026-02-17T05:59:33.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.656+0000 7f5603c43640 1 librados: init done 2026-02-17T05:59:33.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.657+0000 7f5603c43640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:33.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.657+0000 7f5603c43640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:33.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.657+0000 7f5603c43640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:33.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.657+0000 7f5603c43640 1 -- 10.20.193.16:0/2224600588 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55fc0704d0 con 0x7f55fc0c86f0 2026-02-17T05:59:33.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.669+0000 7f55ea7fc640 1 -- 10.20.193.16:0/2224600588 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f55f0042810 con 0x7f55fc0c86f0 2026-02-17T05:59:33.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.669+0000 7f55ea7fc640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55cc079190 0x7f55cc07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:33.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.669+0000 7f56019b8640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55cc079190 0x7f55cc07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:33.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.669+0000 7f56019b8640 10 monclient: get_auth_request con 0x7f55cc079190 auth_method 0 2026-02-17T05:59:33.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.669+0000 7f55ea7fc640 10 client.4794.objecter ms_handle_connect 0x7f55cc079190 2026-02-17T05:59:33.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.669+0000 7f56019b8640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55cc079190 0x7f55cc07b650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0046e0 tx=0x7f55ec0091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:33.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.774+0000 7f55ea7fc640 1 -- 10.20.193.16:0/2224600588 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f55f002f880 con 0x7f55fc0c86f0 2026-02-17T05:59:33.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.774+0000 7f55ea7fc640 10 client.4794.objecter ms_dispatch2 0x7f55fc000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:33.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.774+0000 7f55ea7fc640 3 client.4794.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:33.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.774+0000 7f55ea7fc640 3 client.4794.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:33.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.774+0000 7f55ea7fc640 20 client.4794.objecter dump_active .. 0 homeless 2026-02-17T05:59:33.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.774+0000 7f55ea7fc640 1 -- 10.20.193.16:0/2224600588 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f55f0087710 con 0x7f55fc0c86f0 2026-02-17T05:59:33.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.774+0000 7f55ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:33.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.774+0000 7f55ea7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:33.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.873+0000 7f5603c43640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:33.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.873+0000 7f5603c43640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:33.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.873+0000 7f5603c43640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:33.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.873+0000 7f5603c43640 1 -- 10.20.193.16:0/2224600588 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f55fc1573f0 con 0x7f55fc0c86f0 2026-02-17T05:59:33.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.879+0000 7f55ea7fc640 1 -- 10.20.193.16:0/2224600588 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f55f008ba10 con 0x7f55fc0c86f0 2026-02-17T05:59:33.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.879+0000 7f55ea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:33.879 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.879+0000 7f55ea7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:33.880 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:33.880 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:33.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 10 librados: watch_flush enter 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 10 librados: watch_flush exit 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 20 client.4794.objecter shutdown clearing up homeless session... 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 10 client.4794.objecter successfully canceled tick 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 -- 10.20.193.16:0/2224600588 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55cc079190 msgr2=0x7f55cc07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55cc079190 0x7f55cc07b650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0046e0 tx=0x7f55ec0091c0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 10 monclient: shutdown 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 -- 10.20.193.16:0/2224600588 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 msgr2=0x7f55fc13f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 0x7f55fc13f6f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f55f003a040 tx=0x7f55f00047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 -- 10.20.193.16:0/2224600588 shutdown_connections 2026-02-17T05:59:33.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f55cc079190 0x7f55cc07b650 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f55fc1536f0 0x7f55fc06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f55fc0c86f0 0x7f55fc13f6f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 --2- 10.20.193.16:0/2224600588 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f55fc0700f0 0x7f55fc1f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:33.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 -- 10.20.193.16:0/2224600588 >> 10.20.193.16:0/2224600588 conn(0x7f55fc0cd770 msgr2=0x7f55fc14dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:33.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 -- 10.20.193.16:0/2224600588 shutdown_connections 2026-02-17T05:59:33.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 -- 10.20.193.16:0/2224600588 wait complete. 2026-02-17T05:59:33.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:33.880+0000 7f5603c43640 1 librados: shutdown 2026-02-17T05:59:36.889 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:36.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.955+0000 7f5c6b5d3640 10 monclient: get_monmap_and_config 2026-02-17T05:59:36.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.955+0000 7f5c6b5d3640 10 monclient: build_initial_monmap 2026-02-17T05:59:36.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.955+0000 7f5c6b5d3640 10 monclient: monmap: 2026-02-17T05:59:36.960 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:36.961 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:36.961 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:36.956935+0000 2026-02-17T05:59:36.961 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:36.956935+0000 2026-02-17T05:59:36.961 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:36.961 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:36.961 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:36.961 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:36.962 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:36.962 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:36.962 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:36.962 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:36.962 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:36.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 1 Processor -- start 2026-02-17T05:59:36.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 1 -- start start 2026-02-17T05:59:36.962 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 10 monclient: init 2026-02-17T05:59:36.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:36.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 10 monclient: _add_conns ranks=[3,4,0,2,1,5] 2026-02-17T05:59:36.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f5c640700f0 0x7f5c640704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:36.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 10 monclient(hunting): picked mon.noname-a con 0x7f5c640700f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:36.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f5c640c7fe0 0x7f5c640c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:36.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 10 monclient(hunting): picked mon.noname-b con 0x7f5c640c7fe0 addr v2:10.20.193.128:3300/0 2026-02-17T05:59:36.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5c641536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:59:36.963 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c640c8970 con 0x7f5c641536f0 2026-02-17T05:59:36.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:36.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5c640c4820 con 0x7f5c640700f0 2026-02-17T05:59:36.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:36.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f5c640c4b00 con 0x7f5c640c7fe0 2026-02-17T05:59:36.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:36.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.956+0000 7f5c6b5d3640 10 monclient(hunting): authenticate will time out at 1229.525908s 2026-02-17T05:59:36.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c68b47640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f5c640c7fe0 0x7f5c640c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:36.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c69348640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f5c640700f0 0x7f5c640704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:36.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c69348640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f5c640700f0 0x7f5c640704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:37856/0 (socket says 10.20.193.16:37856) 2026-02-17T05:59:36.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c69348640 1 -- 10.20.193.16:0/1240112823 learned_addr learned my addr 10.20.193.16:0/1240112823 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:36.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c69348640 10 monclient(hunting): get_auth_request con 0x7f5c640700f0 auth_method 0 2026-02-17T05:59:36.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c69348640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:36.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c69348640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:36.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c69348640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:36.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c68b47640 10 monclient(hunting): get_auth_request con 0x7f5c640c7fe0 auth_method 0 2026-02-17T05:59:36.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c68b47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:36.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c68b47640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:36.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c68b47640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:36.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c69348640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:36.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c69348640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:36.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c69348640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:36.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c68b47640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:36.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.957+0000 7f5c68b47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:36.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.958+0000 7f5c68b47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:36.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.958+0000 7f5c68b47640 10 monclient(hunting): handle_auth_done global_id 4876 payload 274 2026-02-17T05:59:36.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.958+0000 7f5c68b47640 1 -- 10.20.193.16:0/1240112823 >> v2:10.20.193.16:3300/0 conn(0x7f5c640700f0 msgr2=0x7f5c640704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:36.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.958+0000 7f5c68b47640 1 --2- 10.20.193.16:0/1240112823 >> v2:10.20.193.16:3300/0 conn(0x7f5c640700f0 0x7f5c640704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:36.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.958+0000 7f5c68b47640 1 -- 10.20.193.16:0/1240112823 >> v1:10.20.193.16:6789/0 conn(0x7f5c641536f0 legacy=0x7f5c64153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:36.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.959+0000 7f5c68b47640 10 monclient: _finish_hunting 0 2026-02-17T05:59:36.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.959+0000 7f5c68b47640 1 monclient: found mon.noname-b 2026-02-17T05:59:36.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.959+0000 7f5c68b47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:36.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.959+0000 7f5c68b47640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:59:36.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.959+0000 7f5c68b47640 1 -- 10.20.193.16:0/1240112823 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c640c4de0 con 0x7f5c640c7fe0 2026-02-17T05:59:36.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.959+0000 7f5c68b47640 1 --2- 10.20.193.16:0/1240112823 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5c640c7fe0 0x7f5c640c8430 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c640c5b80 tx=0x7f5c5802ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80ddbdd0b8712fd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:36.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 1 -- 10.20.193.16:0/1240112823 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5803c070 con 0x7f5c640c7fe0 2026-02-17T05:59:36.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:36.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:36.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 10 monclient: dump: 2026-02-17T05:59:36.977 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:36.977 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:36.977 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:36.977 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:36.977 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:36.977 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 1 -- 10.20.193.16:0/1240112823 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c5802f9c0 con 0x7f5c640c7fe0 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c6b5d3640 5 monclient: authenticate success, global_id 4876 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c6b5d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:36.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 1 -- 10.20.193.16:0/1240112823 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5802fd90 con 0x7f5c640c7fe0 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c53fff640 10 monclient: dump: 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:36.979 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c6b5d3640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.961+0000 7f5c6b5d3640 4 set_mon_vals no callback set 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 10 monclient: shutdown 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/1240112823 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5c640c7fe0 msgr2=0x7f5c640c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 1 --2- 10.20.193.16:0/1240112823 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5c640c7fe0 0x7f5c640c8430 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c640c5b80 tx=0x7f5c5802ed50 comp rx=0 tx=0).stop 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/1240112823 shutdown_connections 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 1 --2- 10.20.193.16:0/1240112823 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5c640c7fe0 0x7f5c640c8430 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 1 --2- 10.20.193.16:0/1240112823 >> v2:10.20.193.16:3300/0 conn(0x7f5c640700f0 0x7f5c640704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:36.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/1240112823 >> 10.20.193.16:0/1240112823 conn(0x7f5c640cd770 msgr2=0x7f5c640cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/1240112823 shutdown_connections 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/1240112823 wait complete. 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 10 monclient: build_initial_monmap 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 10 monclient: monmap: 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:36.981 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 1 librados: starting msgr at 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.962+0000 7f5c6b5d3640 1 librados: starting objecter 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 1 Processor -- start 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 1 -- start start 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 1 librados: setting wanted keys 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 1 librados: calling monclient init 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient: init 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5c640700f0 0x7f5c6406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:36.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient(hunting): picked mon.noname-b con 0x7f5c640700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5c640c7fe0 0x7f5c641f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient(hunting): picked mon.noname-a con 0x7f5c640c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5c641536f0 0x7f5c64145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient(hunting): picked mon.noname-c con 0x7f5c641536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c640c5680 con 0x7f5c640c7fe0 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f5c640c53a0 con 0x7f5c640700f0 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64074c00 con 0x7f5c641536f0 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:36.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c6b5d3640 10 monclient(hunting): authenticate will time out at 1229.532483s 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c68b47640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5c640c7fe0 0x7f5c641f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c69348640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5c640700f0 0x7f5c6406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c68b47640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5c640c7fe0 0x7f5c641f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:37860/0 (socket says 10.20.193.16:37860) 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c68b47640 1 -- 10.20.193.16:0/3138427128 learned_addr learned my addr 10.20.193.16:0/3138427128 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c68b47640 10 monclient(hunting): get_auth_request con 0x7f5c640c7fe0 auth_method 0 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c68b47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c68b47640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c68b47640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c69b49640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5c641536f0 0x7f5c64145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c69348640 10 monclient(hunting): get_auth_request con 0x7f5c640700f0 auth_method 0 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c69348640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:36.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c69348640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.963+0000 7f5c69348640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c68b47640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c68b47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c68b47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c69b49640 10 monclient(hunting): get_auth_request con 0x7f5c641536f0 auth_method 0 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c69b49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c69b49640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c69b49640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c69348640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c69348640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c69348640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c69b49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:36.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c69b49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.964+0000 7f5c69b49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.965+0000 7f5c69348640 10 monclient(hunting): handle_auth_done global_id 4879 payload 824 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.965+0000 7f5c69348640 1 -- 10.20.193.16:0/3138427128 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5c641536f0 msgr2=0x7f5c64145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.965+0000 7f5c69348640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5c641536f0 0x7f5c64145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.965+0000 7f5c69348640 1 -- 10.20.193.16:0/3138427128 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5c640c7fe0 msgr2=0x7f5c641f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.965+0000 7f5c69348640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5c640c7fe0 0x7f5c641f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.965+0000 7f5c69348640 10 monclient: _finish_hunting 0 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.965+0000 7f5c69348640 1 monclient: found mon.noname-b 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.965+0000 7f5c69348640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.965+0000 7f5c69348640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.965+0000 7f5c69348640 1 -- 10.20.193.16:0/3138427128 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c641f7e20 con 0x7f5c640700f0 2026-02-17T05:59:36.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.966+0000 7f5c69348640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5c640700f0 0x7f5c6406d270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5c54006dc0 tx=0x7f5c5400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:36.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.966+0000 7f5c51ffb640 10 client.?.objecter ms_handle_connect 0x7f5c640700f0 2026-02-17T05:59:36.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.966+0000 7f5c51ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:36.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c51ffb640 1 -- 10.20.193.16:0/3138427128 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54017070 con 0x7f5c640700f0 2026-02-17T05:59:36.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c51ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:36.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c51ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:36.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c51ffb640 10 monclient: dump: 2026-02-17T05:59:36.987 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:36.987 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c51ffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c51ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 5 monclient: authenticate success, global_id 4879 2026-02-17T05:59:36.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 10 monclient: _renew_subs 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/3138427128 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c641f3b50 con 0x7f5c640700f0 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 10 client.4879.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 10 monclient: _renew_subs 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/3138427128 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c641f7b40 con 0x7f5c640700f0 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 1 librados: init done 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.967+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/3138427128 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c640704d0 con 0x7f5c640700f0 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.968+0000 7f5c51ffb640 1 -- 10.20.193.16:0/3138427128 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c54005a90 con 0x7f5c640700f0 2026-02-17T05:59:36.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.968+0000 7f5c51ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.968+0000 7f5c51ffb640 1 -- 10.20.193.16:0/3138427128 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54021060 con 0x7f5c640700f0 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.968+0000 7f5c51ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.968+0000 7f5c51ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.968+0000 7f5c51ffb640 10 monclient: dump: 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:36.990 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:36.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:36.968+0000 7f5c537fe640 4 set_mon_vals no callback set 2026-02-17T05:59:37.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.078+0000 7f5c51ffb640 1 -- 10.20.193.16:0/3138427128 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f5c5400c910 con 0x7f5c640700f0 2026-02-17T05:59:37.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.078+0000 7f5c51ffb640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5c34079140 0x7f5c3407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:37.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.078+0000 7f5c68b47640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5c34079140 0x7f5c3407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:37.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.078+0000 7f5c68b47640 10 monclient: get_auth_request con 0x7f5c34079140 auth_method 0 2026-02-17T05:59:37.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.078+0000 7f5c68b47640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5c34079140 0x7f5c3407b600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5c5802ed90 tx=0x7f5c58033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:37.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.078+0000 7f5c51ffb640 10 client.4879.objecter ms_handle_connect 0x7f5c34079140 2026-02-17T05:59:37.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.081+0000 7f5c51ffb640 1 -- 10.20.193.16:0/3138427128 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f5c5405e5d0 con 0x7f5c640700f0 2026-02-17T05:59:37.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.081+0000 7f5c51ffb640 10 client.4879.objecter ms_dispatch2 0x7f5c64000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:37.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.081+0000 7f5c51ffb640 3 client.4879.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:37.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.081+0000 7f5c51ffb640 3 client.4879.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:37.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.081+0000 7f5c51ffb640 20 client.4879.objecter dump_active .. 0 homeless 2026-02-17T05:59:37.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.088+0000 7f5c51ffb640 1 -- 10.20.193.16:0/3138427128 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5c54062590 con 0x7f5c640700f0 2026-02-17T05:59:37.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.088+0000 7f5c51ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:37.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.088+0000 7f5c51ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:37.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.187+0000 7f5c6b5d3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:37.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.187+0000 7f5c6b5d3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:37.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.187+0000 7f5c6b5d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T05:59:37.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.187+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/3138427128 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5c641573f0 con 0x7f5c640700f0 2026-02-17T05:59:37.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.190+0000 7f5c51ffb640 1 -- 10.20.193.16:0/3138427128 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f5c54066890 con 0x7f5c640700f0 2026-02-17T05:59:37.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.190+0000 7f5c51ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:37.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.190+0000 7f5c51ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:37.191 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:37.191 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:37.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 10 librados: watch_flush enter 2026-02-17T05:59:37.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 10 librados: watch_flush exit 2026-02-17T05:59:37.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 20 client.4879.objecter shutdown clearing up homeless session... 2026-02-17T05:59:37.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 10 client.4879.objecter successfully canceled tick 2026-02-17T05:59:37.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/3138427128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5c34079140 msgr2=0x7f5c3407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:37.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5c34079140 0x7f5c3407b600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5c5802ed90 tx=0x7f5c58033000 comp rx=0 tx=0).stop 2026-02-17T05:59:37.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 10 monclient: shutdown 2026-02-17T05:59:37.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:37.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/3138427128 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5c640700f0 msgr2=0x7f5c6406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:37.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5c640700f0 0x7f5c6406d270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5c54006dc0 tx=0x7f5c5400ed70 comp rx=0 tx=0).stop 2026-02-17T05:59:37.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/3138427128 shutdown_connections 2026-02-17T05:59:37.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5c34079140 0x7f5c3407b600 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:37.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5c641536f0 0x7f5c64145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:37.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5c640c7fe0 0x7f5c641f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:37.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 --2- 10.20.193.16:0/3138427128 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5c640700f0 0x7f5c6406d270 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:37.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/3138427128 >> 10.20.193.16:0/3138427128 conn(0x7f5c640cd770 msgr2=0x7f5c6414ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:37.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/3138427128 shutdown_connections 2026-02-17T05:59:37.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 -- 10.20.193.16:0/3138427128 wait complete. 2026-02-17T05:59:37.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:37.191+0000 7f5c6b5d3640 1 librados: shutdown 2026-02-17T05:59:40.201 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:40.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.269+0000 7fd101ef2640 10 monclient: get_monmap_and_config 2026-02-17T05:59:40.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.269+0000 7fd101ef2640 10 monclient: build_initial_monmap 2026-02-17T05:59:40.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.269+0000 7fd101ef2640 10 monclient: monmap: 2026-02-17T05:59:40.271 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:40.271 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:40.271 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:40.270515+0000 2026-02-17T05:59:40.271 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:40.270515+0000 2026-02-17T05:59:40.271 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:40.271 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:40.271 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:40.271 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:40.271 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:40.272 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:40.272 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:40.272 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:40.272 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:40.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.269+0000 7fd101ef2640 1 Processor -- start 2026-02-17T05:59:40.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 1 -- start start 2026-02-17T05:59:40.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 10 monclient: init 2026-02-17T05:59:40.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:40.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 10 monclient: _add_conns ranks=[5,2,0,3,1,4] 2026-02-17T05:59:40.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd0fc0700f0 0x7fd0fc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:40.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 10 monclient(hunting): picked mon.noname-c con 0x7fd0fc0700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:40.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd0fc0c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T05:59:40.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd0fc1536f0 addr v1:10.20.193.16:6789/0 2026-02-17T05:59:40.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0fc0c4820 con 0x7fd0fc1536f0 2026-02-17T05:59:40.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0fc0c4b00 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd0fc0c4de0 con 0x7fd0fc0700f0 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd101ef2640 10 monclient(hunting): authenticate will time out at 1232.839575s 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd0fb7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd0fc0700f0 0x7fd0fc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd0fbfff640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fd0fc1536f0 0x7fd0fc153b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:35014/0 (socket says 10.20.193.16:35014) 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.270+0000 7fd0fbfff640 1 -- 10.20.193.16:0/1298453140 learned_addr learned my addr 10.20.193.16:0/1298453140 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.271+0000 7fd0fb7fe640 10 monclient(hunting): get_auth_request con 0x7fd0fc0700f0 auth_method 0 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.271+0000 7fd0fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.271+0000 7fd0fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.271+0000 7fd0fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.271+0000 7fd0fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.271+0000 7fd0fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:40.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.271+0000 7fd0fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fa7fc640 1 -- 10.20.193.16:0/1298453140 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021597748 0 0) 0x7fd0fc0c4820 con 0x7fd0fc1536f0 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fa7fc640 10 monclient(hunting): _negotiate 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fa7fc640 10 monclient(hunting): my global_id is 4803 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fa7fc640 1 -- 10.20.193.16:0/1298453140 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0e0003620 con 0x7fd0fc1536f0 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fb7fe640 10 monclient(hunting): handle_auth_done global_id 4928 payload 274 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fb7fe640 1 -- 10.20.193.16:0/1298453140 >> v1:10.20.193.148:6789/0 conn(0x7fd0fc0c7fe0 legacy=0x7fd0fc0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fb7fe640 1 -- 10.20.193.16:0/1298453140 >> v1:10.20.193.16:6789/0 conn(0x7fd0fc1536f0 legacy=0x7fd0fc153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fb7fe640 1 monclient: found mon.noname-c 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:59:40.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.272+0000 7fd0fb7fe640 1 -- 10.20.193.16:0/1298453140 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc0c50c0 con 0x7fd0fc0700f0 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.275+0000 7fd0fb7fe640 1 --2- 10.20.193.16:0/1298453140 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd0fc0700f0 0x7fd0fc0704d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80098a0 tx=0x7fd0e802edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a1bf0207808786 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 1 -- 10.20.193.16:0/1298453140 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e803c070 con 0x7fd0fc0700f0 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 10 monclient: dump: 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 1 -- 10.20.193.16:0/1298453140 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e802fae0 con 0x7fd0fc0700f0 2026-02-17T05:59:40.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 5 monclient: authenticate success, global_id 4928 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 1 -- 10.20.193.16:0/1298453140 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8037440 con 0x7fd0fc0700f0 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd0fa7fc640 10 monclient: dump: 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 4 set_mon_vals no callback set 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 10 monclient: shutdown 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 1 -- 10.20.193.16:0/1298453140 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd0fc0700f0 msgr2=0x7fd0fc0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 1 --2- 10.20.193.16:0/1298453140 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd0fc0700f0 0x7fd0fc0704d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80098a0 tx=0x7fd0e802edd0 comp rx=0 tx=0).stop 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 1 -- 10.20.193.16:0/1298453140 shutdown_connections 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 1 --2- 10.20.193.16:0/1298453140 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd0fc0700f0 0x7fd0fc0704d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 1 -- 10.20.193.16:0/1298453140 >> 10.20.193.16:0/1298453140 conn(0x7fd0fc0cd770 msgr2=0x7fd0fc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 1 -- 10.20.193.16:0/1298453140 shutdown_connections 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.277+0000 7fd101ef2640 1 -- 10.20.193.16:0/1298453140 wait complete. 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient: build_initial_monmap 2026-02-17T05:59:40.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient: monmap: 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 librados: starting msgr at 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 librados: starting objecter 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 Processor -- start 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 -- start start 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 librados: setting wanted keys 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 librados: calling monclient init 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient: init 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd0fc0700f0 0x7fd0fc1e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient(hunting): picked mon.noname-b con 0x7fd0fc0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd0fc0c7fe0 0x7fd0fc148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient(hunting): picked mon.noname-c con 0x7fd0fc0c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd0fc1536f0 0x7fd0fc1e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient(hunting): picked mon.noname-a con 0x7fd0fc1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc074c00 con 0x7fd0fc1536f0 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc0c5680 con 0x7fd0fc0700f0 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:40.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc074d80 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.278+0000 7fd101ef2640 10 monclient(hunting): authenticate will time out at 1232.847855s 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fbfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd0fc1536f0 0x7fd0fc1e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0faffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd0fc0c7fe0 0x7fd0fc148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fb7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd0fc0700f0 0x7fd0fc1e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fbfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd0fc1536f0 0x7fd0fc1e6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:52100/0 (socket says 10.20.193.16:52100) 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fbfff640 1 -- 10.20.193.16:0/1441435390 learned_addr learned my addr 10.20.193.16:0/1441435390 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fbfff640 10 monclient(hunting): get_auth_request con 0x7fd0fc1536f0 auth_method 0 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0faffd640 10 monclient(hunting): get_auth_request con 0x7fd0fc0c7fe0 auth_method 0 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fb7fe640 10 monclient(hunting): get_auth_request con 0x7fd0fc0700f0 auth_method 0 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.279+0000 7fd0fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0faffd640 10 monclient(hunting): handle_auth_done global_id 4934 payload 824 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0faffd640 1 -- 10.20.193.16:0/1441435390 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd0fc0700f0 msgr2=0x7fd0fc1e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:40.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0faffd640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd0fc0700f0 0x7fd0fc1e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0faffd640 1 -- 10.20.193.16:0/1441435390 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd0fc1536f0 msgr2=0x7fd0fc1e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0faffd640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd0fc1536f0 0x7fd0fc1e6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0faffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0faffd640 1 monclient: found mon.noname-c 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0faffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0faffd640 1 -- 10.20.193.16:0/1441435390 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc1e7600 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.280+0000 7fd0fbfff640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd0fc1536f0 0x7fd0fc1e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.281+0000 7fd0faffd640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd0fc0c7fe0 0x7fd0fc148e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c480 tx=0x7fd0ec00c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd0d7fff640 10 client.?.objecter ms_handle_connect 0x7fd0fc0c7fe0 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd0d7fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd0d7fff640 1 -- 10.20.193.16:0/1441435390 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec017070 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd0d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd0d7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd0d7fff640 10 monclient: dump: 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:40.288 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd0d7fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd0d7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd0d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd101ef2640 5 monclient: authenticate success, global_id 4934 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd101ef2640 10 monclient: _renew_subs 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd101ef2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd101ef2640 1 -- 10.20.193.16:0/1441435390 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0fc1e78e0 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd101ef2640 10 client.4934.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd101ef2640 10 monclient: _renew_subs 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd101ef2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd101ef2640 1 -- 10.20.193.16:0/1441435390 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0fc0cbd80 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.282+0000 7fd101ef2640 1 librados: init done 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd101ef2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd101ef2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd101ef2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd101ef2640 1 -- 10.20.193.16:0/1441435390 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0fc0704d0 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd0d7fff640 1 -- 10.20.193.16:0/1441435390 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0ec0052d0 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd0d7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:40.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd0d7fff640 1 -- 10.20.193.16:0/1441435390 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec021060 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd0d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd0d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd0d7fff640 10 monclient: dump: 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:40.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.283+0000 7fd0f9ffb640 4 set_mon_vals no callback set 2026-02-17T05:59:40.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.393+0000 7fd0d7fff640 1 -- 10.20.193.16:0/1441435390 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fd0ec024080 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.393+0000 7fd0d7fff640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd0cc079140 0x7fd0cc07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:40.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.394+0000 7fd0fb7fe640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd0cc079140 0x7fd0cc07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:40.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.394+0000 7fd0fb7fe640 10 monclient: get_auth_request con 0x7fd0cc079140 auth_method 0 2026-02-17T05:59:40.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.394+0000 7fd0fb7fe640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd0cc079140 0x7fd0cc07b600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8039000 tx=0x7fd0e8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:40.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.394+0000 7fd0d7fff640 10 client.4934.objecter ms_handle_connect 0x7fd0cc079140 2026-02-17T05:59:40.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.396+0000 7fd0d7fff640 1 -- 10.20.193.16:0/1441435390 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fd0ec097040 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.396+0000 7fd0d7fff640 10 client.4934.objecter ms_dispatch2 0x7fd0fc000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:40.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.396+0000 7fd0d7fff640 3 client.4934.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:40.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.396+0000 7fd0d7fff640 3 client.4934.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:40.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.396+0000 7fd0d7fff640 20 client.4934.objecter dump_active .. 0 homeless 2026-02-17T05:59:40.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.402+0000 7fd0d7fff640 1 -- 10.20.193.16:0/1441435390 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd0ec061670 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.402+0000 7fd0d7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:40.403 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.402+0000 7fd0d7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:40.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.501+0000 7fd101ef2640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:40.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.501+0000 7fd101ef2640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:40.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.501+0000 7fd101ef2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:40.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.501+0000 7fd101ef2640 1 -- 10.20.193.16:0/1441435390 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd0fc1573f0 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.504+0000 7fd0d7fff640 1 -- 10.20.193.16:0/1441435390 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd0ec065970 con 0x7fd0fc0c7fe0 2026-02-17T05:59:40.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.504+0000 7fd0d7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:40.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.504+0000 7fd0d7fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:40.505 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:40.505 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.505+0000 7fd101ef2640 10 librados: watch_flush enter 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.505+0000 7fd101ef2640 10 librados: watch_flush exit 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.505+0000 7fd101ef2640 20 client.4934.objecter shutdown clearing up homeless session... 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.505+0000 7fd101ef2640 10 client.4934.objecter successfully canceled tick 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.505+0000 7fd101ef2640 1 -- 10.20.193.16:0/1441435390 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd0cc079140 msgr2=0x7fd0cc07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.505+0000 7fd101ef2640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd0cc079140 0x7fd0cc07b600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8039000 tx=0x7fd0e8033000 comp rx=0 tx=0).stop 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.505+0000 7fd101ef2640 10 monclient: shutdown 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.505+0000 7fd101ef2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.505+0000 7fd101ef2640 1 -- 10.20.193.16:0/1441435390 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd0fc0c7fe0 msgr2=0x7fd0fc148e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.505+0000 7fd101ef2640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd0fc0c7fe0 0x7fd0fc148e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c480 tx=0x7fd0ec00c950 comp rx=0 tx=0).stop 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.506+0000 7fd101ef2640 1 -- 10.20.193.16:0/1441435390 shutdown_connections 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.506+0000 7fd101ef2640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd0cc079140 0x7fd0cc07b600 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.506+0000 7fd101ef2640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd0fc1536f0 0x7fd0fc1e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.506+0000 7fd101ef2640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd0fc0c7fe0 0x7fd0fc148e90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.506+0000 7fd101ef2640 1 --2- 10.20.193.16:0/1441435390 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd0fc0700f0 0x7fd0fc1e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:40.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.506+0000 7fd101ef2640 1 -- 10.20.193.16:0/1441435390 >> 10.20.193.16:0/1441435390 conn(0x7fd0fc0cd770 msgr2=0x7fd0fc151180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:40.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.506+0000 7fd101ef2640 1 -- 10.20.193.16:0/1441435390 shutdown_connections 2026-02-17T05:59:40.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.506+0000 7fd101ef2640 1 -- 10.20.193.16:0/1441435390 wait complete. 2026-02-17T05:59:40.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:40.506+0000 7fd101ef2640 1 librados: shutdown 2026-02-17T05:59:43.515 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.586+0000 7f2c603a9640 10 monclient: get_monmap_and_config 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.586+0000 7f2c603a9640 10 monclient: build_initial_monmap 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.586+0000 7f2c603a9640 10 monclient: monmap: 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:43.587596+0000 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:43.587596+0000 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:43.587 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:43.588 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:43.588 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:43.588 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:43.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.586+0000 7f2c603a9640 1 Processor -- start 2026-02-17T05:59:43.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 1 -- start start 2026-02-17T05:59:43.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 10 monclient: init 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 10 monclient: _add_conns ranks=[2,3,4,5,0,1] 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2c5814f020 addr v1:10.20.193.148:6789/0 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f2c5814f400 0x7f2c5815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 10 monclient(hunting): picked mon.noname-a con 0x7f2c5814f400 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f2c5815c920 0x7f2c580c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 10 monclient(hunting): picked mon.noname-b con 0x7f2c5815c920 addr v2:10.20.193.128:3300/0 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c580c5b10 con 0x7f2c5814f020 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2c5815cd20 con 0x7f2c5814f400 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f2c5806e620 con 0x7f2c5815c920 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.587+0000 7f2c603a9640 10 monclient(hunting): authenticate will time out at 1236.156780s 2026-02-17T05:59:43.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5d91d640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f2c5814f400 0x7f2c5815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5e91f640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f2c5815c920 0x7f2c580c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5e11e640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f2c5814f020 0x7f2c5806dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:57988/0 (socket says 10.20.193.16:57988) 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5e11e640 1 -- 10.20.193.16:0/1127882932 learned_addr learned my addr 10.20.193.16:0/1127882932 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5d91d640 10 monclient(hunting): get_auth_request con 0x7f2c5814f400 auth_method 0 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5d91d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5d91d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5d91d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5e91f640 10 monclient(hunting): get_auth_request con 0x7f2c5815c920 auth_method 0 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5e91f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5e91f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5e91f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5d91d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5d91d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.588+0000 7f2c5d91d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5e91f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5e91f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:43.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5e91f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:43.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5d91d640 10 monclient(hunting): handle_auth_done global_id 4809 payload 274 2026-02-17T05:59:43.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5d91d640 1 -- 10.20.193.16:0/1127882932 >> v2:10.20.193.128:3300/0 conn(0x7f2c5815c920 msgr2=0x7f2c580c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:43.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5d91d640 1 --2- 10.20.193.16:0/1127882932 >> v2:10.20.193.128:3300/0 conn(0x7f2c5815c920 0x7f2c580c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:43.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5d91d640 1 -- 10.20.193.16:0/1127882932 >> v1:10.20.193.148:6789/0 conn(0x7f2c5814f020 legacy=0x7f2c5806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:43.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5d91d640 10 monclient: _finish_hunting 0 2026-02-17T05:59:43.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5d91d640 1 monclient: found mon.noname-a 2026-02-17T05:59:43.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5d91d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:43.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5d91d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:43.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.589+0000 7f2c5d91d640 1 -- 10.20.193.16:0/1127882932 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5806e900 con 0x7f2c5814f400 2026-02-17T05:59:43.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.590+0000 7f2c5d91d640 1 --2- 10.20.193.16:0/1127882932 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5814f400 0x7f2c5815c3e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2c580c9d30 tx=0x7f2c4c02eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209164d8d0a294c9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 1 -- 10.20.193.16:0/1127882932 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c03c070 con 0x7f2c5814f400 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 10 monclient: dump: 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:43.593 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 10 monclient: _finish_auth 0 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 10 monclient: _check_auth_tickets 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 1 -- 10.20.193.16:0/1127882932 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4c02fb80 con 0x7f2c5814f400 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 5 monclient: authenticate success, global_id 4809 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 1 -- 10.20.193.16:0/1127882932 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c037440 con 0x7f2c5814f400 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 4 set_mon_vals no callback set 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c5d11c640 10 monclient: dump: 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:43.594 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 10 monclient: shutdown 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 1 -- 10.20.193.16:0/1127882932 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5814f400 msgr2=0x7f2c5815c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 1 --2- 10.20.193.16:0/1127882932 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5814f400 0x7f2c5815c3e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2c580c9d30 tx=0x7f2c4c02eeb0 comp rx=0 tx=0).stop 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 1 -- 10.20.193.16:0/1127882932 shutdown_connections 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 1 --2- 10.20.193.16:0/1127882932 >> v2:10.20.193.128:3300/0 conn(0x7f2c5815c920 0x7f2c580c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 1 --2- 10.20.193.16:0/1127882932 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5814f400 0x7f2c5815c3e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 1 -- 10.20.193.16:0/1127882932 >> 10.20.193.16:0/1127882932 conn(0x7f2c580cd770 msgr2=0x7f2c580cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 1 -- 10.20.193.16:0/1127882932 shutdown_connections 2026-02-17T05:59:43.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.592+0000 7f2c603a9640 1 -- 10.20.193.16:0/1127882932 wait complete. 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.593+0000 7f2c603a9640 10 monclient: build_initial_monmap 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.593+0000 7f2c603a9640 10 monclient: monmap: 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:43.597 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.593+0000 7f2c603a9640 1 librados: starting msgr at 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.593+0000 7f2c603a9640 1 librados: starting objecter 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.593+0000 7f2c603a9640 1 Processor -- start 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.593+0000 7f2c603a9640 1 -- start start 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.593+0000 7f2c603a9640 1 librados: setting wanted keys 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.593+0000 7f2c603a9640 1 librados: calling monclient init 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.593+0000 7f2c603a9640 10 monclient: init 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2c5814f020 0x7f2c5814db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 10 monclient(hunting): picked mon.noname-b con 0x7f2c5814f020 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c5814f400 0x7f2c581f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 10 monclient(hunting): picked mon.noname-c con 0x7f2c5814f400 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5815c920 0x7f2c581f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 10 monclient(hunting): picked mon.noname-a con 0x7f2c5815c920 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5806f1a0 con 0x7f2c5815c920 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5806eec0 con 0x7f2c5814f020 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5806e2f0 con 0x7f2c5814f400 2026-02-17T05:59:43.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c603a9640 10 monclient(hunting): authenticate will time out at 1236.163126s 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e91f640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5815c920 0x7f2c581f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e91f640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5815c920 0x7f2c581f68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:52114/0 (socket says 10.20.193.16:52114) 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e91f640 1 -- 10.20.193.16:0/2734198253 learned_addr learned my addr 10.20.193.16:0/2734198253 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e91f640 10 monclient(hunting): get_auth_request con 0x7f2c5815c920 auth_method 0 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e91f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e91f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e91f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5d91d640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c5814f400 0x7f2c581f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e11e640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2c5814f020 0x7f2c5814db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e91f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e91f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e91f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5d91d640 10 monclient(hunting): get_auth_request con 0x7f2c5814f400 auth_method 0 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5d91d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5d91d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5d91d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e11e640 10 monclient(hunting): get_auth_request con 0x7f2c5814f020 auth_method 0 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e11e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e11e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.594+0000 7f2c5e11e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:43.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.595+0000 7f2c5d91d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.595+0000 7f2c5d91d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.595+0000 7f2c5d91d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.595+0000 7f2c5e11e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.595+0000 7f2c5e11e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.595+0000 7f2c5e11e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.596+0000 7f2c5e91f640 10 monclient(hunting): handle_auth_done global_id 4812 payload 824 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.596+0000 7f2c5e91f640 1 -- 10.20.193.16:0/2734198253 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c5814f400 msgr2=0x7f2c581f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.596+0000 7f2c5e91f640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c5814f400 0x7f2c581f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.596+0000 7f2c5e91f640 1 -- 10.20.193.16:0/2734198253 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2c5814f020 msgr2=0x7f2c5814db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.596+0000 7f2c5e91f640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2c5814f020 0x7f2c5814db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.596+0000 7f2c5e91f640 10 monclient: _finish_hunting 0 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.596+0000 7f2c5e91f640 1 monclient: found mon.noname-a 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.596+0000 7f2c5e91f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.596+0000 7f2c5e91f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:43.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.596+0000 7f2c5e91f640 1 -- 10.20.193.16:0/2734198253 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c581f5010 con 0x7f2c5815c920 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.597+0000 7f2c5e91f640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5815c920 0x7f2c581f68d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400efc0 tx=0x7f2c5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.597+0000 7f2c3effd640 10 client.?.objecter ms_handle_connect 0x7f2c5815c920 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.597+0000 7f2c3effd640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 1 -- 10.20.193.16:0/2734198253 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54018020 con 0x7f2c5815c920 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 10 monclient: dump: 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:43.601 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 10 monclient: _finish_auth 0 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 1 -- 10.20.193.16:0/2734198253 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c54014480 con 0x7f2c5815c920 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 1 -- 10.20.193.16:0/2734198253 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54014870 con 0x7f2c5815c920 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c3effd640 10 monclient: dump: 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c5c91b640 4 set_mon_vals no callback set 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 5 monclient: authenticate success, global_id 4812 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 10 monclient: _renew_subs 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 1 -- 10.20.193.16:0/2734198253 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c581f52f0 con 0x7f2c5815c920 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 10 client.4812.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 10 monclient: _renew_subs 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 1 -- 10.20.193.16:0/2734198253 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c581f58b0 con 0x7f2c5815c920 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 1 librados: init done 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:43.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:43.603 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:43.603 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.599+0000 7f2c603a9640 1 -- 10.20.193.16:0/2734198253 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5815c3e0 con 0x7f2c5815c920 2026-02-17T05:59:43.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.611+0000 7f2c3effd640 1 -- 10.20.193.16:0/2734198253 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f2c54014a10 con 0x7f2c5815c920 2026-02-17T05:59:43.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.611+0000 7f2c3effd640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c28079190 0x7f2c2807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:43.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.611+0000 7f2c5e11e640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c28079190 0x7f2c2807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:43.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.611+0000 7f2c5e11e640 10 monclient: get_auth_request con 0x7f2c28079190 auth_method 0 2026-02-17T05:59:43.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.612+0000 7f2c5e11e640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c28079190 0x7f2c2807b650 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2c48002770 tx=0x7f2c48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:43.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.612+0000 7f2c3effd640 10 client.4812.objecter ms_handle_connect 0x7f2c28079190 2026-02-17T05:59:43.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.613+0000 7f2c3effd640 1 -- 10.20.193.16:0/2734198253 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f2c5405f020 con 0x7f2c5815c920 2026-02-17T05:59:43.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.613+0000 7f2c3effd640 10 client.4812.objecter ms_dispatch2 0x7f2c58000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:43.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.613+0000 7f2c3effd640 3 client.4812.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:43.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.613+0000 7f2c3effd640 3 client.4812.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:43.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.613+0000 7f2c3effd640 20 client.4812.objecter dump_active .. 0 homeless 2026-02-17T05:59:43.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.715+0000 7f2c3effd640 1 -- 10.20.193.16:0/2734198253 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2c54022070 con 0x7f2c5815c920 2026-02-17T05:59:43.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.715+0000 7f2c3effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:43.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.715+0000 7f2c3effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:43.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.815+0000 7f2c603a9640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:43.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.815+0000 7f2c603a9640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:43.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.815+0000 7f2c603a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:43.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.815+0000 7f2c603a9640 1 -- 10.20.193.16:0/2734198253 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2c581562c0 con 0x7f2c5815c920 2026-02-17T05:59:43.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.818+0000 7f2c3effd640 1 -- 10.20.193.16:0/2734198253 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f2c54010040 con 0x7f2c5815c920 2026-02-17T05:59:43.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.818+0000 7f2c3effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:43.819 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.818+0000 7f2c3effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:43.819 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:43.820 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:43.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 10 librados: watch_flush enter 2026-02-17T05:59:43.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 10 librados: watch_flush exit 2026-02-17T05:59:43.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 20 client.4812.objecter shutdown clearing up homeless session... 2026-02-17T05:59:43.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 10 client.4812.objecter successfully canceled tick 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 -- 10.20.193.16:0/2734198253 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c28079190 msgr2=0x7f2c2807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c28079190 0x7f2c2807b650 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2c48002770 tx=0x7f2c48009040 comp rx=0 tx=0).stop 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 10 monclient: shutdown 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 -- 10.20.193.16:0/2734198253 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5815c920 msgr2=0x7f2c581f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5815c920 0x7f2c581f68d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400efc0 tx=0x7f2c5400c490 comp rx=0 tx=0).stop 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 -- 10.20.193.16:0/2734198253 shutdown_connections 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c28079190 0x7f2c2807b650 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2c5815c920 0x7f2c581f68d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2c5814f400 0x7f2c581f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:43.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 --2- 10.20.193.16:0/2734198253 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2c5814f020 0x7f2c5814db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:43.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 -- 10.20.193.16:0/2734198253 >> 10.20.193.16:0/2734198253 conn(0x7f2c580cd770 msgr2=0x7f2c580cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:43.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 -- 10.20.193.16:0/2734198253 shutdown_connections 2026-02-17T05:59:43.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 -- 10.20.193.16:0/2734198253 wait complete. 2026-02-17T05:59:43.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:43.819+0000 7f2c603a9640 1 librados: shutdown 2026-02-17T05:59:44.073 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:05:22.900 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:44.075 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:05:22.903 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:44.082 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:05:22.910 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:44.104 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:05:22.930 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:44.104 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:05:22.893 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:44.105 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:05:22.894 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:44.105 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:05:22.931 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:44.113 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:05:22.939 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:44.114 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:05:22.904 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T05:59:46.829 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:46.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.900+0000 7f81257bc640 10 monclient: get_monmap_and_config 2026-02-17T05:59:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.900+0000 7f81257bc640 10 monclient: build_initial_monmap 2026-02-17T05:59:46.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.900+0000 7f81257bc640 10 monclient: monmap: 2026-02-17T05:59:46.902 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:46.902 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:46.902 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:46.901827+0000 2026-02-17T05:59:46.902 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:46.901827+0000 2026-02-17T05:59:46.902 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:46.903 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:46.903 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:46.903 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:46.903 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:46.903 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:46.903 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:46.903 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 1 Processor -- start 2026-02-17T05:59:46.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 1 -- start start 2026-02-17T05:59:46.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 10 monclient: init 2026-02-17T05:59:46.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 10 monclient: _add_conns ranks=[3,5,1,0,4,2] 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f812014f020 0x7f812006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 10 monclient(hunting): picked mon.noname-a con 0x7f812014f020 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f812006e120 0x7f812015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 10 monclient(hunting): picked mon.noname-c con 0x7f812006e120 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f812014f400 addr v1:10.20.193.128:6789/0 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81200c5b10 con 0x7f812014f400 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f812006e620 con 0x7f812014f020 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:46.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f812006e900 con 0x7f812006e120 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.901+0000 7f81257bc640 10 monclient(hunting): authenticate will time out at 1239.470896s 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811effd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f812014f020 0x7f812006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811e7fc640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f812006e120 0x7f812015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811f7fe640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f812014f400 0x7f81200c6010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:40918/0 (socket says 10.20.193.16:40918) 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811f7fe640 1 -- 10.20.193.16:0/2604095808 learned_addr learned my addr 10.20.193.16:0/2604095808 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811effd640 10 monclient(hunting): get_auth_request con 0x7f812014f020 auth_method 0 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811e7fc640 10 monclient(hunting): get_auth_request con 0x7f812006e120 auth_method 0 2026-02-17T05:59:46.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.902+0000 7f811effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.903+0000 7f811e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.903+0000 7f811e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.903+0000 7f811e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.904+0000 7f811e7fc640 10 monclient(hunting): handle_auth_done global_id 4943 payload 274 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.904+0000 7f811e7fc640 1 -- 10.20.193.16:0/2604095808 >> v2:10.20.193.16:3300/0 conn(0x7f812014f020 msgr2=0x7f812006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.904+0000 7f811e7fc640 1 --2- 10.20.193.16:0/2604095808 >> v2:10.20.193.16:3300/0 conn(0x7f812014f020 0x7f812006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:46.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.904+0000 7f811e7fc640 1 -- 10.20.193.16:0/2604095808 >> v1:10.20.193.128:6789/0 conn(0x7f812014f400 legacy=0x7f81200c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.904+0000 7f811e7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:59:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.904+0000 7f811e7fc640 1 monclient: found mon.noname-c 2026-02-17T05:59:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.904+0000 7f811e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.904+0000 7f811e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:59:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.904+0000 7f811e7fc640 1 -- 10.20.193.16:0/2604095808 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812006ebe0 con 0x7f812006e120 2026-02-17T05:59:46.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.905+0000 7f811e7fc640 1 --2- 10.20.193.16:0/2604095808 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f812006e120 0x7f812015c3e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8120156400 tx=0x7f810802eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ab956b9d0fdb76 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 1 -- 10.20.193.16:0/2604095808 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810803c070 con 0x7f812006e120 2026-02-17T05:59:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:46.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 10 monclient: dump: 2026-02-17T05:59:46.918 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:46.918 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:46.918 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:46.918 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:46.918 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:46.918 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:46.918 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:46.918 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:46.918 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:46.918 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 1 -- 10.20.193.16:0/2604095808 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810802fb80 con 0x7f812006e120 2026-02-17T05:59:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f81257bc640 5 monclient: authenticate success, global_id 4943 2026-02-17T05:59:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f81257bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 1 -- 10.20.193.16:0/2604095808 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108037440 con 0x7f812006e120 2026-02-17T05:59:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f81257bc640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:46.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f81257bc640 4 set_mon_vals no callback set 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.906+0000 7f811dffb640 10 monclient: dump: 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:46.920 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 10 monclient: shutdown 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 1 -- 10.20.193.16:0/2604095808 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f812006e120 msgr2=0x7f812015c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 1 --2- 10.20.193.16:0/2604095808 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f812006e120 0x7f812015c3e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8120156400 tx=0x7f810802eeb0 comp rx=0 tx=0).stop 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 1 -- 10.20.193.16:0/2604095808 shutdown_connections 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 1 --2- 10.20.193.16:0/2604095808 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f812006e120 0x7f812015c3e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 1 --2- 10.20.193.16:0/2604095808 >> v2:10.20.193.16:3300/0 conn(0x7f812014f020 0x7f812006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 1 -- 10.20.193.16:0/2604095808 >> 10.20.193.16:0/2604095808 conn(0x7f81200cd770 msgr2=0x7f81200cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 1 -- 10.20.193.16:0/2604095808 shutdown_connections 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 1 -- 10.20.193.16:0/2604095808 wait complete. 2026-02-17T05:59:46.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 10 monclient: build_initial_monmap 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.907+0000 7f81257bc640 10 monclient: monmap: 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:46.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 librados: starting msgr at 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 librados: starting objecter 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 Processor -- start 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 -- start start 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 librados: setting wanted keys 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 librados: calling monclient init 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient: init 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f812006e120 0x7f81201580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient(hunting): picked mon.noname-b con 0x7f812006e120 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f812014f020 0x7f8120161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:46.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient(hunting): picked mon.noname-c con 0x7f812014f020 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f812014f400 0x7f812014aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient(hunting): picked mon.noname-a con 0x7f812014f400 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81200c6720 con 0x7f812014f400 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f812006f1a0 con 0x7f812006e120 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81200c68a0 con 0x7f812014f020 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.908+0000 7f81257bc640 10 monclient(hunting): authenticate will time out at 1239.477936s 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811f7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f812014f400 0x7f812014aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:46.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811e7fc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f812014f020 0x7f8120161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811f7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f812014f400 0x7f812014aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:52128/0 (socket says 10.20.193.16:52128) 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811effd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f812006e120 0x7f81201580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811f7fe640 1 -- 10.20.193.16:0/3691345815 learned_addr learned my addr 10.20.193.16:0/3691345815 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811f7fe640 10 monclient(hunting): get_auth_request con 0x7f812014f400 auth_method 0 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811e7fc640 10 monclient(hunting): get_auth_request con 0x7f812014f020 auth_method 0 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:46.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811effd640 10 monclient(hunting): get_auth_request con 0x7f812006e120 auth_method 0 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.909+0000 7f811effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.911+0000 7f811e7fc640 10 monclient(hunting): handle_auth_done global_id 4946 payload 824 2026-02-17T05:59:46.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.911+0000 7f811e7fc640 1 -- 10.20.193.16:0/3691345815 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f812006e120 msgr2=0x7f81201580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.911+0000 7f811e7fc640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f812006e120 0x7f81201580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.911+0000 7f811e7fc640 1 -- 10.20.193.16:0/3691345815 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f812014f400 msgr2=0x7f812014aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.911+0000 7f811e7fc640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f812014f400 0x7f812014aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.911+0000 7f811e7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.911+0000 7f811e7fc640 1 monclient: found mon.noname-c 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.911+0000 7f811e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.911+0000 7f811e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.911+0000 7f811e7fc640 1 -- 10.20.193.16:0/3691345815 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81201627a0 con 0x7f812014f020 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.912+0000 7f811e7fc640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f812014f020 0x7f8120161f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8108036000 tx=0x7f81080047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.912+0000 7f80ff7fe640 10 client.?.objecter ms_handle_connect 0x7f812014f020 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.912+0000 7f80ff7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 1 -- 10.20.193.16:0/3691345815 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810803c040 con 0x7f812014f020 2026-02-17T05:59:46.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 10 monclient: dump: 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:46.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 5 monclient: authenticate success, global_id 4946 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 10 monclient: _renew_subs 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 1 -- 10.20.193.16:0/3691345815 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8120162a80 con 0x7f812014f020 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 10 client.4946.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 10 monclient: _renew_subs 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 1 -- 10.20.193.16:0/3691345815 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812013ee50 con 0x7f812014f020 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 1 librados: init done 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f81257bc640 1 -- 10.20.193.16:0/3691345815 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f812006dbe0 con 0x7f812014f020 2026-02-17T05:59:46.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 1 -- 10.20.193.16:0/3691345815 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810802fac0 con 0x7f812014f020 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 1 -- 10.20.193.16:0/3691345815 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108042670 con 0x7f812014f020 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f80ff7fe640 10 monclient: dump: 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:46.930 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:46.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:46.913+0000 7f811d7fa640 4 set_mon_vals no callback set 2026-02-17T05:59:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.021+0000 7f80ff7fe640 1 -- 10.20.193.16:0/3691345815 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f8108042810 con 0x7f812014f020 2026-02-17T05:59:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.022+0000 7f80ff7fe640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f80ec079140 0x7f80ec07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.022+0000 7f811effd640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f80ec079140 0x7f80ec07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:47.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.022+0000 7f811effd640 10 monclient: get_auth_request con 0x7f80ec079140 auth_method 0 2026-02-17T05:59:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.022+0000 7f811effd640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f80ec079140 0x7f80ec07b600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8114009ba0 tx=0x7f8114009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:47.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.022+0000 7f80ff7fe640 10 client.4946.objecter ms_handle_connect 0x7f80ec079140 2026-02-17T05:59:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.024+0000 7f80ff7fe640 1 -- 10.20.193.16:0/3691345815 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f81080bcfd0 con 0x7f812014f020 2026-02-17T05:59:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.024+0000 7f80ff7fe640 10 client.4946.objecter ms_dispatch2 0x7f8120000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.024+0000 7f80ff7fe640 3 client.4946.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.024+0000 7f80ff7fe640 3 client.4946.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:47.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.024+0000 7f80ff7fe640 20 client.4946.objecter dump_active .. 0 homeless 2026-02-17T05:59:47.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.030+0000 7f80ff7fe640 1 -- 10.20.193.16:0/3691345815 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8108087710 con 0x7f812014f020 2026-02-17T05:59:47.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.030+0000 7f80ff7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:47.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.030+0000 7f80ff7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.130+0000 7f81257bc640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.130+0000 7f81257bc640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.130+0000 7f81257bc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T05:59:47.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.130+0000 7f81257bc640 1 -- 10.20.193.16:0/3691345815 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f81200c7b50 con 0x7f812014f020 2026-02-17T05:59:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f80ff7fe640 1 -- 10.20.193.16:0/3691345815 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f810808ba10 con 0x7f812014f020 2026-02-17T05:59:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f80ff7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:47.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f80ff7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:47.133 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:47.133 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f81257bc640 10 librados: watch_flush enter 2026-02-17T05:59:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f81257bc640 10 librados: watch_flush exit 2026-02-17T05:59:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f81257bc640 20 client.4946.objecter shutdown clearing up homeless session... 2026-02-17T05:59:47.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f81257bc640 10 client.4946.objecter successfully canceled tick 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f81257bc640 1 -- 10.20.193.16:0/3691345815 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f80ec079140 msgr2=0x7f80ec07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f81257bc640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f80ec079140 0x7f80ec07b600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8114009ba0 tx=0x7f8114009290 comp rx=0 tx=0).stop 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f81257bc640 10 monclient: shutdown 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f81257bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f81257bc640 1 -- 10.20.193.16:0/3691345815 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f812014f020 msgr2=0x7f8120161f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.133+0000 7f81257bc640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f812014f020 0x7f8120161f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8108036000 tx=0x7f81080047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.134+0000 7f81257bc640 1 -- 10.20.193.16:0/3691345815 shutdown_connections 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.134+0000 7f81257bc640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f80ec079140 0x7f80ec07b600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.134+0000 7f81257bc640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f812014f400 0x7f812014aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.134+0000 7f81257bc640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f812014f020 0x7f8120161f00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:47.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.134+0000 7f81257bc640 1 --2- 10.20.193.16:0/3691345815 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f812006e120 0x7f81201580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.134+0000 7f81257bc640 1 -- 10.20.193.16:0/3691345815 >> 10.20.193.16:0/3691345815 conn(0x7f81200cd770 msgr2=0x7f81200702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.134+0000 7f81257bc640 1 -- 10.20.193.16:0/3691345815 shutdown_connections 2026-02-17T05:59:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.134+0000 7f81257bc640 1 -- 10.20.193.16:0/3691345815 wait complete. 2026-02-17T05:59:47.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:47.134+0000 7f81257bc640 1 librados: shutdown 2026-02-17T05:59:50.144 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:50.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.214+0000 7f8eff291640 10 monclient: get_monmap_and_config 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.214+0000 7f8eff291640 10 monclient: build_initial_monmap 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.214+0000 7f8eff291640 10 monclient: monmap: 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:50.215261+0000 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:50.215261+0000 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:50.215 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.214+0000 7f8eff291640 1 Processor -- start 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.214+0000 7f8eff291640 1 -- start start 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.214+0000 7f8eff291640 10 monclient: init 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8eff291640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8eff291640 10 monclient: _add_conns ranks=[0,2,1,5,4,3] 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8eff291640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8ef814f020 addr v1:10.20.193.16:6789/0 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8eff291640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8ef814f400 addr v1:10.20.193.148:6789/0 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8eff291640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f8ef815caf0 addr v1:10.20.193.128:6789/0 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8eff291640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef80c5b10 con 0x7f8ef814f020 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8eff291640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef806e620 con 0x7f8ef815caf0 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8eff291640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef806e900 con 0x7f8ef814f400 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8eff291640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8eff291640 10 monclient(hunting): authenticate will time out at 1242.784315s 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8efd807640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f8ef815caf0 0x7f8ef80c6010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:41548/0 (socket says 10.20.193.16:41548) 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8efc805640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f8ef814f400 0x7f8ef815c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:48810/0 (socket says 10.20.193.16:48810) 2026-02-17T05:59:50.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.215+0000 7f8efd807640 1 -- 10.20.193.16:0/2409377873 learned_addr learned my addr 10.20.193.16:0/2409377873 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.243+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1390953344 0 0) 0x7f8ef80c5b10 con 0x7f8ef814f020 2026-02-17T05:59:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.243+0000 7f8ee7fff640 10 monclient(hunting): _negotiate 2026-02-17T05:59:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.243+0000 7f8ee7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.243+0000 7f8ee7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.243+0000 7f8ee7fff640 10 monclient(hunting): my global_id is 4821 2026-02-17T05:59:50.243 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.243+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ee0003620 con 0x7f8ef814f020 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 <== mon.0 v1:10.20.193.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3548852860 0 0) 0x7f8ee0003620 con 0x7f8ef814f020 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 >> v1:10.20.193.148:6789/0 conn(0x7f8ef814f400 legacy=0x7f8ef815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 >> v1:10.20.193.128:6789/0 conn(0x7f8ef815caf0 legacy=0x7f8ef80c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.16:6789/0 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 --> v1:10.20.193.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef806ebe0 con 0x7f8ef814f020 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 <== mon.0 v1:10.20.193.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f8ee8002f00 con 0x7f8ef814f020 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient: dump: 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] but i'm connected to v1:10.20.193.16:6789/0, reconnecting 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 >> v1:10.20.193.16:6789/0 conn(0x7f8ef814f020 legacy=0x7f8ef806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 --2- 10.20.193.16:0/2409377873 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8ee0003c20 0x7f8ee0004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient(hunting): picked mon.c con 0x7f8ee0003c20 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 --2- 10.20.193.16:0/2409377873 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ee0004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient(hunting): picked mon.a con 0x7f8ee0004900 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 --2- 10.20.193.16:0/2409377873 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8ee0005510 0x7f8ee000de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient(hunting): picked mon.b con 0x7f8ee0005510 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee000e6b0 con 0x7f8ee0004900 2026-02-17T05:59:50.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee000e860 con 0x7f8ee0005510 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee000ea60 con 0x7f8ee0003c20 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.245+0000 7f8ee7fff640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.246+0000 7f8efc805640 1 --2- 10.20.193.16:0/2409377873 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ee0004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.247+0000 7f8efc805640 10 monclient(hunting): get_auth_request con 0x7f8ee0004900 auth_method 0 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.247+0000 7f8efc805640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.247+0000 7f8efc805640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.247+0000 7f8efc805640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.247+0000 7f8efc805640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.247+0000 7f8efc805640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:50.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.247+0000 7f8efc805640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 10 monclient(hunting): handle_auth_done global_id 4821 payload 293 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 1 -- 10.20.193.16:0/2409377873 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8ee0003c20 msgr2=0x7f8ee0004040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 1 --2- 10.20.193.16:0/2409377873 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8ee0003c20 0x7f8ee0004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 1 -- 10.20.193.16:0/2409377873 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8ee0005510 msgr2=0x7f8ee000de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 1 --2- 10.20.193.16:0/2409377873 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8ee0005510 0x7f8ee000de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 10 monclient: _finish_hunting 0 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 1 monclient: found mon.a 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 1 -- 10.20.193.16:0/2409377873 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee0003620 con 0x7f8ee0004900 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efd807640 1 -- 10.20.193.16:0/2409377873 reap_dead start 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 10 monclient: _finish_auth 0 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 10 monclient: _check_auth_tickets 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8eff291640 5 monclient: authenticate success, global_id 4821 2026-02-17T05:59:50.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8eff291640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:50.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.248+0000 7f8efc805640 1 --2- 10.20.193.16:0/2409377873 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ee0004d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0030c0 tx=0x7f8eec02ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9e0c132fdf413c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:50.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec002ce0 con 0x7f8ee0004900 2026-02-17T05:59:50.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8ee7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8ee7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8ee7fff640 10 monclient: dump: 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eec033070 con 0x7f8ee0004900 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8eff291640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8ee7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8ee7fff640 1 -- 10.20.193.16:0/2409377873 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec005e30 con 0x7f8ee0004900 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8ee7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8ee7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8ee7fff640 10 monclient: dump: 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8eff291640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:50.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8eff291640 4 set_mon_vals no callback set 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8eff291640 10 monclient: shutdown 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8eff291640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8eff291640 1 -- 10.20.193.16:0/2409377873 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 msgr2=0x7f8ee0004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8eff291640 1 --2- 10.20.193.16:0/2409377873 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ee0004d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0030c0 tx=0x7f8eec02ee10 comp rx=0 tx=0).stop 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8eff291640 1 -- 10.20.193.16:0/2409377873 shutdown_connections 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8eff291640 1 --2- 10.20.193.16:0/2409377873 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ee0004d80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.250+0000 7f8eff291640 1 -- 10.20.193.16:0/2409377873 >> 10.20.193.16:0/2409377873 conn(0x7f8ef80cd770 msgr2=0x7f8ef80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 1 -- 10.20.193.16:0/2409377873 shutdown_connections 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 1 -- 10.20.193.16:0/2409377873 wait complete. 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 10 monclient: build_initial_monmap 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 10 monclient: monmap: 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 1 librados: starting msgr at 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 1 librados: starting objecter 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 1 Processor -- start 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 1 -- start start 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 1 librados: setting wanted keys 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 1 librados: calling monclient init 2026-02-17T05:59:50.252 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.251+0000 7f8eff291640 10 monclient: init 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ef815b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 10 monclient(hunting): picked mon.noname-a con 0x7f8ee0004900 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8ef8073010 0x7f8ef80c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 10 monclient(hunting): picked mon.noname-b con 0x7f8ef8073010 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8ef80c7190 0x7f8ef81f7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 10 monclient(hunting): picked mon.noname-c con 0x7f8ef80c7190 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef815c3e0 con 0x7f8ee0004900 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef806f1a0 con 0x7f8ef8073010 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef815c560 con 0x7f8ef80c7190 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8eff291640 10 monclient(hunting): authenticate will time out at 1242.821117s 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8efd006640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ef815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8efd006640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ef815b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:43176/0 (socket says 10.20.193.16:43176) 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8efd006640 1 -- 10.20.193.16:0/1468168035 learned_addr learned my addr 10.20.193.16:0/1468168035 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8efd006640 10 monclient(hunting): get_auth_request con 0x7f8ee0004900 auth_method 0 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8efd006640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8efd006640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8efd006640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8efd006640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8efd006640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:50.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.252+0000 7f8efd006640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:50.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.254+0000 7f8efd006640 10 monclient(hunting): handle_auth_done global_id 4824 payload 824 2026-02-17T05:59:50.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.254+0000 7f8efd006640 1 -- 10.20.193.16:0/1468168035 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8ef80c7190 msgr2=0x7f8ef81f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:50.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.254+0000 7f8efd006640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8ef80c7190 0x7f8ef81f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:50.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.254+0000 7f8efd006640 1 -- 10.20.193.16:0/1468168035 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8ef8073010 msgr2=0x7f8ef80c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:50.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.254+0000 7f8efd006640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8ef8073010 0x7f8ef80c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:50.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.254+0000 7f8efd006640 10 monclient: _finish_hunting 0 2026-02-17T05:59:50.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.254+0000 7f8efd006640 1 monclient: found mon.noname-a 2026-02-17T05:59:50.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.254+0000 7f8efd006640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:50.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.254+0000 7f8efd006640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:50.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.254+0000 7f8efd006640 1 -- 10.20.193.16:0/1468168035 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef815b760 con 0x7f8ee0004900 2026-02-17T05:59:50.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.255+0000 7f8efd006640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ef815b220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c820 tx=0x7f8ee800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:50.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.255+0000 7f8ee5ffb640 10 client.?.objecter ms_handle_connect 0x7f8ee0004900 2026-02-17T05:59:50.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.255+0000 7f8ee5ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 1 -- 10.20.193.16:0/1468168035 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8019070 con 0x7f8ee0004900 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 10 monclient: dump: 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 1 -- 10.20.193.16:0/1468168035 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee8007ce0 con 0x7f8ee0004900 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 1 -- 10.20.193.16:0/1468168035 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8014670 con 0x7f8ee0004900 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee77fe640 4 set_mon_vals no callback set 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8ee5ffb640 10 monclient: dump: 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8eff291640 5 monclient: authenticate success, global_id 4824 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8eff291640 10 monclient: _renew_subs 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8eff291640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:50.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8eff291640 1 -- 10.20.193.16:0/1468168035 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef8159ea0 con 0x7f8ee0004900 2026-02-17T05:59:50.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8eff291640 10 client.4824.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:50.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8eff291640 10 monclient: _renew_subs 2026-02-17T05:59:50.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8eff291640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:50.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8eff291640 1 -- 10.20.193.16:0/1468168035 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef815a460 con 0x7f8ee0004900 2026-02-17T05:59:50.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.256+0000 7f8eff291640 1 librados: init done 2026-02-17T05:59:50.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.257+0000 7f8eff291640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:50.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.257+0000 7f8eff291640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:50.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.257+0000 7f8eff291640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:50.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.257+0000 7f8eff291640 1 -- 10.20.193.16:0/1468168035 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef806dbe0 con 0x7f8ee0004900 2026-02-17T05:59:50.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.268+0000 7f8ee5ffb640 1 -- 10.20.193.16:0/1468168035 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f8ee8008040 con 0x7f8ee0004900 2026-02-17T05:59:50.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.268+0000 7f8ee5ffb640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8ec8079140 0x7f8ec807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:50.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.268+0000 7f8efc805640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8ec8079140 0x7f8ec807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:50.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.269+0000 7f8efc805640 10 monclient: get_auth_request con 0x7f8ec8079140 auth_method 0 2026-02-17T05:59:50.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.269+0000 7f8ee5ffb640 10 client.4824.objecter ms_handle_connect 0x7f8ec8079140 2026-02-17T05:59:50.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.269+0000 7f8efc805640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8ec8079140 0x7f8ec807b600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8eec037000 tx=0x7f8eec031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:50.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.271+0000 7f8ee5ffb640 1 -- 10.20.193.16:0/1468168035 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f8ee805df50 con 0x7f8ee0004900 2026-02-17T05:59:50.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.271+0000 7f8ee5ffb640 10 client.4824.objecter ms_dispatch2 0x7f8ef8000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:50.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.271+0000 7f8ee5ffb640 3 client.4824.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:50.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.271+0000 7f8ee5ffb640 3 client.4824.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:50.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.271+0000 7f8ee5ffb640 20 client.4824.objecter dump_active .. 0 homeless 2026-02-17T05:59:50.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.374+0000 7f8ee5ffb640 1 -- 10.20.193.16:0/1468168035 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8ee8023020 con 0x7f8ee0004900 2026-02-17T05:59:50.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.374+0000 7f8ee5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:50.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.374+0000 7f8ee5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:50.479 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.478+0000 7f8eff291640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:50.479 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.478+0000 7f8eff291640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:50.479 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.478+0000 7f8eff291640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:50.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.478+0000 7f8eff291640 1 -- 10.20.193.16:0/1468168035 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8ef80c64c0 con 0x7f8ee0004900 2026-02-17T05:59:50.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.481+0000 7f8ee5ffb640 1 -- 10.20.193.16:0/1468168035 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8ee8062610 con 0x7f8ee0004900 2026-02-17T05:59:50.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.481+0000 7f8ee5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:50.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.481+0000 7f8ee5ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:50.483 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:50.483 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:50.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 10 librados: watch_flush enter 2026-02-17T05:59:50.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 10 librados: watch_flush exit 2026-02-17T05:59:50.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 20 client.4824.objecter shutdown clearing up homeless session... 2026-02-17T05:59:50.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 10 client.4824.objecter successfully canceled tick 2026-02-17T05:59:50.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 -- 10.20.193.16:0/1468168035 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8ec8079140 msgr2=0x7f8ec807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:50.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8ec8079140 0x7f8ec807b600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8eec037000 tx=0x7f8eec031000 comp rx=0 tx=0).stop 2026-02-17T05:59:50.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 10 monclient: shutdown 2026-02-17T05:59:50.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:50.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 -- 10.20.193.16:0/1468168035 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 msgr2=0x7f8ef815b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:50.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ef815b220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c820 tx=0x7f8ee800ccf0 comp rx=0 tx=0).stop 2026-02-17T05:59:50.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 -- 10.20.193.16:0/1468168035 shutdown_connections 2026-02-17T05:59:50.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8ec8079140 0x7f8ec807b600 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:50.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8ef8073010 0x7f8ef80c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:50.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8ef80c7190 0x7f8ef81f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:50.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 --2- 10.20.193.16:0/1468168035 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8ee0004900 0x7f8ef815b220 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:50.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 -- 10.20.193.16:0/1468168035 >> 10.20.193.16:0/1468168035 conn(0x7f8ef80cd770 msgr2=0x7f8ef80c92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:50.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 -- 10.20.193.16:0/1468168035 shutdown_connections 2026-02-17T05:59:50.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 -- 10.20.193.16:0/1468168035 wait complete. 2026-02-17T05:59:50.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:50.482+0000 7f8eff291640 1 librados: shutdown 2026-02-17T05:59:53.493 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:53.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.563+0000 7fcf2eb49640 10 monclient: get_monmap_and_config 2026-02-17T05:59:53.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.563+0000 7fcf2eb49640 10 monclient: build_initial_monmap 2026-02-17T05:59:53.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.563+0000 7fcf2eb49640 10 monclient: monmap: 2026-02-17T05:59:53.564 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:53.564245+0000 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:53.564245+0000 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:53.565 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:53.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.563+0000 7fcf2eb49640 1 Processor -- start 2026-02-17T05:59:53.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.563+0000 7fcf2eb49640 1 -- start start 2026-02-17T05:59:53.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.563+0000 7fcf2eb49640 10 monclient: init 2026-02-17T05:59:53.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:53.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 10 monclient: _add_conns ranks=[2,3,5,1,4,0] 2026-02-17T05:59:53.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcf280700f0 addr v1:10.20.193.148:6789/0 2026-02-17T05:59:53.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fcf280c86f0 0x7fcf280704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:53.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 10 monclient(hunting): picked mon.noname-a con 0x7fcf280c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:53.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fcf281536f0 0x7fcf28153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:53.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 10 monclient(hunting): picked mon.noname-c con 0x7fcf281536f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:53.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf280c4820 con 0x7fcf280700f0 2026-02-17T05:59:53.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:53.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcf280c4b00 con 0x7fcf280c86f0 2026-02-17T05:59:53.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:53.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fcf280c4de0 con 0x7fcf281536f0 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf2eb49640 10 monclient(hunting): authenticate will time out at 1246.133206s 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf1ffff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fcf280c86f0 0x7fcf280704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf1ffff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fcf280c86f0 0x7fcf280704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:43190/0 (socket says 10.20.193.16:43190) 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf1ffff640 1 -- 10.20.193.16:0/3855328698 learned_addr learned my addr 10.20.193.16:0/3855328698 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf1ffff640 10 monclient(hunting): get_auth_request con 0x7fcf280c86f0 auth_method 0 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf1ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf1ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.564+0000 7fcf1ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.565+0000 7fcf1ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.565+0000 7fcf1ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:53.570 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.565+0000 7fcf1ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.565+0000 7fcf2d0bf640 1 --2- 10.20.193.16:0/3855328698 >> v2:10.20.193.148:3300/0 conn(0x7fcf281536f0 0x7fcf28153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.565+0000 7fcf2d0bf640 10 monclient(hunting): get_auth_request con 0x7fcf281536f0 auth_method 0 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.565+0000 7fcf2d0bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.565+0000 7fcf2d0bf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.565+0000 7fcf2d0bf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.566+0000 7fcf1ffff640 10 monclient(hunting): handle_auth_done global_id 4827 payload 274 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.566+0000 7fcf1ffff640 1 -- 10.20.193.16:0/3855328698 >> v2:10.20.193.148:3300/0 conn(0x7fcf281536f0 msgr2=0x7fcf28153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.566+0000 7fcf1ffff640 1 --2- 10.20.193.16:0/3855328698 >> v2:10.20.193.148:3300/0 conn(0x7fcf281536f0 0x7fcf28153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.566+0000 7fcf1ffff640 1 -- 10.20.193.16:0/3855328698 >> v1:10.20.193.148:6789/0 conn(0x7fcf280700f0 legacy=0x7fcf280c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.566+0000 7fcf1ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:53.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.566+0000 7fcf1ffff640 1 monclient: found mon.noname-a 2026-02-17T05:59:53.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.566+0000 7fcf1ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:53.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.566+0000 7fcf1ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:53.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.566+0000 7fcf1ffff640 1 -- 10.20.193.16:0/3855328698 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf280c50c0 con 0x7fcf280c86f0 2026-02-17T05:59:53.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.566+0000 7fcf1ffff640 1 --2- 10.20.193.16:0/3855328698 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280c86f0 0x7fcf280704d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcf280c9540 tx=0x7fcf1002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3252c1b0e69846 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:53.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 1 -- 10.20.193.16:0/3855328698 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1003c070 con 0x7fcf280c86f0 2026-02-17T05:59:53.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 10 monclient: dump: 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:53.579 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 1 -- 10.20.193.16:0/3855328698 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf1002fb30 con 0x7fcf280c86f0 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf2eb49640 5 monclient: authenticate success, global_id 4827 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 1 -- 10.20.193.16:0/3855328698 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10037440 con 0x7fcf280c86f0 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf2eb49640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf2eb49640 4 set_mon_vals no callback set 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:53.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.568+0000 7fcf1f7fe640 10 monclient: dump: 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:53.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 10 monclient: shutdown 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 -- 10.20.193.16:0/3855328698 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280c86f0 msgr2=0x7fcf280704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 --2- 10.20.193.16:0/3855328698 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280c86f0 0x7fcf280704d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcf280c9540 tx=0x7fcf1002ee60 comp rx=0 tx=0).stop 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 -- 10.20.193.16:0/3855328698 shutdown_connections 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 --2- 10.20.193.16:0/3855328698 >> v2:10.20.193.148:3300/0 conn(0x7fcf281536f0 0x7fcf28153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 --2- 10.20.193.16:0/3855328698 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280c86f0 0x7fcf280704d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 -- 10.20.193.16:0/3855328698 >> 10.20.193.16:0/3855328698 conn(0x7fcf280cd770 msgr2=0x7fcf280cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 -- 10.20.193.16:0/3855328698 shutdown_connections 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 -- 10.20.193.16:0/3855328698 wait complete. 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 10 monclient: build_initial_monmap 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 10 monclient: monmap: 2026-02-17T05:59:53.582 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 librados: starting msgr at 2026-02-17T05:59:53.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 librados: starting objecter 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.569+0000 7fcf2eb49640 1 Processor -- start 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 1 -- start start 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 1 librados: setting wanted keys 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 1 librados: calling monclient init 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient: init 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280700f0 0x7fcf281f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient(hunting): picked mon.noname-a con 0x7fcf280700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcf280c86f0 0x7fcf28139b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient(hunting): picked mon.noname-c con 0x7fcf280c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:53.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcf281536f0 0x7fcf281e7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient(hunting): picked mon.noname-b con 0x7fcf281536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf28154060 con 0x7fcf280700f0 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fcf280c5680 con 0x7fcf281536f0 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcf28074c00 con 0x7fcf280c86f0 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2eb49640 10 monclient(hunting): authenticate will time out at 1246.139407s 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2c8be640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280700f0 0x7fcf281f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2c8be640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280700f0 0x7fcf281f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:43194/0 (socket says 10.20.193.16:43194) 2026-02-17T05:59:53.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2c8be640 1 -- 10.20.193.16:0/4241721256 learned_addr learned my addr 10.20.193.16:0/4241721256 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2c8be640 10 monclient(hunting): get_auth_request con 0x7fcf280700f0 auth_method 0 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2c8be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2c8be640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2c8be640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf1ffff640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcf280c86f0 0x7fcf28139b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2d0bf640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcf281536f0 0x7fcf281e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2c8be640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2c8be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.570+0000 7fcf2c8be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf1ffff640 10 monclient(hunting): get_auth_request con 0x7fcf280c86f0 auth_method 0 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf1ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:53.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf1ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf1ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf2d0bf640 10 monclient(hunting): get_auth_request con 0x7fcf281536f0 auth_method 0 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf2d0bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf2d0bf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf2d0bf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf1ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf1ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf1ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf2d0bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf2d0bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.571+0000 7fcf2d0bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.572+0000 7fcf2c8be640 10 monclient(hunting): handle_auth_done global_id 4830 payload 824 2026-02-17T05:59:53.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.572+0000 7fcf2c8be640 1 -- 10.20.193.16:0/4241721256 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcf280c86f0 msgr2=0x7fcf28139b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.572+0000 7fcf2c8be640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcf280c86f0 0x7fcf28139b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.572+0000 7fcf2c8be640 1 -- 10.20.193.16:0/4241721256 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcf281536f0 msgr2=0x7fcf281e7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.572+0000 7fcf2c8be640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcf281536f0 0x7fcf281e7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.572+0000 7fcf2c8be640 10 monclient: _finish_hunting 0 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.572+0000 7fcf2c8be640 1 monclient: found mon.noname-a 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.572+0000 7fcf2c8be640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.572+0000 7fcf2c8be640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.572+0000 7fcf2c8be640 1 -- 10.20.193.16:0/4241721256 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf28074220 con 0x7fcf280700f0 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.573+0000 7fcf2c8be640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280700f0 0x7fcf281f21c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800c480 tx=0x7fcf1800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.573+0000 7fcf1d7fa640 10 client.?.objecter ms_handle_connect 0x7fcf280700f0 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.573+0000 7fcf1d7fa640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 1 -- 10.20.193.16:0/4241721256 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18017070 con 0x7fcf280700f0 2026-02-17T05:59:53.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 10 monclient: dump: 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:53.589 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 10 monclient: _finish_auth 0 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 10 monclient: _check_auth_tickets 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 1 -- 10.20.193.16:0/4241721256 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf180052f0 con 0x7fcf280700f0 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 1 -- 10.20.193.16:0/4241721256 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf180036f0 con 0x7fcf280700f0 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1effd640 4 set_mon_vals no callback set 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf1d7fa640 10 monclient: dump: 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:53.590 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf2eb49640 5 monclient: authenticate success, global_id 4830 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf2eb49640 10 monclient: _renew_subs 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf2eb49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf2eb49640 1 -- 10.20.193.16:0/4241721256 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf28074500 con 0x7fcf280700f0 2026-02-17T05:59:53.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf2eb49640 10 client.4830.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:53.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf2eb49640 10 monclient: _renew_subs 2026-02-17T05:59:53.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf2eb49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:53.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf2eb49640 1 -- 10.20.193.16:0/4241721256 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf281e6e50 con 0x7fcf280700f0 2026-02-17T05:59:53.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.574+0000 7fcf2eb49640 1 librados: init done 2026-02-17T05:59:53.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.575+0000 7fcf2eb49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:53.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.575+0000 7fcf2eb49640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:53.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.575+0000 7fcf2eb49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:53.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.575+0000 7fcf2eb49640 1 -- 10.20.193.16:0/4241721256 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf280704d0 con 0x7fcf280700f0 2026-02-17T05:59:53.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.587+0000 7fcf1d7fa640 1 -- 10.20.193.16:0/4241721256 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fcf18003890 con 0x7fcf280700f0 2026-02-17T05:59:53.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.587+0000 7fcf1d7fa640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcef8079190 0x7fcef807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:53.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.587+0000 7fcf1ffff640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcef8079190 0x7fcef807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:53.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.587+0000 7fcf1ffff640 10 monclient: get_auth_request con 0x7fcef8079190 auth_method 0 2026-02-17T05:59:53.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.588+0000 7fcf1d7fa640 10 client.4830.objecter ms_handle_connect 0x7fcef8079190 2026-02-17T05:59:53.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.588+0000 7fcf1ffff640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcef8079190 0x7fcef807b650 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf28070b90 tx=0x7fcf10039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:53.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.693+0000 7fcf1d7fa640 1 -- 10.20.193.16:0/4241721256 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fcf1805d560 con 0x7fcf280700f0 2026-02-17T05:59:53.694 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.693+0000 7fcf1d7fa640 10 client.4830.objecter ms_dispatch2 0x7fcf28000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:53.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.693+0000 7fcf1d7fa640 3 client.4830.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:53.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.693+0000 7fcf1d7fa640 3 client.4830.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:53.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.693+0000 7fcf1d7fa640 20 client.4830.objecter dump_active .. 0 homeless 2026-02-17T05:59:53.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.693+0000 7fcf1d7fa640 1 -- 10.20.193.16:0/4241721256 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcf18021020 con 0x7fcf280700f0 2026-02-17T05:59:53.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.693+0000 7fcf1d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:53.695 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.693+0000 7fcf1d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:53.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.794+0000 7fcf2eb49640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:53.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.794+0000 7fcf2eb49640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:53.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.794+0000 7fcf2eb49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:53.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.794+0000 7fcf2eb49640 1 -- 10.20.193.16:0/4241721256 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcf2814f610 con 0x7fcf280700f0 2026-02-17T05:59:53.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.796+0000 7fcf1d7fa640 1 -- 10.20.193.16:0/4241721256 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fcf18061520 con 0x7fcf280700f0 2026-02-17T05:59:53.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.796+0000 7fcf1d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:53.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.796+0000 7fcf1d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:53.797 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:53.797 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:53.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.798+0000 7fcf2eb49640 10 librados: watch_flush enter 2026-02-17T05:59:53.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 10 librados: watch_flush exit 2026-02-17T05:59:53.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 20 client.4830.objecter shutdown clearing up homeless session... 2026-02-17T05:59:53.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 10 client.4830.objecter successfully canceled tick 2026-02-17T05:59:53.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 -- 10.20.193.16:0/4241721256 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcef8079190 msgr2=0x7fcef807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:53.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcef8079190 0x7fcef807b650 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf28070b90 tx=0x7fcf10039040 comp rx=0 tx=0).stop 2026-02-17T05:59:53.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 10 monclient: shutdown 2026-02-17T05:59:53.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:53.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 -- 10.20.193.16:0/4241721256 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280700f0 msgr2=0x7fcf281f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:53.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280700f0 0x7fcf281f21c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800c480 tx=0x7fcf1800c950 comp rx=0 tx=0).stop 2026-02-17T05:59:53.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 -- 10.20.193.16:0/4241721256 shutdown_connections 2026-02-17T05:59:53.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcef8079190 0x7fcef807b650 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:53.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcf281536f0 0x7fcf281e7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:53.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcf280c86f0 0x7fcf28139b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:53.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 --2- 10.20.193.16:0/4241721256 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcf280700f0 0x7fcf281f21c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:53.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 -- 10.20.193.16:0/4241721256 >> 10.20.193.16:0/4241721256 conn(0x7fcf280cd770 msgr2=0x7fcf281e8290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:53.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 -- 10.20.193.16:0/4241721256 shutdown_connections 2026-02-17T05:59:53.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 -- 10.20.193.16:0/4241721256 wait complete. 2026-02-17T05:59:53.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:53.799+0000 7fcf2eb49640 1 librados: shutdown 2026-02-17T05:59:56.809 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:59:56.881 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.880+0000 7f4c0dfb6640 10 monclient: get_monmap_and_config 2026-02-17T05:59:56.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.880+0000 7f4c0dfb6640 10 monclient: build_initial_monmap 2026-02-17T05:59:56.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.880+0000 7f4c0dfb6640 10 monclient: monmap: 2026-02-17T05:59:56.882 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:56.882 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:56.882 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:59:56.881729+0000 2026-02-17T05:59:56.882 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:59:56.881729+0000 2026-02-17T05:59:56.882 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:56.882 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:56.882 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T05:59:56.882 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T05:59:56.883 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T05:59:56.883 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T05:59:56.883 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T05:59:56.883 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T05:59:56.883 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:56.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 1 Processor -- start 2026-02-17T05:59:56.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 1 -- start start 2026-02-17T05:59:56.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 10 monclient: init 2026-02-17T05:59:56.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:56.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 10 monclient: _add_conns ranks=[5,3,1,0,2,4] 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f4c080700f0 0x7f4c080704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 10 monclient(hunting): picked mon.noname-c con 0x7f4c080700f0 addr v2:10.20.193.148:3300/0 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f4c080c7fe0 0x7f4c080c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 10 monclient(hunting): picked mon.noname-a con 0x7f4c080c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f4c081536f0 addr v1:10.20.193.128:6789/0 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c080c8970 con 0x7f4c081536f0 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4c080c4820 con 0x7f4c080c7fe0 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f4c080c4b00 con 0x7f4c080700f0 2026-02-17T05:59:56.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.881+0000 7f4c0dfb6640 10 monclient(hunting): authenticate will time out at 1249.450803s 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c06ffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f4c080c7fe0 0x7f4c080c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c077fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f4c080700f0 0x7f4c080704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c07fff640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f4c081536f0 0x7f4c08153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:41558/0 (socket says 10.20.193.16:41558) 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c07fff640 1 -- 10.20.193.16:0/961442106 learned_addr learned my addr 10.20.193.16:0/961442106 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c06ffd640 10 monclient(hunting): get_auth_request con 0x7f4c080c7fe0 auth_method 0 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c06ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c06ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c06ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c077fe640 10 monclient(hunting): get_auth_request con 0x7f4c080700f0 auth_method 0 2026-02-17T05:59:56.888 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c077fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c077fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c06ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c06ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c06ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.882+0000 7f4c077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c077fe640 10 monclient(hunting): handle_auth_done global_id 4958 payload 274 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c077fe640 1 -- 10.20.193.16:0/961442106 >> v2:10.20.193.16:3300/0 conn(0x7f4c080c7fe0 msgr2=0x7f4c080c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c077fe640 1 --2- 10.20.193.16:0/961442106 >> v2:10.20.193.16:3300/0 conn(0x7f4c080c7fe0 0x7f4c080c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c077fe640 1 -- 10.20.193.16:0/961442106 >> v1:10.20.193.128:6789/0 conn(0x7f4c081536f0 legacy=0x7f4c08153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:56.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c077fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:56.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c077fe640 1 monclient: found mon.noname-c 2026-02-17T05:59:56.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:56.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c077fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T05:59:56.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c077fe640 1 -- 10.20.193.16:0/961442106 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c080c4de0 con 0x7f4c080700f0 2026-02-17T05:59:56.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c067fc640 1 -- 10.20.193.16:0/961442106 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 822262641 0 0) 0x7f4c080c8970 con 0x7f4c081536f0 2026-02-17T05:59:56.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.883+0000 7f4c067fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T05:59:56.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.884+0000 7f4c077fe640 1 --2- 10.20.193.16:0/961442106 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c080700f0 0x7f4c080704d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4c08070b90 tx=0x7f4bf402ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aaa7c4e72e47732 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:56.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 1 -- 10.20.193.16:0/961442106 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf403c070 con 0x7f4c080700f0 2026-02-17T05:59:56.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:56.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:56.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 10 monclient: dump: 2026-02-17T05:59:56.897 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:56.897 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:56.897 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:56.897 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:56.897 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:56.897 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:56.897 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 1 -- 10.20.193.16:0/961442106 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf402fa60 con 0x7f4c080700f0 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c0dfb6640 5 monclient: authenticate success, global_id 4958 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c0dfb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:56.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 1 -- 10.20.193.16:0/961442106 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf402fe30 con 0x7f4c080700f0 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c0dfb6640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c0dfb6640 4 set_mon_vals no callback set 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c067fc640 10 monclient: dump: 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:56.899 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c0dfb6640 10 monclient: shutdown 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c0dfb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/961442106 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c080700f0 msgr2=0x7f4c080704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.886+0000 7f4c0dfb6640 1 --2- 10.20.193.16:0/961442106 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c080700f0 0x7f4c080704d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4c08070b90 tx=0x7f4bf402ed50 comp rx=0 tx=0).stop 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/961442106 shutdown_connections 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 --2- 10.20.193.16:0/961442106 >> v2:10.20.193.16:3300/0 conn(0x7f4c080c7fe0 0x7f4c080c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 --2- 10.20.193.16:0/961442106 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c080700f0 0x7f4c080704d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/961442106 >> 10.20.193.16:0/961442106 conn(0x7f4c080cd770 msgr2=0x7f4c080cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/961442106 shutdown_connections 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/961442106 wait complete. 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 10 monclient: build_initial_monmap 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 10 monclient: monmap: 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T05:59:56.901 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 librados: starting msgr at 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 librados: starting objecter 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 Processor -- start 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 -- start start 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 librados: setting wanted keys 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 1 librados: calling monclient init 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.887+0000 7f4c0dfb6640 10 monclient: init 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c080700f0 0x7f4c081f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:56.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 10 monclient(hunting): picked mon.noname-b con 0x7f4c080700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c080c7fe0 0x7f4c08148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 10 monclient(hunting): picked mon.noname-c con 0x7f4c080c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c081536f0 0x7f4c081f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 10 monclient(hunting): picked mon.noname-a con 0x7f4c081536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c080c5680 con 0x7f4c081536f0 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f4c080c53a0 con 0x7f4c080700f0 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08074c00 con 0x7f4c080c7fe0 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c0dfb6640 10 monclient(hunting): authenticate will time out at 1249.457192s 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c07fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c081536f0 0x7f4c081f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c06ffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c080c7fe0 0x7f4c08148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c077fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c080700f0 0x7f4c081f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c07fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c081536f0 0x7f4c081f6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:43206/0 (socket says 10.20.193.16:43206) 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c07fff640 1 -- 10.20.193.16:0/4170244580 learned_addr learned my addr 10.20.193.16:0/4170244580 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c07fff640 10 monclient(hunting): get_auth_request con 0x7f4c081536f0 auth_method 0 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c07fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c07fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c06ffd640 10 monclient(hunting): get_auth_request con 0x7f4c080c7fe0 auth_method 0 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c06ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c06ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c06ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c077fe640 10 monclient(hunting): get_auth_request con 0x7f4c080700f0 auth_method 0 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c077fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c077fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c06ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c06ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c06ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:56.905 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.888+0000 7f4c077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.890+0000 7f4c07fff640 10 monclient(hunting): handle_auth_done global_id 4836 payload 824 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.890+0000 7f4c07fff640 1 -- 10.20.193.16:0/4170244580 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c080c7fe0 msgr2=0x7f4c08148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.890+0000 7f4c07fff640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c080c7fe0 0x7f4c08148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.890+0000 7f4c07fff640 1 -- 10.20.193.16:0/4170244580 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c080700f0 msgr2=0x7f4c081f4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.890+0000 7f4c07fff640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c080700f0 0x7f4c081f4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.890+0000 7f4c07fff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.890+0000 7f4c07fff640 1 monclient: found mon.noname-a 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.890+0000 7f4c07fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.890+0000 7f4c07fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T05:59:56.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.890+0000 7f4c07fff640 1 -- 10.20.193.16:0/4170244580 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0806d270 con 0x7f4c081536f0 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.891+0000 7f4c07fff640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c081536f0 0x7f4c081f6ef0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00cc50 tx=0x7f4bfc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.891+0000 7f4be3fff640 10 client.?.objecter ms_handle_connect 0x7f4c081536f0 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.891+0000 7f4be3fff640 10 client.?.objecter resend_mon_ops 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 1 -- 10.20.193.16:0/4170244580 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc018070 con 0x7f4c081536f0 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 10 monclient: dump: 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:56.907 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 1 -- 10.20.193.16:0/4170244580 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bfc0070d0 con 0x7f4c081536f0 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 1 -- 10.20.193.16:0/4170244580 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc007540 con 0x7f4c081536f0 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4c0dfb6640 5 monclient: authenticate success, global_id 4836 2026-02-17T05:59:56.908 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4c0dfb6640 10 monclient: _renew_subs 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4c05ffb640 4 set_mon_vals no callback set 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4c0dfb6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/4170244580 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0806d550 con 0x7f4c081536f0 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.892+0000 7f4be3fff640 10 monclient: dump: 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:56.909 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.893+0000 7f4c0dfb6640 10 client.4836.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.893+0000 7f4c0dfb6640 10 monclient: _renew_subs 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.893+0000 7f4c0dfb6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.893+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/4170244580 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0806db10 con 0x7f4c081536f0 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.893+0000 7f4c0dfb6640 1 librados: init done 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.893+0000 7f4c0dfb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.893+0000 7f4c0dfb6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:56.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.893+0000 7f4c0dfb6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:56.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.893+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/4170244580 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c080704d0 con 0x7f4c081536f0 2026-02-17T05:59:56.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.905+0000 7f4be3fff640 1 -- 10.20.193.16:0/4170244580 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f4bfc0076e0 con 0x7f4c081536f0 2026-02-17T05:59:56.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.905+0000 7f4be3fff640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4bd8079140 0x7f4bd807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:56.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.905+0000 7f4c077fe640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4bd8079140 0x7f4bd807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:56.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.905+0000 7f4c077fe640 10 monclient: get_auth_request con 0x7f4bd8079140 auth_method 0 2026-02-17T05:59:56.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.906+0000 7f4c077fe640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4bd8079140 0x7f4bd807b600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4039000 tx=0x7f4bf4033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:56.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.906+0000 7f4be3fff640 10 client.4836.objecter ms_handle_connect 0x7f4bd8079140 2026-02-17T05:59:56.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.992+0000 7f4be3fff640 1 -- 10.20.193.16:0/4170244580 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f4bfc007a00 con 0x7f4c081536f0 2026-02-17T05:59:56.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.992+0000 7f4be3fff640 10 client.4836.objecter ms_dispatch2 0x7f4c08000b90 osd_map(34..34 src has 1..34) 2026-02-17T05:59:56.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.992+0000 7f4be3fff640 3 client.4836.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T05:59:56.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.992+0000 7f4be3fff640 3 client.4836.objecter handle_osd_map decoding full epoch 34 2026-02-17T05:59:56.993 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:56.992+0000 7f4be3fff640 20 client.4836.objecter dump_active .. 0 homeless 2026-02-17T05:59:57.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.009+0000 7f4be3fff640 1 -- 10.20.193.16:0/4170244580 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4bfc062040 con 0x7f4c081536f0 2026-02-17T05:59:57.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.009+0000 7f4be3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:57.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.009+0000 7f4be3fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:57.112 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.112+0000 7f4c0dfb6640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:59:57.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.112+0000 7f4c0dfb6640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:57.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.112+0000 7f4c0dfb6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T05:59:57.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.112+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/4170244580 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4c080744b0 con 0x7f4c081536f0 2026-02-17T05:59:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.114+0000 7f4be3fff640 1 -- 10.20.193.16:0/4170244580 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f4bfc066340 con 0x7f4c081536f0 2026-02-17T05:59:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.114+0000 7f4be3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:59:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.114+0000 7f4be3fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:57.115 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T05:59:57.115 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T05:59:57.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 10 librados: watch_flush enter 2026-02-17T05:59:57.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 10 librados: watch_flush exit 2026-02-17T05:59:57.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 20 client.4836.objecter shutdown clearing up homeless session... 2026-02-17T05:59:57.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 10 client.4836.objecter successfully canceled tick 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/4170244580 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4bd8079140 msgr2=0x7f4bd807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4bd8079140 0x7f4bd807b600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4039000 tx=0x7f4bf4033000 comp rx=0 tx=0).stop 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 10 monclient: shutdown 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/4170244580 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c081536f0 msgr2=0x7f4c081f6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c081536f0 0x7f4c081f6ef0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00cc50 tx=0x7f4bfc00a430 comp rx=0 tx=0).stop 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/4170244580 shutdown_connections 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4bd8079140 0x7f4bd807b600 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4c081536f0 0x7f4c081f6ef0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4c080c7fe0 0x7f4c08148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 --2- 10.20.193.16:0/4170244580 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4c080700f0 0x7f4c081f4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/4170244580 >> 10.20.193.16:0/4170244580 conn(0x7f4c080cd770 msgr2=0x7f4c0814e430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/4170244580 shutdown_connections 2026-02-17T05:59:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 -- 10.20.193.16:0/4170244580 wait complete. 2026-02-17T05:59:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T05:59:57.115+0000 7f4c0dfb6640 1 librados: shutdown 2026-02-17T06:00:00.125 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:00.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.196+0000 7fb70005b640 10 monclient: get_monmap_and_config 2026-02-17T06:00:00.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.196+0000 7fb70005b640 10 monclient: build_initial_monmap 2026-02-17T06:00:00.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.196+0000 7fb70005b640 10 monclient: monmap: 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:00.197542+0000 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:00.197542+0000 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:00.198 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:00.199 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:00.199 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:00.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.196+0000 7fb70005b640 1 Processor -- start 2026-02-17T06:00:00.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.196+0000 7fb70005b640 1 -- start start 2026-02-17T06:00:00.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.196+0000 7fb70005b640 10 monclient: init 2026-02-17T06:00:00.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 10 monclient: _add_conns ranks=[0,1,5,3,2,4] 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6f80700f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb6f80c86f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fb6f8153e00 0x7fb6f80704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 10 monclient(hunting): picked mon.noname-c con 0x7fb6f8153e00 addr v2:10.20.193.148:3300/0 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f80c4820 con 0x7fb6f80700f0 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f80c4b00 con 0x7fb6f80c86f0 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f80c4de0 con 0x7fb6f8153e00 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb70005b640 10 monclient(hunting): authenticate will time out at 1252.766497s 2026-02-17T06:00:00.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb6fddd0640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fb6f80700f0 0x7fb6f80c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:37640/0 (socket says 10.20.193.16:37640) 2026-02-17T06:00:00.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.197+0000 7fb6fddd0640 1 -- 10.20.193.16:0/870858243 learned_addr learned my addr 10.20.193.16:0/870858243 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:00.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.198+0000 7fb6fe5d1640 1 --2- 10.20.193.16:0/870858243 >> v2:10.20.193.148:3300/0 conn(0x7fb6f8153e00 0x7fb6f80704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:00.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.198+0000 7fb6fe5d1640 10 monclient(hunting): get_auth_request con 0x7fb6f8153e00 auth_method 0 2026-02-17T06:00:00.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.198+0000 7fb6fe5d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:00.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.198+0000 7fb6fe5d1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:00.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.198+0000 7fb6fe5d1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:00.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.198+0000 7fb6fe5d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:00.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.198+0000 7fb6fe5d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:00.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.198+0000 7fb6fe5d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:00.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 1 -- 10.20.193.16:0/870858243 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 212057623 0 0) 0x7fb6f80c4820 con 0x7fb6f80700f0 2026-02-17T06:00:00.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 10 monclient(hunting): _negotiate 2026-02-17T06:00:00.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:00.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:00.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 10 monclient(hunting): my global_id is 4839 2026-02-17T06:00:00.204 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 1 -- 10.20.193.16:0/870858243 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6e0003620 con 0x7fb6f80700f0 2026-02-17T06:00:00.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 1 -- 10.20.193.16:0/870858243 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939095476 0 0) 0x7fb6f80c4b00 con 0x7fb6f80c86f0 2026-02-17T06:00:00.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 10 monclient(hunting): _negotiate 2026-02-17T06:00:00.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:00.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:00.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 10 monclient(hunting): my global_id is 4909 2026-02-17T06:00:00.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fcdce640 1 -- 10.20.193.16:0/870858243 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6f80c4820 con 0x7fb6f80c86f0 2026-02-17T06:00:00.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fe5d1640 10 monclient(hunting): handle_auth_done global_id 4964 payload 274 2026-02-17T06:00:00.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fe5d1640 1 -- 10.20.193.16:0/870858243 >> v1:10.20.193.128:6789/0 conn(0x7fb6f80c86f0 legacy=0x7fb6f81536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:00.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fe5d1640 1 -- 10.20.193.16:0/870858243 >> v1:10.20.193.16:6789/0 conn(0x7fb6f80700f0 legacy=0x7fb6f80c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:00.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fe5d1640 10 monclient: _finish_hunting 0 2026-02-17T06:00:00.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fe5d1640 1 monclient: found mon.noname-c 2026-02-17T06:00:00.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fe5d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:00.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fe5d1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:00:00.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.199+0000 7fb6fe5d1640 1 -- 10.20.193.16:0/870858243 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f80c50c0 con 0x7fb6f8153e00 2026-02-17T06:00:00.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.200+0000 7fb6fe5d1640 1 --2- 10.20.193.16:0/870858243 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb6f8153e00 0x7fb6f80704d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009d90 tx=0x7fb6f402f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea01fba0a859a010 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:00.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.201+0000 7fb6fcdce640 1 -- 10.20.193.16:0/870858243 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f403c070 con 0x7fb6f8153e00 2026-02-17T06:00:00.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.201+0000 7fb6fcdce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:00.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.201+0000 7fb6fcdce640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.201+0000 7fb6fcdce640 10 monclient: dump: 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:00.216 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.201+0000 7fb6fcdce640 10 monclient: _finish_auth 0 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.201+0000 7fb6fcdce640 10 monclient: _check_auth_tickets 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.201+0000 7fb6fcdce640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.201+0000 7fb70005b640 5 monclient: authenticate success, global_id 4964 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.201+0000 7fb70005b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.202+0000 7fb6fcdce640 1 -- 10.20.193.16:0/870858243 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f4037440 con 0x7fb6f8153e00 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.202+0000 7fb6fcdce640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.202+0000 7fb6fcdce640 1 -- 10.20.193.16:0/870858243 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40377f0 con 0x7fb6f8153e00 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.202+0000 7fb6fcdce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.202+0000 7fb6fcdce640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.202+0000 7fb6fcdce640 10 monclient: dump: 2026-02-17T06:00:00.217 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.202+0000 7fb70005b640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.202+0000 7fb70005b640 4 set_mon_vals no callback set 2026-02-17T06:00:00.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 10 monclient: shutdown 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 -- 10.20.193.16:0/870858243 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb6f8153e00 msgr2=0x7fb6f80704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 --2- 10.20.193.16:0/870858243 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb6f8153e00 0x7fb6f80704d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009d90 tx=0x7fb6f402f1e0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 -- 10.20.193.16:0/870858243 shutdown_connections 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 --2- 10.20.193.16:0/870858243 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb6f8153e00 0x7fb6f80704d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 -- 10.20.193.16:0/870858243 >> 10.20.193.16:0/870858243 conn(0x7fb6f80cd770 msgr2=0x7fb6f80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 -- 10.20.193.16:0/870858243 shutdown_connections 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 -- 10.20.193.16:0/870858243 wait complete. 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 10 monclient: build_initial_monmap 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 10 monclient: monmap: 2026-02-17T06:00:00.219 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 librados: starting msgr at 2026-02-17T06:00:00.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 librados: starting objecter 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 Processor -- start 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 -- start start 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 librados: setting wanted keys 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 1 librados: calling monclient init 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.203+0000 7fb70005b640 10 monclient: init 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb6f80700f0 0x7fb6f8142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 10 monclient(hunting): picked mon.noname-b con 0x7fb6f80700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb6f80c86f0 0x7fb6f8142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 10 monclient(hunting): picked mon.noname-c con 0x7fb6f80c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:00.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb6f8153e00 0x7fb6f81f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 10 monclient(hunting): picked mon.noname-a con 0x7fb6f8153e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8074c00 con 0x7fb6f8153e00 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f80c5680 con 0x7fb6f80700f0 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8074d80 con 0x7fb6f80c86f0 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb70005b640 10 monclient(hunting): authenticate will time out at 1252.773185s 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fe5d1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb6f8153e00 0x7fb6f81f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fd5cf640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb6f80c86f0 0x7fb6f8142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:00.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fe5d1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb6f8153e00 0x7fb6f81f33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:38146/0 (socket says 10.20.193.16:38146) 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fe5d1640 1 -- 10.20.193.16:0/2952256311 learned_addr learned my addr 10.20.193.16:0/2952256311 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fe5d1640 10 monclient(hunting): get_auth_request con 0x7fb6f8153e00 auth_method 0 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fe5d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fe5d1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fe5d1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fddd0640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb6f80700f0 0x7fb6f8142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fd5cf640 10 monclient(hunting): get_auth_request con 0x7fb6f80c86f0 auth_method 0 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fd5cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fd5cf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fd5cf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fddd0640 10 monclient(hunting): get_auth_request con 0x7fb6f80700f0 auth_method 0 2026-02-17T06:00:00.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fddd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fddd0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fddd0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fe5d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fe5d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.204+0000 7fb6fe5d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.205+0000 7fb6fd5cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.205+0000 7fb6fd5cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.205+0000 7fb6fd5cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.205+0000 7fb6fddd0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.205+0000 7fb6fddd0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.205+0000 7fb6fddd0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:00.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.206+0000 7fb6fe5d1640 10 monclient(hunting): handle_auth_done global_id 4842 payload 824 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.206+0000 7fb6fe5d1640 1 -- 10.20.193.16:0/2952256311 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb6f80c86f0 msgr2=0x7fb6f8142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.206+0000 7fb6fe5d1640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb6f80c86f0 0x7fb6f8142060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.206+0000 7fb6fe5d1640 1 -- 10.20.193.16:0/2952256311 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb6f80700f0 msgr2=0x7fb6f8142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.206+0000 7fb6fe5d1640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb6f80700f0 0x7fb6f8142d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.206+0000 7fb6fe5d1640 10 monclient: _finish_hunting 0 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.206+0000 7fb6fe5d1640 1 monclient: found mon.noname-a 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.206+0000 7fb6fe5d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.206+0000 7fb6fe5d1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.206+0000 7fb6fe5d1640 1 -- 10.20.193.16:0/2952256311 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f8151f90 con 0x7fb6f8153e00 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.207+0000 7fb6fe5d1640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb6f8153e00 0x7fb6f81f33b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6f403a040 tx=0x7fb6f4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.207+0000 7fb6e67fc640 10 client.?.objecter ms_handle_connect 0x7fb6f8153e00 2026-02-17T06:00:00.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.207+0000 7fb6e67fc640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 1 -- 10.20.193.16:0/2952256311 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f403c050 con 0x7fb6f8153e00 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 10 monclient: dump: 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:00.226 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:00.227 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:00.227 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:00.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 10 monclient: _finish_auth 0 2026-02-17T06:00:00.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 10 monclient: _check_auth_tickets 2026-02-17T06:00:00.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:00.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 1 -- 10.20.193.16:0/2952256311 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f4048070 con 0x7fb6f8153e00 2026-02-17T06:00:00.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb70005b640 5 monclient: authenticate success, global_id 4842 2026-02-17T06:00:00.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb70005b640 10 monclient: _renew_subs 2026-02-17T06:00:00.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb70005b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:00.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb70005b640 1 -- 10.20.193.16:0/2952256311 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f8152270 con 0x7fb6f8153e00 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 1 -- 10.20.193.16:0/2952256311 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4004040 con 0x7fb6f8153e00 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e67fc640 10 monclient: dump: 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:00.228 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb6e7fff640 4 set_mon_vals no callback set 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb70005b640 10 client.4842.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb70005b640 10 monclient: _renew_subs 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb70005b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb70005b640 1 -- 10.20.193.16:0/2952256311 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f8141580 con 0x7fb6f8153e00 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.208+0000 7fb70005b640 1 librados: init done 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.209+0000 7fb70005b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.209+0000 7fb70005b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.209+0000 7fb70005b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:00.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.209+0000 7fb70005b640 1 -- 10.20.193.16:0/2952256311 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f80c9050 con 0x7fb6f8153e00 2026-02-17T06:00:00.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.315+0000 7fb6e67fc640 1 -- 10.20.193.16:0/2952256311 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fb6f40041e0 con 0x7fb6f8153e00 2026-02-17T06:00:00.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.316+0000 7fb6e67fc640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb6c8079190 0x7fb6c807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:00.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.316+0000 7fb6fddd0640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb6c8079190 0x7fb6c807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:00.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.316+0000 7fb6fddd0640 10 monclient: get_auth_request con 0x7fb6c8079190 auth_method 0 2026-02-17T06:00:00.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.316+0000 7fb6fddd0640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb6c8079190 0x7fb6c807b650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8007b60 tx=0x7fb6e8007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:00.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.316+0000 7fb6e67fc640 10 client.4842.objecter ms_handle_connect 0x7fb6c8079190 2026-02-17T06:00:00.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.323+0000 7fb6e67fc640 1 -- 10.20.193.16:0/2952256311 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fb6f4084290 con 0x7fb6f8153e00 2026-02-17T06:00:00.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.323+0000 7fb6e67fc640 10 client.4842.objecter ms_dispatch2 0x7fb6f8000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:00.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.323+0000 7fb6e67fc640 3 client.4842.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:00.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.323+0000 7fb6e67fc640 3 client.4842.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:00.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.323+0000 7fb6e67fc640 20 client.4842.objecter dump_active .. 0 homeless 2026-02-17T06:00:00.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.324+0000 7fb6e67fc640 1 -- 10.20.193.16:0/2952256311 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb6f4088250 con 0x7fb6f8153e00 2026-02-17T06:00:00.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.324+0000 7fb6e67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:00.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.324+0000 7fb6e67fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:00.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.424+0000 7fb70005b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:00.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.424+0000 7fb70005b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:00.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.424+0000 7fb70005b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:00.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.424+0000 7fb70005b640 1 -- 10.20.193.16:0/2952256311 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb6f80704d0 con 0x7fb6f8153e00 2026-02-17T06:00:00.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.427+0000 7fb6e67fc640 1 -- 10.20.193.16:0/2952256311 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fb6f408c550 con 0x7fb6f8153e00 2026-02-17T06:00:00.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.427+0000 7fb6e67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:00.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.427+0000 7fb6e67fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:00.428 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:00.428 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:00.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 10 librados: watch_flush enter 2026-02-17T06:00:00.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 10 librados: watch_flush exit 2026-02-17T06:00:00.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 20 client.4842.objecter shutdown clearing up homeless session... 2026-02-17T06:00:00.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 10 client.4842.objecter successfully canceled tick 2026-02-17T06:00:00.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 -- 10.20.193.16:0/2952256311 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb6c8079190 msgr2=0x7fb6c807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb6c8079190 0x7fb6c807b650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8007b60 tx=0x7fb6e8007ab0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 10 monclient: shutdown 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 -- 10.20.193.16:0/2952256311 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb6f8153e00 msgr2=0x7fb6f81f33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb6f8153e00 0x7fb6f81f33b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6f403a040 tx=0x7fb6f4004b40 comp rx=0 tx=0).stop 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 -- 10.20.193.16:0/2952256311 shutdown_connections 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb6c8079190 0x7fb6c807b650 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb6f8153e00 0x7fb6f81f33b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb6f80c86f0 0x7fb6f8142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 --2- 10.20.193.16:0/2952256311 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb6f80700f0 0x7fb6f8142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 -- 10.20.193.16:0/2952256311 >> 10.20.193.16:0/2952256311 conn(0x7fb6f80cd770 msgr2=0x7fb6f814de00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.428+0000 7fb70005b640 1 -- 10.20.193.16:0/2952256311 shutdown_connections 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.429+0000 7fb70005b640 1 -- 10.20.193.16:0/2952256311 wait complete. 2026-02-17T06:00:00.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:00.429+0000 7fb70005b640 1 librados: shutdown 2026-02-17T06:00:03.439 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:03.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.507+0000 7fea549de640 10 monclient: get_monmap_and_config 2026-02-17T06:00:03.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.507+0000 7fea549de640 10 monclient: build_initial_monmap 2026-02-17T06:00:03.509 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.507+0000 7fea549de640 10 monclient: monmap: 2026-02-17T06:00:03.509 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:03.509 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:03.509 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:03.508702+0000 2026-02-17T06:00:03.509 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:03.508702+0000 2026-02-17T06:00:03.509 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:03.509 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:03.509 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:03.509 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:03.510 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:03.510 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:03.510 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:03.510 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:03.510 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:03.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 1 Processor -- start 2026-02-17T06:00:03.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 1 -- start start 2026-02-17T06:00:03.510 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 10 monclient: init 2026-02-17T06:00:03.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:03.512 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 10 monclient: _add_conns ranks=[2,5,1,3,4,0] 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fea4c0700f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fea4c0c86f0 0x7fea4c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 10 monclient(hunting): picked mon.noname-c con 0x7fea4c0c86f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fea4c1536f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea4c0c4820 con 0x7fea4c1536f0 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea4c0c4b00 con 0x7fea4c0700f0 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fea4c0c4de0 con 0x7fea4c0c86f0 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.508+0000 7fea549de640 10 monclient(hunting): authenticate will time out at 1256.077687s 2026-02-17T06:00:03.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.509+0000 7fea52f54640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fea4c1536f0 0x7fea4c153b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:60852/0 (socket says 10.20.193.16:60852) 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.509+0000 7fea52f54640 1 -- 10.20.193.16:0/736131647 learned_addr learned my addr 10.20.193.16:0/736131647 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.509+0000 7fea51f52640 1 --2- 10.20.193.16:0/736131647 >> v2:10.20.193.148:3300/0 conn(0x7fea4c0c86f0 0x7fea4c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.509+0000 7fea51f52640 10 monclient(hunting): get_auth_request con 0x7fea4c0c86f0 auth_method 0 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.509+0000 7fea51f52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.509+0000 7fea51f52640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.509+0000 7fea51f52640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.510+0000 7fea51f52640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.510+0000 7fea51f52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.510+0000 7fea51f52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.510+0000 7fea51751640 1 -- 10.20.193.16:0/736131647 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3478088375 0 0) 0x7fea4c0c4820 con 0x7fea4c1536f0 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.510+0000 7fea51751640 10 monclient(hunting): _negotiate 2026-02-17T06:00:03.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.510+0000 7fea51751640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.510+0000 7fea51751640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.510+0000 7fea51751640 10 monclient(hunting): my global_id is 4915 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.510+0000 7fea51751640 1 -- 10.20.193.16:0/736131647 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea3c003620 con 0x7fea4c1536f0 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.511+0000 7fea51f52640 10 monclient(hunting): handle_auth_done global_id 4970 payload 274 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.511+0000 7fea51f52640 1 -- 10.20.193.16:0/736131647 >> v1:10.20.193.148:6789/0 conn(0x7fea4c0700f0 legacy=0x7fea4c0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.511+0000 7fea51f52640 1 -- 10.20.193.16:0/736131647 >> v1:10.20.193.128:6789/0 conn(0x7fea4c1536f0 legacy=0x7fea4c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.511+0000 7fea51f52640 10 monclient: _finish_hunting 0 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.511+0000 7fea51f52640 1 monclient: found mon.noname-c 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.511+0000 7fea51f52640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.511+0000 7fea51f52640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:00:03.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.511+0000 7fea51f52640 1 -- 10.20.193.16:0/736131647 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c0c50c0 con 0x7fea4c0c86f0 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.512+0000 7fea51f52640 1 --2- 10.20.193.16:0/736131647 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 0x7fea4c0704d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fea380098a0 tx=0x7fea3802edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b74694951871fb41 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 1 -- 10.20.193.16:0/736131647 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3803c070 con 0x7fea4c0c86f0 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 10 monclient: dump: 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:03.521 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 10 monclient: _finish_auth 0 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 10 monclient: _check_auth_tickets 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 5 monclient: authenticate success, global_id 4970 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 1 -- 10.20.193.16:0/736131647 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3802fae0 con 0x7fea4c0c86f0 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:03.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 4 set_mon_vals no callback set 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 1 -- 10.20.193.16:0/736131647 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38037440 con 0x7fea4c0c86f0 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea51751640 10 monclient: dump: 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:03.523 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 10 monclient: shutdown 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 1 -- 10.20.193.16:0/736131647 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 msgr2=0x7fea4c0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 1 --2- 10.20.193.16:0/736131647 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 0x7fea4c0704d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fea380098a0 tx=0x7fea3802edd0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 1 -- 10.20.193.16:0/736131647 shutdown_connections 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 1 --2- 10.20.193.16:0/736131647 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 0x7fea4c0704d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 1 -- 10.20.193.16:0/736131647 >> 10.20.193.16:0/736131647 conn(0x7fea4c0cd770 msgr2=0x7fea4c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 1 -- 10.20.193.16:0/736131647 shutdown_connections 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.515+0000 7fea549de640 1 -- 10.20.193.16:0/736131647 wait complete. 2026-02-17T06:00:03.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.516+0000 7fea549de640 10 monclient: build_initial_monmap 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.516+0000 7fea549de640 10 monclient: monmap: 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:03.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.516+0000 7fea549de640 1 librados: starting msgr at 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.516+0000 7fea549de640 1 librados: starting objecter 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.516+0000 7fea549de640 1 Processor -- start 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.516+0000 7fea549de640 1 -- start start 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.516+0000 7fea549de640 1 librados: setting wanted keys 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.516+0000 7fea549de640 1 librados: calling monclient init 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.516+0000 7fea549de640 10 monclient: init 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.516+0000 7fea549de640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea4c0700f0 0x7fea4c13fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 10 monclient(hunting): picked mon.noname-a con 0x7fea4c0700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 0x7fea4c1f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 10 monclient(hunting): picked mon.noname-c con 0x7fea4c0c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:03.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea4c1536f0 0x7fea4c145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 10 monclient(hunting): picked mon.noname-b con 0x7fea4c1536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c074c00 con 0x7fea4c0700f0 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c0c5680 con 0x7fea4c1536f0 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c074d80 con 0x7fea4c0c86f0 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea549de640 10 monclient(hunting): authenticate will time out at 1256.086068s 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea51f52640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 0x7fea4c1f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea52753640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea4c0700f0 0x7fea4c13fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:03.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea52f54640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea4c1536f0 0x7fea4c145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:03.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea51f52640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 0x7fea4c1f7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:42468/0 (socket says 10.20.193.16:42468) 2026-02-17T06:00:03.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea51f52640 1 -- 10.20.193.16:0/3240440223 learned_addr learned my addr 10.20.193.16:0/3240440223 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:03.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea51f52640 10 monclient(hunting): get_auth_request con 0x7fea4c0c86f0 auth_method 0 2026-02-17T06:00:03.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea51f52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:03.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea51f52640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:03.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea51f52640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:03.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea52753640 10 monclient(hunting): get_auth_request con 0x7fea4c0700f0 auth_method 0 2026-02-17T06:00:03.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea52753640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:03.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea52753640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:03.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea52753640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea52f54640 10 monclient(hunting): get_auth_request con 0x7fea4c1536f0 auth_method 0 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea52f54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea52f54640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea52f54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea51f52640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea51f52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.517+0000 7fea51f52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.518+0000 7fea52f54640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.518+0000 7fea52f54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.518+0000 7fea52f54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.518+0000 7fea52753640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:03.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.518+0000 7fea52753640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.518+0000 7fea52753640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea51f52640 10 monclient(hunting): handle_auth_done global_id 4973 payload 824 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea51f52640 1 -- 10.20.193.16:0/3240440223 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea4c1536f0 msgr2=0x7fea4c145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea51f52640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea4c1536f0 0x7fea4c145300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea51f52640 1 -- 10.20.193.16:0/3240440223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea4c0700f0 msgr2=0x7fea4c13fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea51f52640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea4c0700f0 0x7fea4c13fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea51f52640 10 monclient: _finish_hunting 0 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea51f52640 1 monclient: found mon.noname-c 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea51f52640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea51f52640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea51f52640 1 -- 10.20.193.16:0/3240440223 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c1f7c60 con 0x7fea4c0c86f0 2026-02-17T06:00:03.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.519+0000 7fea52f54640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea4c1536f0 0x7fea4c145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:00:03.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.520+0000 7fea51f52640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 0x7fea4c1f7290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fea38009130 tx=0x7fea380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:03.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.520+0000 7fea36ffd640 10 client.?.objecter ms_handle_connect 0x7fea4c0c86f0 2026-02-17T06:00:03.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.520+0000 7fea36ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:03.532 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 1 -- 10.20.193.16:0/3240440223 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3803c040 con 0x7fea4c0c86f0 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 10 monclient: dump: 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:03.533 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 5 monclient: authenticate success, global_id 4973 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 1 -- 10.20.193.16:0/3240440223 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea38042630 con 0x7fea4c0c86f0 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 10 monclient: _renew_subs 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 1 -- 10.20.193.16:0/3240440223 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c1f7f40 con 0x7fea4c0c86f0 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:03.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 1 -- 10.20.193.16:0/3240440223 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38042a20 con 0x7fea4c0c86f0 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea36ffd640 10 monclient: dump: 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:03.535 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea50f50640 4 set_mon_vals no callback set 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 10 client.4973.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 10 monclient: _renew_subs 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 1 -- 10.20.193.16:0/3240440223 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c13f340 con 0x7fea4c0c86f0 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 1 librados: init done 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:03.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.522+0000 7fea549de640 1 -- 10.20.193.16:0/3240440223 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4c0704d0 con 0x7fea4c0c86f0 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.533+0000 7fea36ffd640 1 -- 10.20.193.16:0/3240440223 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fea380413b0 con 0x7fea4c0c86f0 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.534+0000 7fea36ffd640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea1c079140 0x7fea1c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.534+0000 7fea52753640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea1c079140 0x7fea1c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.534+0000 7fea52753640 10 monclient: get_auth_request con 0x7fea1c079140 auth_method 0 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.534+0000 7fea52753640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea1c079140 0x7fea1c07b600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fea44002770 tx=0x7fea44007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.534+0000 7fea36ffd640 10 client.4973.objecter ms_handle_connect 0x7fea1c079140 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.535+0000 7fea36ffd640 1 -- 10.20.193.16:0/3240440223 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fea3802fa20 con 0x7fea4c0c86f0 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.535+0000 7fea36ffd640 10 client.4973.objecter ms_dispatch2 0x7fea4c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.535+0000 7fea36ffd640 3 client.4973.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.535+0000 7fea36ffd640 3 client.4973.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:03.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.536+0000 7fea36ffd640 20 client.4973.objecter dump_active .. 0 homeless 2026-02-17T06:00:03.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.639+0000 7fea36ffd640 1 -- 10.20.193.16:0/3240440223 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fea38087230 con 0x7fea4c0c86f0 2026-02-17T06:00:03.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.639+0000 7fea36ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:03.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.639+0000 7fea36ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:03.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.738+0000 7fea549de640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:03.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.738+0000 7fea549de640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:03.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.738+0000 7fea549de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:03.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.738+0000 7fea549de640 1 -- 10.20.193.16:0/3240440223 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fea4c06b6f0 con 0x7fea4c0c86f0 2026-02-17T06:00:03.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.742+0000 7fea36ffd640 1 -- 10.20.193.16:0/3240440223 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fea3808b530 con 0x7fea4c0c86f0 2026-02-17T06:00:03.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.742+0000 7fea36ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:03.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.742+0000 7fea36ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:03.743 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:03.743 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:03.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 10 librados: watch_flush enter 2026-02-17T06:00:03.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 10 librados: watch_flush exit 2026-02-17T06:00:03.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 20 client.4973.objecter shutdown clearing up homeless session... 2026-02-17T06:00:03.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 10 client.4973.objecter successfully canceled tick 2026-02-17T06:00:03.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 -- 10.20.193.16:0/3240440223 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea1c079140 msgr2=0x7fea1c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:03.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea1c079140 0x7fea1c07b600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fea44002770 tx=0x7fea44007ef0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 10 monclient: shutdown 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 -- 10.20.193.16:0/3240440223 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 msgr2=0x7fea4c1f7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 0x7fea4c1f7290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fea38009130 tx=0x7fea380047c0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 -- 10.20.193.16:0/3240440223 shutdown_connections 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fea1c079140 0x7fea1c07b600 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fea4c1536f0 0x7fea4c145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fea4c0c86f0 0x7fea4c1f7290 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 --2- 10.20.193.16:0/3240440223 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fea4c0700f0 0x7fea4c13fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 -- 10.20.193.16:0/3240440223 >> 10.20.193.16:0/3240440223 conn(0x7fea4c0cd770 msgr2=0x7fea4c14dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 -- 10.20.193.16:0/3240440223 shutdown_connections 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 -- 10.20.193.16:0/3240440223 wait complete. 2026-02-17T06:00:03.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:03.743+0000 7fea549de640 1 librados: shutdown 2026-02-17T06:00:04.101 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:05:42.929 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:04.102 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:05:42.930 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:04.111 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:05:42.938 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:04.132 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:05:42.958 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:04.132 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:05:42.921 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:04.133 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:05:42.959 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:04.133 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:05:42.922 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:04.141 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:05:42.967 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:04.141 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:05:42.930 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:06.753 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:06.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.820+0000 7f3ce4d7e640 10 monclient: get_monmap_and_config 2026-02-17T06:00:06.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.820+0000 7f3ce4d7e640 10 monclient: build_initial_monmap 2026-02-17T06:00:06.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.820+0000 7f3ce4d7e640 10 monclient: monmap: 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:06.821356+0000 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:06.821356+0000 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:06.822 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:06.823 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:06.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.820+0000 7f3ce4d7e640 1 Processor -- start 2026-02-17T06:00:06.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.820+0000 7f3ce4d7e640 1 -- start start 2026-02-17T06:00:06.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.820+0000 7f3ce4d7e640 10 monclient: init 2026-02-17T06:00:06.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:06.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 10 monclient: _add_conns ranks=[2,4,1,5,3,0] 2026-02-17T06:00:06.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3ce00700f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:00:06.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f3ce00c86f0 0x7f3ce00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 10 monclient(hunting): picked mon.noname-b con 0x7f3ce00c86f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3ce01536f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ce00c4820 con 0x7f3ce01536f0 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ce00c4b00 con 0x7f3ce00700f0 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f3ce00c4de0 con 0x7f3ce00c86f0 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3ce4d7e640 10 monclient(hunting): authenticate will time out at 1259.390390s 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3cddd74640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f3ce00c86f0 0x7f3ce00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3cde575640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f3ce00700f0 0x7f3ce00c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:54310/0 (socket says 10.20.193.16:54310) 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.821+0000 7f3cde575640 1 -- 10.20.193.16:0/4082371179 learned_addr learned my addr 10.20.193.16:0/4082371179 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:06.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.822+0000 7f3cddd74640 10 monclient(hunting): get_auth_request con 0x7f3ce00c86f0 auth_method 0 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.822+0000 7f3cddd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.822+0000 7f3cddd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.822+0000 7f3cddd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.822+0000 7f3cddd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.822+0000 7f3cddd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.822+0000 7f3cddd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.823+0000 7f3cddd74640 10 monclient(hunting): handle_auth_done global_id 4921 payload 274 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.823+0000 7f3cddd74640 1 -- 10.20.193.16:0/4082371179 >> v1:10.20.193.148:6789/0 conn(0x7f3ce00700f0 legacy=0x7f3ce00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.823+0000 7f3cddd74640 1 -- 10.20.193.16:0/4082371179 >> v1:10.20.193.128:6789/0 conn(0x7f3ce01536f0 legacy=0x7f3ce0153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.823+0000 7f3cddd74640 10 monclient: _finish_hunting 0 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.823+0000 7f3cddd74640 1 monclient: found mon.noname-b 2026-02-17T06:00:06.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.823+0000 7f3cddd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:06.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.823+0000 7f3cddd74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:00:06.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.823+0000 7f3cddd74640 1 -- 10.20.193.16:0/4082371179 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce00c50c0 con 0x7f3ce00c86f0 2026-02-17T06:00:06.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.824+0000 7f3cddd74640 1 --2- 10.20.193.16:0/4082371179 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce00c86f0 0x7f3ce00704d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80098a0 tx=0x7f3cc802ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd16307383386e6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:06.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 1 -- 10.20.193.16:0/4082371179 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc803c070 con 0x7f3ce00c86f0 2026-02-17T06:00:06.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:06.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:06.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 10 monclient: dump: 2026-02-17T06:00:06.833 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:06.833 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:06.833 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:06.833 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:06.833 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:06.833 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:06.833 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 10 monclient: _finish_auth 0 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 10 monclient: _check_auth_tickets 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3ce4d7e640 5 monclient: authenticate success, global_id 4921 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 1 -- 10.20.193.16:0/4082371179 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc802fa40 con 0x7f3ce00c86f0 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3ce4d7e640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3ce4d7e640 4 set_mon_vals no callback set 2026-02-17T06:00:06.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 1 -- 10.20.193.16:0/4082371179 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc802fe10 con 0x7f3ce00c86f0 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3cdd573640 10 monclient: dump: 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:06.835 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3ce4d7e640 10 monclient: shutdown 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3ce4d7e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/4082371179 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce00c86f0 msgr2=0x7f3ce00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.826+0000 7f3ce4d7e640 1 --2- 10.20.193.16:0/4082371179 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce00c86f0 0x7f3ce00704d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80098a0 tx=0x7f3cc802ed80 comp rx=0 tx=0).stop 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/4082371179 shutdown_connections 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 --2- 10.20.193.16:0/4082371179 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce00c86f0 0x7f3ce00704d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/4082371179 >> 10.20.193.16:0/4082371179 conn(0x7f3ce00cd770 msgr2=0x7f3ce00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/4082371179 shutdown_connections 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/4082371179 wait complete. 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 10 monclient: build_initial_monmap 2026-02-17T06:00:06.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 10 monclient: monmap: 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 librados: starting msgr at 2026-02-17T06:00:06.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 librados: starting objecter 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 Processor -- start 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 -- start start 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 librados: setting wanted keys 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 1 librados: calling monclient init 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.827+0000 7f3ce4d7e640 10 monclient: init 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ce00700f0 0x7f3ce006ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 10 monclient(hunting): picked mon.noname-a con 0x7f3ce00700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3ce00c86f0 0x7f3ce01f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 10 monclient(hunting): picked mon.noname-c con 0x7f3ce00c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:06.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce01536f0 0x7f3ce0156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 10 monclient(hunting): picked mon.noname-b con 0x7f3ce01536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0074c00 con 0x7f3ce00700f0 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce00c5680 con 0x7f3ce01536f0 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0074d80 con 0x7f3ce00c86f0 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3ce4d7e640 10 monclient(hunting): authenticate will time out at 1259.397302s 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cde575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ce00700f0 0x7f3ce006ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cded76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce01536f0 0x7f3ce0156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cded76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce01536f0 0x7f3ce0156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:47090/0 (socket says 10.20.193.16:47090) 2026-02-17T06:00:06.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cded76640 1 -- 10.20.193.16:0/1586508110 learned_addr learned my addr 10.20.193.16:0/1586508110 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cded76640 10 monclient(hunting): get_auth_request con 0x7f3ce01536f0 auth_method 0 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cded76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cded76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cded76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cde575640 10 monclient(hunting): get_auth_request con 0x7f3ce00700f0 auth_method 0 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cde575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cde575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cde575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cddd74640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3ce00c86f0 0x7f3ce01f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cded76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:06.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.828+0000 7f3cded76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cded76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cddd74640 10 monclient(hunting): get_auth_request con 0x7f3ce00c86f0 auth_method 0 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cddd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cddd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cddd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cde575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cde575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cde575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cddd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cddd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.829+0000 7f3cddd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:06.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.830+0000 7f3cded76640 10 monclient(hunting): handle_auth_done global_id 4924 payload 824 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.830+0000 7f3cded76640 1 -- 10.20.193.16:0/1586508110 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3ce00c86f0 msgr2=0x7f3ce01f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.830+0000 7f3cded76640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3ce00c86f0 0x7f3ce01f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.830+0000 7f3cded76640 1 -- 10.20.193.16:0/1586508110 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ce00700f0 msgr2=0x7f3ce006ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.830+0000 7f3cded76640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ce00700f0 0x7f3ce006ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.830+0000 7f3cded76640 10 monclient: _finish_hunting 0 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.830+0000 7f3cded76640 1 monclient: found mon.noname-b 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.830+0000 7f3cded76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.830+0000 7f3cded76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.830+0000 7f3cded76640 1 -- 10.20.193.16:0/1586508110 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce006f160 con 0x7f3ce01536f0 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.831+0000 7f3cded76640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce01536f0 0x7f3ce0156750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000b9c0 tx=0x7f3cd000be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.831+0000 7f3cc6ffd640 10 client.?.objecter ms_handle_connect 0x7f3ce01536f0 2026-02-17T06:00:06.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.831+0000 7f3cc6ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:06.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3cc6ffd640 1 -- 10.20.193.16:0/1586508110 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000c790 con 0x7f3ce01536f0 2026-02-17T06:00:06.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3cc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:06.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3cc6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:06.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3cc6ffd640 10 monclient: dump: 2026-02-17T06:00:06.844 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3cc6ffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3cc6ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:06.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3cc6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3ce4d7e640 5 monclient: authenticate success, global_id 4924 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3ce4d7e640 10 monclient: _renew_subs 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3ce4d7e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/1586508110 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce006d8a0 con 0x7f3ce01536f0 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3ce4d7e640 10 client.4924.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3ce4d7e640 10 monclient: _renew_subs 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3ce4d7e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/1586508110 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce006de60 con 0x7f3ce01536f0 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.832+0000 7f3ce4d7e640 1 librados: init done 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3cc6ffd640 1 -- 10.20.193.16:0/1586508110 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd0010070 con 0x7f3ce01536f0 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3cc6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:06.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3cc6ffd640 1 -- 10.20.193.16:0/1586508110 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000cb30 con 0x7f3ce01536f0 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3cc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3cc6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3cc6ffd640 10 monclient: dump: 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:06.847 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:06.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3cdcd72640 4 set_mon_vals no callback set 2026-02-17T06:00:06.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3ce4d7e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:06.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3ce4d7e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:06.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3ce4d7e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:06.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.833+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/1586508110 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce00704d0 con 0x7f3ce01536f0 2026-02-17T06:00:06.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.941+0000 7f3cc6ffd640 1 -- 10.20.193.16:0/1586508110 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f3cd001d030 con 0x7f3ce01536f0 2026-02-17T06:00:06.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.941+0000 7f3cc6ffd640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3cac079140 0x7f3cac07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:06.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.941+0000 7f3cde575640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3cac079140 0x7f3cac07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:06.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.941+0000 7f3cde575640 10 monclient: get_auth_request con 0x7f3cac079140 auth_method 0 2026-02-17T06:00:06.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.941+0000 7f3cc6ffd640 10 client.4924.objecter ms_handle_connect 0x7f3cac079140 2026-02-17T06:00:06.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.941+0000 7f3cde575640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3cac079140 0x7f3cac07b600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4002770 tx=0x7f3cd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:06.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.943+0000 7f3cc6ffd640 1 -- 10.20.193.16:0/1586508110 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f3cd000ccd0 con 0x7f3ce01536f0 2026-02-17T06:00:06.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.943+0000 7f3cc6ffd640 10 client.4924.objecter ms_dispatch2 0x7f3ce0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:06.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.943+0000 7f3cc6ffd640 3 client.4924.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:06.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.943+0000 7f3cc6ffd640 3 client.4924.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:06.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.943+0000 7f3cc6ffd640 20 client.4924.objecter dump_active .. 0 homeless 2026-02-17T06:00:06.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.949+0000 7f3cc6ffd640 1 -- 10.20.193.16:0/1586508110 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3cd0024080 con 0x7f3ce01536f0 2026-02-17T06:00:06.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.949+0000 7f3cc6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:06.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:06.949+0000 7f3cc6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:07.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.047+0000 7f3ce4d7e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:07.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.047+0000 7f3ce4d7e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:07.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.047+0000 7f3ce4d7e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:07.048 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.047+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/1586508110 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3ce00ce070 con 0x7f3ce01536f0 2026-02-17T06:00:07.050 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.050+0000 7f3cc6ffd640 1 -- 10.20.193.16:0/1586508110 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3cd0061c90 con 0x7f3ce01536f0 2026-02-17T06:00:07.050 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.050+0000 7f3cc6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:07.050 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.050+0000 7f3cc6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:07.050 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:07.051 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:07.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 10 librados: watch_flush enter 2026-02-17T06:00:07.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 10 librados: watch_flush exit 2026-02-17T06:00:07.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 20 client.4924.objecter shutdown clearing up homeless session... 2026-02-17T06:00:07.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 10 client.4924.objecter successfully canceled tick 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/1586508110 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3cac079140 msgr2=0x7f3cac07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3cac079140 0x7f3cac07b600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4002770 tx=0x7f3cd4009040 comp rx=0 tx=0).stop 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 10 monclient: shutdown 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/1586508110 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce01536f0 msgr2=0x7f3ce0156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce01536f0 0x7f3ce0156750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000b9c0 tx=0x7f3cd000be90 comp rx=0 tx=0).stop 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/1586508110 shutdown_connections 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3cac079140 0x7f3cac07b600 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3ce01536f0 0x7f3ce0156750 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3ce00c86f0 0x7f3ce01f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 --2- 10.20.193.16:0/1586508110 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3ce00700f0 0x7f3ce006ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/1586508110 >> 10.20.193.16:0/1586508110 conn(0x7f3ce00cd770 msgr2=0x7f3ce0145030 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:07.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/1586508110 shutdown_connections 2026-02-17T06:00:07.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 -- 10.20.193.16:0/1586508110 wait complete. 2026-02-17T06:00:07.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:07.051+0000 7f3ce4d7e640 1 librados: shutdown 2026-02-17T06:00:10.061 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.129+0000 7f04e69c1640 10 monclient: get_monmap_and_config 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.129+0000 7f04e69c1640 10 monclient: build_initial_monmap 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.129+0000 7f04e69c1640 10 monclient: monmap: 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:10.130806+0000 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:10.130806+0000 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:10.130 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 1 Processor -- start 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 1 -- start start 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 10 monclient: init 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 10 monclient: _add_conns ranks=[4,5,1,2,0,3] 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f04e0155ac0 0x7f04e0073890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 10 monclient(hunting): picked mon.noname-b con 0x7f04e0155ac0 addr v2:10.20.193.128:3300/0 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f04e0155ea0 0x7f04e015d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 10 monclient(hunting): picked mon.noname-c con 0x7f04e0155ea0 addr v2:10.20.193.148:3300/0 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f04e006fe80 addr v1:10.20.193.128:6789/0 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e0072510 con 0x7f04e006fe80 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f04e00c8d10 con 0x7f04e0155ac0 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f04e00727f0 con 0x7f04e0155ea0 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.130+0000 7f04e69c1640 10 monclient(hunting): authenticate will time out at 1262.699889s 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04df7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f04e0155ea0 0x7f04e015d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:10.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04dffff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f04e0155ac0 0x7f04e0073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04df7fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f04e0155ea0 0x7f04e015d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:42490/0 (socket says 10.20.193.16:42490) 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04df7fe640 1 -- 10.20.193.16:0/2318881933 learned_addr learned my addr 10.20.193.16:0/2318881933 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04df7fe640 10 monclient(hunting): get_auth_request con 0x7f04e0155ea0 auth_method 0 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04df7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04df7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04dffff640 10 monclient(hunting): get_auth_request con 0x7f04e0155ac0 auth_method 0 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04dffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04dffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.131+0000 7f04df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.132+0000 7f04df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.132+0000 7f04dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.132+0000 7f04dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:10.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.132+0000 7f04dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:10.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.133+0000 7f04df7fe640 10 monclient(hunting): handle_auth_done global_id 4982 payload 274 2026-02-17T06:00:10.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.133+0000 7f04df7fe640 1 -- 10.20.193.16:0/2318881933 >> v2:10.20.193.128:3300/0 conn(0x7f04e0155ac0 msgr2=0x7f04e0073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:10.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.133+0000 7f04df7fe640 1 --2- 10.20.193.16:0/2318881933 >> v2:10.20.193.128:3300/0 conn(0x7f04e0155ac0 0x7f04e0073890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:10.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.133+0000 7f04df7fe640 1 -- 10.20.193.16:0/2318881933 >> v1:10.20.193.128:6789/0 conn(0x7f04e006fe80 legacy=0x7f04e00702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:10.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.133+0000 7f04df7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:00:10.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.133+0000 7f04df7fe640 1 monclient: found mon.noname-c 2026-02-17T06:00:10.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.133+0000 7f04df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:10.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.133+0000 7f04df7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:00:10.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.133+0000 7f04df7fe640 1 -- 10.20.193.16:0/2318881933 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e0072ad0 con 0x7f04e0155ea0 2026-02-17T06:00:10.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.134+0000 7f04df7fe640 1 --2- 10.20.193.16:0/2318881933 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e0155ea0 0x7f04e015d1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f04e0070f40 tx=0x7f04d002eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95ba9e6f47073fdc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:10.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.135+0000 7f04deffd640 1 -- 10.20.193.16:0/2318881933 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d003c070 con 0x7f04e0155ea0 2026-02-17T06:00:10.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.135+0000 7f04deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:10.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.135+0000 7f04deffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:10.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.135+0000 7f04deffd640 10 monclient: dump: 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.135+0000 7f04deffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.135+0000 7f04deffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.135+0000 7f04deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.135+0000 7f04e69c1640 5 monclient: authenticate success, global_id 4982 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.135+0000 7f04e69c1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04deffd640 1 -- 10.20.193.16:0/2318881933 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d002fb40 con 0x7f04e0155ea0 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04deffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04deffd640 1 -- 10.20.193.16:0/2318881933 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0037440 con 0x7f04e0155ea0 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04deffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04deffd640 10 monclient: dump: 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:10.136 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 4 set_mon_vals no callback set 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 10 monclient: shutdown 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 1 -- 10.20.193.16:0/2318881933 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e0155ea0 msgr2=0x7f04e015d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 1 --2- 10.20.193.16:0/2318881933 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e0155ea0 0x7f04e015d1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f04e0070f40 tx=0x7f04d002eed0 comp rx=0 tx=0).stop 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 1 -- 10.20.193.16:0/2318881933 shutdown_connections 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 1 --2- 10.20.193.16:0/2318881933 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e0155ea0 0x7f04e015d1c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 1 --2- 10.20.193.16:0/2318881933 >> v2:10.20.193.128:3300/0 conn(0x7f04e0155ac0 0x7f04e0073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 1 -- 10.20.193.16:0/2318881933 >> 10.20.193.16:0/2318881933 conn(0x7f04e00cd770 msgr2=0x7f04e00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 1 -- 10.20.193.16:0/2318881933 shutdown_connections 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 1 -- 10.20.193.16:0/2318881933 wait complete. 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 10 monclient: build_initial_monmap 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 10 monclient: monmap: 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:10.137 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 1 librados: starting msgr at 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.136+0000 7f04e69c1640 1 librados: starting objecter 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 1 Processor -- start 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 1 -- start start 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 1 librados: setting wanted keys 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 1 librados: calling monclient init 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient: init 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e006fe80 0x7f04e00c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient(hunting): picked mon.noname-c con 0x7f04e006fe80 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04e0155ac0 0x7f04e015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient(hunting): picked mon.noname-b con 0x7f04e0155ac0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04e0155ea0 0x7f04e01f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:10.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient(hunting): picked mon.noname-a con 0x7f04e0155ea0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0073370 con 0x7f04e0155ea0 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0073090 con 0x7f04e0155ac0 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f04e015cb30 con 0x7f04e006fe80 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.137+0000 7f04e69c1640 10 monclient(hunting): authenticate will time out at 1262.706745s 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04dffff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e006fe80 0x7f04e00c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04df7fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04e0155ac0 0x7f04e015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04e4f37640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04e0155ea0 0x7f04e01f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04dffff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e006fe80 0x7f04e00c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:42498/0 (socket says 10.20.193.16:42498) 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04dffff640 1 -- 10.20.193.16:0/1155844307 learned_addr learned my addr 10.20.193.16:0/1155844307 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04dffff640 10 monclient(hunting): get_auth_request con 0x7f04e006fe80 auth_method 0 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04dffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04dffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04e4f37640 10 monclient(hunting): get_auth_request con 0x7f04e0155ea0 auth_method 0 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04e4f37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04e4f37640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04e4f37640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04df7fe640 10 monclient(hunting): get_auth_request con 0x7f04e0155ac0 auth_method 0 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04df7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04df7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04e4f37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04e4f37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:10.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.138+0000 7f04e4f37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04dffff640 10 monclient(hunting): handle_auth_done global_id 4985 payload 824 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04dffff640 1 -- 10.20.193.16:0/1155844307 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04e0155ac0 msgr2=0x7f04e015f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04dffff640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04e0155ac0 0x7f04e015f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04dffff640 1 -- 10.20.193.16:0/1155844307 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04e0155ea0 msgr2=0x7f04e01f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04dffff640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04e0155ea0 0x7f04e01f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04dffff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04dffff640 1 monclient: found mon.noname-c 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04dffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04dffff640 1 -- 10.20.193.16:0/1155844307 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e014bed0 con 0x7f04e006fe80 2026-02-17T06:00:10.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.139+0000 7f04df7fe640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04e0155ac0 0x7f04e015f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:00:10.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.140+0000 7f04dffff640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e006fe80 0x7f04e00c4a60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00e8c0 tx=0x7f04cc00ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:10.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.140+0000 7f04dcff9640 10 client.?.objecter ms_handle_connect 0x7f04e006fe80 2026-02-17T06:00:10.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.140+0000 7f04dcff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 1 -- 10.20.193.16:0/1155844307 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc017070 con 0x7f04e006fe80 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 10 monclient: dump: 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:10.142 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 10 monclient: _finish_auth 0 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 5 monclient: authenticate success, global_id 4985 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 10 monclient: _renew_subs 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 1 -- 10.20.193.16:0/1155844307 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e01f4560 con 0x7f04e006fe80 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 10 client.4985.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 10 monclient: _renew_subs 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 1 -- 10.20.193.16:0/1155844307 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e01f5160 con 0x7f04e006fe80 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 1 librados: init done 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 1 -- 10.20.193.16:0/1155844307 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04cc005a90 con 0x7f04e006fe80 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 1 -- 10.20.193.16:0/1155844307 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc021060 con 0x7f04e006fe80 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04dcff9640 10 monclient: dump: 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04de7fc640 4 set_mon_vals no callback set 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:10.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.142+0000 7f04e69c1640 1 -- 10.20.193.16:0/1155844307 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e0073890 con 0x7f04e006fe80 2026-02-17T06:00:10.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.247+0000 7f04dcff9640 1 -- 10.20.193.16:0/1155844307 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f04cc024060 con 0x7f04e006fe80 2026-02-17T06:00:10.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.248+0000 7f04dcff9640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04b0079140 0x7f04b007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:10.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.248+0000 7f04df7fe640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04b0079140 0x7f04b007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:10.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.248+0000 7f04df7fe640 10 monclient: get_auth_request con 0x7f04b0079140 auth_method 0 2026-02-17T06:00:10.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.248+0000 7f04df7fe640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04b0079140 0x7f04b007b600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f04e006f470 tx=0x7f04d0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:10.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.248+0000 7f04dcff9640 10 client.4985.objecter ms_handle_connect 0x7f04b0079140 2026-02-17T06:00:10.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.249+0000 7f04dcff9640 1 -- 10.20.193.16:0/1155844307 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f04cc05e020 con 0x7f04e006fe80 2026-02-17T06:00:10.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.249+0000 7f04dcff9640 10 client.4985.objecter ms_dispatch2 0x7f04e0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:10.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.249+0000 7f04dcff9640 3 client.4985.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:10.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.249+0000 7f04dcff9640 3 client.4985.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:10.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.249+0000 7f04dcff9640 20 client.4985.objecter dump_active .. 0 homeless 2026-02-17T06:00:10.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.256+0000 7f04dcff9640 1 -- 10.20.193.16:0/1155844307 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f04cc061fe0 con 0x7f04e006fe80 2026-02-17T06:00:10.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.256+0000 7f04dcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:10.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.256+0000 7f04dcff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:10.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.354+0000 7f04e69c1640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:10.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.354+0000 7f04e69c1640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:10.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.354+0000 7f04e69c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:10.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.354+0000 7f04e69c1640 1 -- 10.20.193.16:0/1155844307 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f04e015f030 con 0x7f04e006fe80 2026-02-17T06:00:10.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04dcff9640 1 -- 10.20.193.16:0/1155844307 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f04cc0662e0 con 0x7f04e006fe80 2026-02-17T06:00:10.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04dcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:10.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04dcff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:10.357 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:10.358 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:10.359 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 10 librados: watch_flush enter 2026-02-17T06:00:10.359 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 10 librados: watch_flush exit 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 20 client.4985.objecter shutdown clearing up homeless session... 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 10 client.4985.objecter successfully canceled tick 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 -- 10.20.193.16:0/1155844307 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04b0079140 msgr2=0x7f04b007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04b0079140 0x7f04b007b600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f04e006f470 tx=0x7f04d0033000 comp rx=0 tx=0).stop 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 10 monclient: shutdown 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 -- 10.20.193.16:0/1155844307 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e006fe80 msgr2=0x7f04e00c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e006fe80 0x7f04e00c4a60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00e8c0 tx=0x7f04cc00ed90 comp rx=0 tx=0).stop 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 -- 10.20.193.16:0/1155844307 shutdown_connections 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04b0079140 0x7f04b007b600 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:10.360 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04e0155ac0 0x7f04e015f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:10.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04e0155ea0 0x7f04e01f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:10.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 --2- 10.20.193.16:0/1155844307 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04e006fe80 0x7f04e00c4a60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:10.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 -- 10.20.193.16:0/1155844307 >> 10.20.193.16:0/1155844307 conn(0x7f04e00cd770 msgr2=0x7f04e006b610 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:10.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 -- 10.20.193.16:0/1155844307 shutdown_connections 2026-02-17T06:00:10.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 -- 10.20.193.16:0/1155844307 wait complete. 2026-02-17T06:00:10.361 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:10.357+0000 7f04e69c1640 1 librados: shutdown 2026-02-17T06:00:13.367 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:13.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.436+0000 7fd24e3bc640 10 monclient: get_monmap_and_config 2026-02-17T06:00:13.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.436+0000 7fd24e3bc640 10 monclient: build_initial_monmap 2026-02-17T06:00:13.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.436+0000 7fd24e3bc640 10 monclient: monmap: 2026-02-17T06:00:13.438 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:13.438 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:13.438 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:13.437833+0000 2026-02-17T06:00:13.438 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:13.437833+0000 2026-02-17T06:00:13.438 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:13.438 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:13.438 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:13.439 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:13.439 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:13.439 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:13.439 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:13.439 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:13.439 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:13.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 1 Processor -- start 2026-02-17T06:00:13.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 1 -- start start 2026-02-17T06:00:13.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 10 monclient: init 2026-02-17T06:00:13.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:13.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 10 monclient: _add_conns ranks=[0,2,4,5,3,1] 2026-02-17T06:00:13.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd24814f020 addr v1:10.20.193.16:6789/0 2026-02-17T06:00:13.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd24814f400 addr v1:10.20.193.148:6789/0 2026-02-17T06:00:13.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fd24815caf0 0x7fd2480c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:13.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 10 monclient(hunting): picked mon.noname-b con 0x7fd24815caf0 addr v2:10.20.193.128:3300/0 2026-02-17T06:00:13.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2480c5b10 con 0x7fd24814f020 2026-02-17T06:00:13.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd24806e620 con 0x7fd24814f400 2026-02-17T06:00:13.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:13.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fd24806e900 con 0x7fd24815caf0 2026-02-17T06:00:13.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:13.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.437+0000 7fd24e3bc640 10 monclient(hunting): authenticate will time out at 1266.006817s 2026-02-17T06:00:13.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.438+0000 7fd247fff640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fd24814f020 0x7fd24806dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:33928/0 (socket says 10.20.193.16:33928) 2026-02-17T06:00:13.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.438+0000 7fd247fff640 1 -- 10.20.193.16:0/1810347391 learned_addr learned my addr 10.20.193.16:0/1810347391 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:13.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.438+0000 7fd24c932640 1 --2- 10.20.193.16:0/1810347391 >> v2:10.20.193.128:3300/0 conn(0x7fd24815caf0 0x7fd2480c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:13.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.438+0000 7fd24c932640 10 monclient(hunting): get_auth_request con 0x7fd24815caf0 auth_method 0 2026-02-17T06:00:13.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.438+0000 7fd24c932640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:13.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.438+0000 7fd24c932640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:13.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.438+0000 7fd24c932640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:13.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.438+0000 7fd24c932640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:13.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.438+0000 7fd24c932640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:13.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.438+0000 7fd24c932640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:13.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 1 -- 10.20.193.16:0/1810347391 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2745388035 0 0) 0x7fd2480c5b10 con 0x7fd24814f020 2026-02-17T06:00:13.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 10 monclient(hunting): _negotiate 2026-02-17T06:00:13.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:13.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:13.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 10 monclient(hunting): my global_id is 4854 2026-02-17T06:00:13.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 1 -- 10.20.193.16:0/1810347391 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd22c003620 con 0x7fd24814f020 2026-02-17T06:00:13.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 1 -- 10.20.193.16:0/1810347391 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 987818799 0 0) 0x7fd24806e620 con 0x7fd24814f400 2026-02-17T06:00:13.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 10 monclient(hunting): _negotiate 2026-02-17T06:00:13.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:13.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:13.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 10 monclient(hunting): my global_id is 4988 2026-02-17T06:00:13.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.439+0000 7fd246ffd640 1 -- 10.20.193.16:0/1810347391 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2480c5b10 con 0x7fd24814f400 2026-02-17T06:00:13.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.440+0000 7fd24c932640 10 monclient(hunting): handle_auth_done global_id 4933 payload 274 2026-02-17T06:00:13.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.440+0000 7fd24c932640 1 -- 10.20.193.16:0/1810347391 >> v1:10.20.193.148:6789/0 conn(0x7fd24814f400 legacy=0x7fd24815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:13.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.440+0000 7fd24c932640 1 -- 10.20.193.16:0/1810347391 >> v1:10.20.193.16:6789/0 conn(0x7fd24814f020 legacy=0x7fd24806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:13.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.440+0000 7fd24c932640 10 monclient: _finish_hunting 0 2026-02-17T06:00:13.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.440+0000 7fd24c932640 1 monclient: found mon.noname-b 2026-02-17T06:00:13.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.440+0000 7fd24c932640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:13.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.440+0000 7fd24c932640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:00:13.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.440+0000 7fd24c932640 1 -- 10.20.193.16:0/1810347391 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24806ebe0 con 0x7fd24815caf0 2026-02-17T06:00:13.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.441+0000 7fd24c932640 1 --2- 10.20.193.16:0/1810347391 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd24815caf0 0x7fd2480c6010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009d90 tx=0x7fd23c02f2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82df955b3e110f17 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:13.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 1 -- 10.20.193.16:0/1810347391 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c03c070 con 0x7fd24815caf0 2026-02-17T06:00:13.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:13.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:13.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 10 monclient: dump: 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:13.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:13.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:13.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 1 -- 10.20.193.16:0/1810347391 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c037440 con 0x7fd24815caf0 2026-02-17T06:00:13.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:13.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 5 monclient: authenticate success, global_id 4933 2026-02-17T06:00:13.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 1 -- 10.20.193.16:0/1810347391 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c037810 con 0x7fd24815caf0 2026-02-17T06:00:13.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:13.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 4 set_mon_vals no callback set 2026-02-17T06:00:13.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:13.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:13.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd246ffd640 10 monclient: dump: 2026-02-17T06:00:13.458 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:13.458 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:13.458 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:13.458 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:13.458 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:13.458 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:13.458 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:13.458 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:13.458 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:13.458 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:13.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 10 monclient: shutdown 2026-02-17T06:00:13.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:13.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 1 -- 10.20.193.16:0/1810347391 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd24815caf0 msgr2=0x7fd2480c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:13.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 1 --2- 10.20.193.16:0/1810347391 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd24815caf0 0x7fd2480c6010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009d90 tx=0x7fd23c02f2c0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 1 -- 10.20.193.16:0/1810347391 shutdown_connections 2026-02-17T06:00:13.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 1 --2- 10.20.193.16:0/1810347391 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd24815caf0 0x7fd2480c6010 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 1 -- 10.20.193.16:0/1810347391 >> 10.20.193.16:0/1810347391 conn(0x7fd2480cd770 msgr2=0x7fd2480cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:13.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 1 -- 10.20.193.16:0/1810347391 shutdown_connections 2026-02-17T06:00:13.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.443+0000 7fd24e3bc640 1 -- 10.20.193.16:0/1810347391 wait complete. 2026-02-17T06:00:13.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient: build_initial_monmap 2026-02-17T06:00:13.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient: monmap: 2026-02-17T06:00:13.460 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:13.460 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:13.460 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:13.460 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:13.460 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:13.461 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:13.461 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:13.461 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:13.461 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:13.461 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:13.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 librados: starting msgr at 2026-02-17T06:00:13.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 librados: starting objecter 2026-02-17T06:00:13.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 Processor -- start 2026-02-17T06:00:13.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 -- start start 2026-02-17T06:00:13.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 librados: setting wanted keys 2026-02-17T06:00:13.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 librados: calling monclient init 2026-02-17T06:00:13.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient: init 2026-02-17T06:00:13.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:13.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:00:13.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd24814f020 0x7fd2481f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:13.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient(hunting): picked mon.noname-a con 0x7fd24814f020 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:13.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd24814f400 0x7fd2481ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:13.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient(hunting): picked mon.noname-b con 0x7fd24814f400 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:13.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd24815caf0 0x7fd248158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:13.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient(hunting): picked mon.noname-c con 0x7fd24815caf0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:13.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:13.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd24806e2f0 con 0x7fd24814f020 2026-02-17T06:00:13.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:13.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd24806f1a0 con 0x7fd24814f400 2026-02-17T06:00:13.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:13.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2480c6550 con 0x7fd24815caf0 2026-02-17T06:00:13.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:13.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.444+0000 7fd24e3bc640 10 monclient(hunting): authenticate will time out at 1266.013886s 2026-02-17T06:00:13.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd247fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd24814f020 0x7fd2481f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:13.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd2477fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd24814f400 0x7fd2481ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:13.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd24c932640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd24815caf0 0x7fd248158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:13.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd247fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd24814f020 0x7fd2481f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51952/0 (socket says 10.20.193.16:51952) 2026-02-17T06:00:13.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd247fff640 1 -- 10.20.193.16:0/2373913418 learned_addr learned my addr 10.20.193.16:0/2373913418 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:13.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd247fff640 10 monclient(hunting): get_auth_request con 0x7fd24814f020 auth_method 0 2026-02-17T06:00:13.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd247fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:13.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd247fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:13.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd247fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:13.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd2477fe640 10 monclient(hunting): get_auth_request con 0x7fd24814f400 auth_method 0 2026-02-17T06:00:13.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd2477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:13.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd2477fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:13.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd2477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:13.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd24c932640 10 monclient(hunting): get_auth_request con 0x7fd24815caf0 auth_method 0 2026-02-17T06:00:13.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd24c932640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:13.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd24c932640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:13.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd24c932640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:13.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd2477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:13.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd2477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:13.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd2477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:13.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd24c932640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:13.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd24c932640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd24c932640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd247fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd247fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.445+0000 7fd247fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.446+0000 7fd2477fe640 10 monclient(hunting): handle_auth_done global_id 4936 payload 824 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.446+0000 7fd2477fe640 1 -- 10.20.193.16:0/2373913418 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd24815caf0 msgr2=0x7fd248158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.446+0000 7fd2477fe640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd24815caf0 0x7fd248158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.446+0000 7fd2477fe640 1 -- 10.20.193.16:0/2373913418 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd24814f020 msgr2=0x7fd2481f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.446+0000 7fd2477fe640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd24814f020 0x7fd2481f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.446+0000 7fd2477fe640 10 monclient: _finish_hunting 0 2026-02-17T06:00:13.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.446+0000 7fd2477fe640 1 monclient: found mon.noname-b 2026-02-17T06:00:13.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.446+0000 7fd2477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:13.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.446+0000 7fd2477fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:00:13.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.446+0000 7fd2477fe640 1 -- 10.20.193.16:0/2373913418 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2481586b0 con 0x7fd24814f400 2026-02-17T06:00:13.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.448+0000 7fd2477fe640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd24814f400 0x7fd2481ef4f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c4d0 tx=0x7fd23800c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:13.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.448+0000 7fd244ff9640 10 client.?.objecter ms_handle_connect 0x7fd24814f400 2026-02-17T06:00:13.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.448+0000 7fd244ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:13.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 1 -- 10.20.193.16:0/2373913418 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380052d0 con 0x7fd24814f400 2026-02-17T06:00:13.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:13.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:13.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 10 monclient: dump: 2026-02-17T06:00:13.469 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:13.469 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:13.469 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:13.469 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:13.469 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:13.469 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:13.469 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:13.469 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:13.469 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:13.469 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:13.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 10 monclient: _finish_auth 0 2026-02-17T06:00:13.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:13.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:13.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 1 -- 10.20.193.16:0/2373913418 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd238005470 con 0x7fd24814f400 2026-02-17T06:00:13.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:13.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 1 -- 10.20.193.16:0/2373913418 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238003930 con 0x7fd24814f400 2026-02-17T06:00:13.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:13.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:13.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd244ff9640 10 monclient: dump: 2026-02-17T06:00:13.471 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:13.471 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:13.471 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:13.471 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:13.471 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:13.471 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:13.471 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:13.471 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:13.471 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:13.471 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:13.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd24e3bc640 5 monclient: authenticate success, global_id 4936 2026-02-17T06:00:13.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd24e3bc640 10 monclient: _renew_subs 2026-02-17T06:00:13.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd2467fc640 4 set_mon_vals no callback set 2026-02-17T06:00:13.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd24e3bc640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:13.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd24e3bc640 1 -- 10.20.193.16:0/2373913418 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2481f2e70 con 0x7fd24814f400 2026-02-17T06:00:13.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd24e3bc640 10 client.4936.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:13.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd24e3bc640 10 monclient: _renew_subs 2026-02-17T06:00:13.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd24e3bc640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:13.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd24e3bc640 1 -- 10.20.193.16:0/2373913418 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2481f3430 con 0x7fd24814f400 2026-02-17T06:00:13.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.449+0000 7fd24e3bc640 1 librados: init done 2026-02-17T06:00:13.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.450+0000 7fd24e3bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:13.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.450+0000 7fd24e3bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:13.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.450+0000 7fd24e3bc640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:13.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.450+0000 7fd24e3bc640 1 -- 10.20.193.16:0/2373913418 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2480c6980 con 0x7fd24814f400 2026-02-17T06:00:13.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.462+0000 7fd244ff9640 1 -- 10.20.193.16:0/2373913418 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fd238003ad0 con 0x7fd24814f400 2026-02-17T06:00:13.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.462+0000 7fd244ff9640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd218079170 0x7fd21807b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:13.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.462+0000 7fd247fff640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd218079170 0x7fd21807b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:13.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.462+0000 7fd247fff640 10 monclient: get_auth_request con 0x7fd218079170 auth_method 0 2026-02-17T06:00:13.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.462+0000 7fd244ff9640 10 client.4936.objecter ms_handle_connect 0x7fd218079170 2026-02-17T06:00:13.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.462+0000 7fd247fff640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd218079170 0x7fd21807b630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd234002410 tx=0x7fd234009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:13.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.464+0000 7fd244ff9640 1 -- 10.20.193.16:0/2373913418 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fd23805e2b0 con 0x7fd24814f400 2026-02-17T06:00:13.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.464+0000 7fd244ff9640 10 client.4936.objecter ms_dispatch2 0x7fd248000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:13.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.464+0000 7fd244ff9640 3 client.4936.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:13.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.464+0000 7fd244ff9640 3 client.4936.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:13.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.464+0000 7fd244ff9640 20 client.4936.objecter dump_active .. 0 homeless 2026-02-17T06:00:13.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.568+0000 7fd244ff9640 1 -- 10.20.193.16:0/2373913418 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd2481586b0 con 0x7fd24814f400 2026-02-17T06:00:13.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.568+0000 7fd244ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:13.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.568+0000 7fd244ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:13.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.667+0000 7fd24e3bc640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:13.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.667+0000 7fd24e3bc640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:13.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.667+0000 7fd24e3bc640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:13.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.667+0000 7fd24e3bc640 1 -- 10.20.193.16:0/2373913418 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd24814e7d0 con 0x7fd24814f400 2026-02-17T06:00:13.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd244ff9640 1 -- 10.20.193.16:0/2373913418 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd238062270 con 0x7fd24814f400 2026-02-17T06:00:13.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd244ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:13.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd244ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:13.671 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:13.671 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:13.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 10 librados: watch_flush enter 2026-02-17T06:00:13.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 10 librados: watch_flush exit 2026-02-17T06:00:13.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 20 client.4936.objecter shutdown clearing up homeless session... 2026-02-17T06:00:13.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 10 client.4936.objecter successfully canceled tick 2026-02-17T06:00:13.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 -- 10.20.193.16:0/2373913418 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd218079170 msgr2=0x7fd21807b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:13.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd218079170 0x7fd21807b630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd234002410 tx=0x7fd234009040 comp rx=0 tx=0).stop 2026-02-17T06:00:13.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 10 monclient: shutdown 2026-02-17T06:00:13.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:13.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 -- 10.20.193.16:0/2373913418 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd24814f400 msgr2=0x7fd2481ef4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:13.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd24814f400 0x7fd2481ef4f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c4d0 tx=0x7fd23800c9a0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 -- 10.20.193.16:0/2373913418 shutdown_connections 2026-02-17T06:00:13.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd218079170 0x7fd21807b630 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd24814f400 0x7fd2481ef4f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd24815caf0 0x7fd248158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 --2- 10.20.193.16:0/2373913418 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd24814f020 0x7fd2481f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 -- 10.20.193.16:0/2373913418 >> 10.20.193.16:0/2373913418 conn(0x7fd2480cd770 msgr2=0x7fd2480c6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:13.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 -- 10.20.193.16:0/2373913418 shutdown_connections 2026-02-17T06:00:13.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 -- 10.20.193.16:0/2373913418 wait complete. 2026-02-17T06:00:13.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:13.671+0000 7fd24e3bc640 1 librados: shutdown 2026-02-17T06:00:16.681 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.750+0000 7f67591c1640 10 monclient: get_monmap_and_config 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.750+0000 7f67591c1640 10 monclient: build_initial_monmap 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.750+0000 7f67591c1640 10 monclient: monmap: 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:16.751621+0000 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:16.751621+0000 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 1 Processor -- start 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 1 -- start start 2026-02-17T06:00:16.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 10 monclient: init 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 10 monclient: _add_conns ranks=[0,2,5,3,4,1] 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6754155da0 addr v1:10.20.193.16:6789/0 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6754156180 addr v1:10.20.193.148:6789/0 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f675415f9d0 0x7f67540d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 10 monclient(hunting): picked mon.noname-c con 0x7f675415f9d0 addr v2:10.20.193.148:3300/0 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f675406f530 con 0x7f6754155da0 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f675406f810 con 0x7f6754156180 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f675406faf0 con 0x7f675415f9d0 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.751+0000 7f67591c1640 10 monclient(hunting): authenticate will time out at 1269.320847s 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.752+0000 7f6752d76640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f6754155da0 0x7f67540c9f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:33932/0 (socket says 10.20.193.16:33932) 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.752+0000 7f6752d76640 1 -- 10.20.193.16:0/2908393190 learned_addr learned my addr 10.20.193.16:0/2908393190 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:16.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.752+0000 7f6753577640 1 --2- 10.20.193.16:0/2908393190 >> v2:10.20.193.148:3300/0 conn(0x7f675415f9d0 0x7f67540d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6753577640 10 monclient(hunting): get_auth_request con 0x7f675415f9d0 auth_method 0 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6753577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6753577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6753577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1359953975 0 0) 0x7f675406f530 con 0x7f6754155da0 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6751d74640 10 monclient(hunting): _negotiate 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6751d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6751d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6751d74640 10 monclient(hunting): my global_id is 4860 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6740003620 con 0x7f6754155da0 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6753577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6753577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:16.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.753+0000 7f6753577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 <== mon.0 v1:10.20.193.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4217822830 0 0) 0x7f6740003620 con 0x7f6754155da0 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 >> v2:10.20.193.148:3300/0 conn(0x7f675415f9d0 msgr2=0x7f67540d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 --2- 10.20.193.16:0/2908393190 >> v2:10.20.193.148:3300/0 conn(0x7f675415f9d0 0x7f67540d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 >> v1:10.20.193.148:6789/0 conn(0x7f6754156180 legacy=0x7f675415f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient: _finish_hunting 0 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.16:6789/0 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 --> v1:10.20.193.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675406fdd0 con 0x7f6754155da0 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient: _finish_auth 0 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient: _check_auth_tickets 2026-02-17T06:00:16.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 <== mon.0 v1:10.20.193.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f6748002f00 con 0x7f6754155da0 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient: dump: 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] but i'm connected to v1:10.20.193.16:6789/0, reconnecting 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 >> v1:10.20.193.16:6789/0 conn(0x7f6754155da0 legacy=0x7f67540c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6740003bf0 0x7f6740004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient(hunting): picked mon.c con 0x7f6740003bf0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6740004900 0x7f6740004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient(hunting): picked mon.a con 0x7f6740004900 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 0x7f674000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient(hunting): picked mon.b con 0x7f67400053e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f675406faf0 con 0x7f6740004900 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f674000c2b0 con 0x7f67400053e0 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f674000c4b0 con 0x7f6740003bf0 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:16.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6751d74640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6753577640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 0x7f674000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6753577640 10 monclient(hunting): get_auth_request con 0x7f67400053e0 auth_method 0 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6753577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6753577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.755+0000 7f6753577640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.756+0000 7f6753577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.756+0000 7f6753577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.756+0000 7f6753577640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.756+0000 7f6752575640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6740004900 0x7f6740004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.756+0000 7f6752575640 10 monclient(hunting): get_auth_request con 0x7f6740004900 auth_method 0 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.756+0000 7f6752575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.756+0000 7f6752575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.756+0000 7f6752575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.756+0000 7f6752d76640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6740003bf0 0x7f6740004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 10 monclient(hunting): handle_auth_done global_id 4860 payload 293 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 1 -- 10.20.193.16:0/2908393190 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6740003bf0 msgr2=0x7f6740004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6740003bf0 0x7f6740004010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 1 -- 10.20.193.16:0/2908393190 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6740004900 msgr2=0x7f6740004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6740004900 0x7f6740004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 10 monclient: _finish_hunting 0 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 1 monclient: found mon.b 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:16.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 1 -- 10.20.193.16:0/2908393190 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6740003620 con 0x7f67400053e0 2026-02-17T06:00:16.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 10 monclient: _finish_auth 0 2026-02-17T06:00:16.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 10 monclient: _check_auth_tickets 2026-02-17T06:00:16.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:16.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f67591c1640 5 monclient: authenticate success, global_id 4860 2026-02-17T06:00:16.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f67591c1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:16.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.757+0000 7f6753577640 1 -- 10.20.193.16:0/2908393190 reap_dead start 2026-02-17T06:00:16.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.758+0000 7f6753577640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 0x7f674000bac0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6744009f90 tx=0x7f674402f5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6367252183d28e64 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674403a020 con 0x7f67400053e0 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f6751d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f6751d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f6751d74640 10 monclient: dump: 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6744003c90 con 0x7f67400053e0 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f6751d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f67591c1640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f6751d74640 1 -- 10.20.193.16:0/2908393190 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744009c10 con 0x7f67400053e0 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f6751d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f67591c1640 4 set_mon_vals no callback set 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f6751d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.759+0000 7f6751d74640 10 monclient: dump: 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:16.760 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 10 monclient: shutdown 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 -- 10.20.193.16:0/2908393190 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 msgr2=0x7f674000bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 0x7f674000bac0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6744009f90 tx=0x7f674402f5b0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 -- 10.20.193.16:0/2908393190 shutdown_connections 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 --2- 10.20.193.16:0/2908393190 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 0x7f674000bac0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 -- 10.20.193.16:0/2908393190 >> 10.20.193.16:0/2908393190 conn(0x7f67540cf600 msgr2=0x7f67540cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 -- 10.20.193.16:0/2908393190 shutdown_connections 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 -- 10.20.193.16:0/2908393190 wait complete. 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 10 monclient: build_initial_monmap 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 10 monclient: monmap: 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 librados: starting msgr at 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 librados: starting objecter 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 Processor -- start 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 -- start start 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 librados: setting wanted keys 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 1 librados: calling monclient init 2026-02-17T06:00:16.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.760+0000 7f67591c1640 10 monclient: init 2026-02-17T06:00:16.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:16.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:00:16.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 0x7f67541f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:16.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 10 monclient(hunting): picked mon.noname-b con 0x7f67400053e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:16.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67541f38c0 0x7f675415ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:16.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 10 monclient(hunting): picked mon.noname-a con 0x7f67541f38c0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:16.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f67540cc280 0x7f6754150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:16.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 10 monclient(hunting): picked mon.noname-c con 0x7f67540cc280 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:16.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67540705d0 con 0x7f67541f38c0 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f675406d870 con 0x7f67400053e0 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6754070750 con 0x7f67540cc280 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f67591c1640 10 monclient(hunting): authenticate will time out at 1269.330369s 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67541f38c0 0x7f675415ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6753577640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f67540cc280 0x7f6754150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67541f38c0 0x7f675415ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51956/0 (socket says 10.20.193.16:51956) 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752575640 1 -- 10.20.193.16:0/141112563 learned_addr learned my addr 10.20.193.16:0/141112563 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752575640 10 monclient(hunting): get_auth_request con 0x7f67541f38c0 auth_method 0 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752d76640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 0x7f67541f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6753577640 10 monclient(hunting): get_auth_request con 0x7f67540cc280 auth_method 0 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6753577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6753577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6753577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752d76640 10 monclient(hunting): get_auth_request con 0x7f67400053e0 auth_method 0 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.761+0000 7f6752575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.762+0000 7f6753577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.762+0000 7f6753577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.762+0000 7f6753577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.762+0000 7f6752d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.762+0000 7f6752d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.762+0000 7f6752d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.763+0000 7f6752575640 10 monclient(hunting): handle_auth_done global_id 4863 payload 824 2026-02-17T06:00:16.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.763+0000 7f6752575640 1 -- 10.20.193.16:0/141112563 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f67540cc280 msgr2=0x7f6754150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.763+0000 7f6752575640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f67540cc280 0x7f6754150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.763+0000 7f6752575640 1 -- 10.20.193.16:0/141112563 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 msgr2=0x7f67541f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.763+0000 7f6752575640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 0x7f67541f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.763+0000 7f6752575640 10 monclient: _finish_hunting 0 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.763+0000 7f6752575640 1 monclient: found mon.noname-a 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.763+0000 7f6752575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.763+0000 7f6752575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.763+0000 7f6752575640 1 -- 10.20.193.16:0/141112563 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67541f7ce0 con 0x7f67541f38c0 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.764+0000 7f6752575640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67541f38c0 0x7f675415ed40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f673c00e7e0 tx=0x7f673c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.764+0000 7f67337fe640 10 client.?.objecter ms_handle_connect 0x7f67541f38c0 2026-02-17T06:00:16.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.764+0000 7f67337fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 1 -- 10.20.193.16:0/141112563 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c002aa0 con 0x7f67541f38c0 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 10 monclient: dump: 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:16.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 5 monclient: authenticate success, global_id 4863 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 10 monclient: _renew_subs 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 1 -- 10.20.193.16:0/141112563 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6754150040 con 0x7f67541f38c0 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 10 client.4863.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 10 monclient: _renew_subs 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 1 -- 10.20.193.16:0/141112563 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67541f40d0 con 0x7f67541f38c0 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 1 librados: init done 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67591c1640 1 -- 10.20.193.16:0/141112563 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67540d01b0 con 0x7f67541f38c0 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 1 -- 10.20.193.16:0/141112563 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673c018070 con 0x7f67541f38c0 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 1 -- 10.20.193.16:0/141112563 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c002e40 con 0x7f67541f38c0 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f6751573640 4 set_mon_vals no callback set 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.765+0000 7f67337fe640 10 monclient: dump: 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:16.766 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:16.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.874+0000 7f67337fe640 1 -- 10.20.193.16:0/141112563 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f673c00c6c0 con 0x7f67541f38c0 2026-02-17T06:00:16.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.874+0000 7f67337fe640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6720079140 0x7f672007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:16.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.874+0000 7f6752d76640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6720079140 0x7f672007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:16.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.874+0000 7f6752d76640 10 monclient: get_auth_request con 0x7f6720079140 auth_method 0 2026-02-17T06:00:16.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.875+0000 7f6752d76640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6720079140 0x7f672007b600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6748004be0 tx=0x7f6748009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:16.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.875+0000 7f67337fe640 10 client.4863.objecter ms_handle_connect 0x7f6720079140 2026-02-17T06:00:16.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.881+0000 7f67337fe640 1 -- 10.20.193.16:0/141112563 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f673c05e2f0 con 0x7f67541f38c0 2026-02-17T06:00:16.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.882+0000 7f67337fe640 10 client.4863.objecter ms_dispatch2 0x7f6754000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:16.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.882+0000 7f67337fe640 3 client.4863.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:16.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.882+0000 7f67337fe640 3 client.4863.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:16.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.882+0000 7f67337fe640 20 client.4863.objecter dump_active .. 0 homeless 2026-02-17T06:00:16.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.882+0000 7f67337fe640 1 -- 10.20.193.16:0/141112563 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f673c0622b0 con 0x7f67541f38c0 2026-02-17T06:00:16.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.882+0000 7f67337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:16.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.882+0000 7f67337fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:16.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.981+0000 7f67591c1640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:16.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.981+0000 7f67591c1640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:16.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.981+0000 7f67591c1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:16.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.981+0000 7f67591c1640 1 -- 10.20.193.16:0/141112563 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6754155390 con 0x7f67541f38c0 2026-02-17T06:00:16.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.983+0000 7f67337fe640 1 -- 10.20.193.16:0/141112563 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f673c0665b0 con 0x7f67541f38c0 2026-02-17T06:00:16.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.983+0000 7f67337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:16.984 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.983+0000 7f67337fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:16.984 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:16.984 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:16.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 10 librados: watch_flush enter 2026-02-17T06:00:16.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 10 librados: watch_flush exit 2026-02-17T06:00:16.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 20 client.4863.objecter shutdown clearing up homeless session... 2026-02-17T06:00:16.986 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 10 client.4863.objecter successfully canceled tick 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 -- 10.20.193.16:0/141112563 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6720079140 msgr2=0x7f672007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6720079140 0x7f672007b600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6748004be0 tx=0x7f6748009040 comp rx=0 tx=0).stop 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 10 monclient: shutdown 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 -- 10.20.193.16:0/141112563 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67541f38c0 msgr2=0x7f675415ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67541f38c0 0x7f675415ed40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f673c00e7e0 tx=0x7f673c00ecb0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 -- 10.20.193.16:0/141112563 shutdown_connections 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6720079140 0x7f672007b600 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f67540cc280 0x7f6754150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67541f38c0 0x7f675415ed40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 --2- 10.20.193.16:0/141112563 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67400053e0 0x7f67541f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:16.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 -- 10.20.193.16:0/141112563 >> 10.20.193.16:0/141112563 conn(0x7f67540cf600 msgr2=0x7f675414ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:16.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.984+0000 7f67591c1640 1 -- 10.20.193.16:0/141112563 shutdown_connections 2026-02-17T06:00:16.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.985+0000 7f67591c1640 1 -- 10.20.193.16:0/141112563 wait complete. 2026-02-17T06:00:16.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:16.985+0000 7f67591c1640 1 librados: shutdown 2026-02-17T06:00:19.994 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:20.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.062+0000 7f052e2fa640 10 monclient: get_monmap_and_config 2026-02-17T06:00:20.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.062+0000 7f052e2fa640 10 monclient: build_initial_monmap 2026-02-17T06:00:20.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.062+0000 7f052e2fa640 10 monclient: monmap: 2026-02-17T06:00:20.063 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:20.063 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:20.063 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:20.063497+0000 2026-02-17T06:00:20.064 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:20.063497+0000 2026-02-17T06:00:20.064 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:20.064 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:20.064 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:20.064 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:20.064 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:20.064 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:20.064 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:20.064 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:20.064 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:20.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.062+0000 7f052e2fa640 1 Processor -- start 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 1 -- start start 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 10 monclient: init 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 10 monclient: _add_conns ranks=[1,3,4,5,0,2] 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0528155ce0 addr v1:10.20.193.128:6789/0 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f05281560c0 0x7f052806e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 10 monclient(hunting): picked mon.noname-a con 0x7f05281560c0 addr v2:10.20.193.16:3300/0 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f05280c9f80 0x7f05280ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 10 monclient(hunting): picked mon.noname-b con 0x7f05280c9f80 addr v2:10.20.193.128:3300/0 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05280705b0 con 0x7f0528155ce0 2026-02-17T06:00:20.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:20.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f052806cbb0 con 0x7f05281560c0 2026-02-17T06:00:20.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:20.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f052806ce90 con 0x7f05280c9f80 2026-02-17T06:00:20.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:20.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f052e2fa640 10 monclient(hunting): authenticate will time out at 1272.632604s 2026-02-17T06:00:20.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.063+0000 7f05277fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f05281560c0 0x7f052806e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:20.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f052c870640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f05280c9f80 0x7f05280ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:20.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f05277fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f05281560c0 0x7f052806e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51962/0 (socket says 10.20.193.16:51962) 2026-02-17T06:00:20.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f05277fe640 1 -- 10.20.193.16:0/1825576995 learned_addr learned my addr 10.20.193.16:0/1825576995 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:20.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f05277fe640 10 monclient(hunting): get_auth_request con 0x7f05281560c0 auth_method 0 2026-02-17T06:00:20.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f05277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:20.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f05277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:20.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f05277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:20.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f052c870640 10 monclient(hunting): get_auth_request con 0x7f05280c9f80 auth_method 0 2026-02-17T06:00:20.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f052c870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:20.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f052c870640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:20.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f052c870640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:20.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f05277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:20.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f05277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:20.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f05277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f052c870640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f052c870640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.064+0000 7f052c870640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.065+0000 7f052c870640 10 monclient(hunting): handle_auth_done global_id 4942 payload 274 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.065+0000 7f052c870640 1 -- 10.20.193.16:0/1825576995 >> v2:10.20.193.16:3300/0 conn(0x7f05281560c0 msgr2=0x7f052806e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.065+0000 7f052c870640 1 --2- 10.20.193.16:0/1825576995 >> v2:10.20.193.16:3300/0 conn(0x7f05281560c0 0x7f052806e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.065+0000 7f052c870640 1 -- 10.20.193.16:0/1825576995 >> v1:10.20.193.128:6789/0 conn(0x7f0528155ce0 legacy=0x7f052806fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.065+0000 7f052c870640 10 monclient: _finish_hunting 0 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.065+0000 7f052c870640 1 monclient: found mon.noname-b 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.065+0000 7f052c870640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:20.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.065+0000 7f052c870640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:00:20.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.065+0000 7f052c870640 1 -- 10.20.193.16:0/1825576995 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052806d170 con 0x7f05280c9f80 2026-02-17T06:00:20.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.066+0000 7f052c870640 1 --2- 10.20.193.16:0/1825576995 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f05280c9f80 0x7f05280ca3a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0528160270 tx=0x7f051c02f250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1696d103b84eb1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:20.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 1 -- 10.20.193.16:0/1825576995 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c03c070 con 0x7f05280c9f80 2026-02-17T06:00:20.079 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:20.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:20.080 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 10 monclient: dump: 2026-02-17T06:00:20.080 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:20.080 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:20.080 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:20.080 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:20.080 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:20.080 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:20.080 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:20.080 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:20.081 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:20.081 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:20.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:20.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:20.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:20.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f052e2fa640 5 monclient: authenticate success, global_id 4942 2026-02-17T06:00:20.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 1 -- 10.20.193.16:0/1825576995 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051c037440 con 0x7f05280c9f80 2026-02-17T06:00:20.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f052e2fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:20.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:20.081 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 1 -- 10.20.193.16:0/1825576995 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c037810 con 0x7f05280c9f80 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f052e2fa640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f052e2fa640 4 set_mon_vals no callback set 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.068+0000 7f0526ffd640 10 monclient: dump: 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:20.082 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:20.083 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:20.083 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:20.083 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:20.083 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:20.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 10 monclient: shutdown 2026-02-17T06:00:20.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:20.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 -- 10.20.193.16:0/1825576995 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f05280c9f80 msgr2=0x7f05280ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:20.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 --2- 10.20.193.16:0/1825576995 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f05280c9f80 0x7f05280ca3a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0528160270 tx=0x7f051c02f250 comp rx=0 tx=0).stop 2026-02-17T06:00:20.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 -- 10.20.193.16:0/1825576995 shutdown_connections 2026-02-17T06:00:20.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 --2- 10.20.193.16:0/1825576995 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f05280c9f80 0x7f05280ca3a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 --2- 10.20.193.16:0/1825576995 >> v2:10.20.193.16:3300/0 conn(0x7f05281560c0 0x7f052806e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 -- 10.20.193.16:0/1825576995 >> 10.20.193.16:0/1825576995 conn(0x7f05280cf600 msgr2=0x7f05280cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 -- 10.20.193.16:0/1825576995 shutdown_connections 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 -- 10.20.193.16:0/1825576995 wait complete. 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 10 monclient: build_initial_monmap 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 10 monclient: monmap: 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:20.084 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 librados: starting msgr at 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 librados: starting objecter 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.069+0000 7f052e2fa640 1 Processor -- start 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 1 -- start start 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 1 librados: setting wanted keys 2026-02-17T06:00:20.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 1 librados: calling monclient init 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient: init 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f05280c9f80 0x7f0528157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient(hunting): picked mon.noname-c con 0x7f05280c9f80 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0528155ce0 0x7f0528153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient(hunting): picked mon.noname-b con 0x7f0528155ce0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f05281560c0 0x7f05281539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient(hunting): picked mon.noname-a con 0x7f05281560c0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f052806da10 con 0x7f05281560c0 2026-02-17T06:00:20.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f052806d730 con 0x7f0528155ce0 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f052806dcf0 con 0x7f05280c9f80 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052e2fa640 10 monclient(hunting): authenticate will time out at 1272.639431s 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052c870640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f05281560c0 0x7f05281539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f05277fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0528155ce0 0x7f0528153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052c870640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f05281560c0 0x7f05281539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51966/0 (socket says 10.20.193.16:51966) 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052c870640 1 -- 10.20.193.16:0/12199050 learned_addr learned my addr 10.20.193.16:0/12199050 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052c870640 10 monclient(hunting): get_auth_request con 0x7f05281560c0 auth_method 0 2026-02-17T06:00:20.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052c870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:20.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052c870640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:20.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f052c870640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:20.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.070+0000 7f0527fff640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f05280c9f80 0x7f0528157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:20.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f05277fe640 10 monclient(hunting): get_auth_request con 0x7f0528155ce0 auth_method 0 2026-02-17T06:00:20.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f05277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:20.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f05277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:20.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f05277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:20.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f052c870640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:20.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f052c870640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:20.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f052c870640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f0527fff640 10 monclient(hunting): get_auth_request con 0x7f05280c9f80 auth_method 0 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f0527fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f0527fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f0527fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f05277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f05277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f05277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f0527fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f0527fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.071+0000 7f0527fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:20.089 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.072+0000 7f052c870640 10 monclient(hunting): handle_auth_done global_id 4869 payload 824 2026-02-17T06:00:20.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.072+0000 7f052c870640 1 -- 10.20.193.16:0/12199050 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f05280c9f80 msgr2=0x7f0528157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:20.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.072+0000 7f052c870640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f05280c9f80 0x7f0528157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:20.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.072+0000 7f052c870640 1 -- 10.20.193.16:0/12199050 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0528155ce0 msgr2=0x7f0528153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:20.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.072+0000 7f052c870640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0528155ce0 0x7f0528153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:20.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.072+0000 7f052c870640 10 monclient: _finish_hunting 0 2026-02-17T06:00:20.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.072+0000 7f052c870640 1 monclient: found mon.noname-a 2026-02-17T06:00:20.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.072+0000 7f052c870640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:20.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.072+0000 7f052c870640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:20.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.072+0000 7f052c870640 1 -- 10.20.193.16:0/12199050 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0528156730 con 0x7f05281560c0 2026-02-17T06:00:20.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.073+0000 7f052c870640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f05281560c0 0x7f05281539b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f051c009e90 tx=0x7f051c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.073+0000 7f0524ff9640 10 client.?.objecter ms_handle_connect 0x7f05281560c0 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.073+0000 7f0524ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 1 -- 10.20.193.16:0/12199050 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c03c050 con 0x7f05281560c0 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 10 monclient: dump: 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:20.091 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 10 monclient: _finish_auth 0 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 1 -- 10.20.193.16:0/12199050 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051c048070 con 0x7f05281560c0 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 5 monclient: authenticate success, global_id 4869 2026-02-17T06:00:20.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 10 monclient: _renew_subs 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 1 -- 10.20.193.16:0/12199050 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0528156a10 con 0x7f05281560c0 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 1 -- 10.20.193.16:0/12199050 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c035a90 con 0x7f05281560c0 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f0524ff9640 10 monclient: dump: 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:20.093 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:20.094 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:20.094 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:20.094 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:20.094 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:20.094 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:20.094 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:20.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f05267fc640 4 set_mon_vals no callback set 2026-02-17T06:00:20.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 10 client.4869.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:20.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 10 monclient: _renew_subs 2026-02-17T06:00:20.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:20.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 1 -- 10.20.193.16:0/12199050 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0528156fd0 con 0x7f05281560c0 2026-02-17T06:00:20.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 1 librados: init done 2026-02-17T06:00:20.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:20.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:20.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:20.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.075+0000 7f052e2fa640 1 -- 10.20.193.16:0/12199050 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f052806c870 con 0x7f05281560c0 2026-02-17T06:00:20.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.087+0000 7f0524ff9640 1 -- 10.20.193.16:0/12199050 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f051c035c30 con 0x7f05281560c0 2026-02-17T06:00:20.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.088+0000 7f0524ff9640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04f80791c0 0x7f04f807b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:20.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.088+0000 7f0527fff640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04f80791c0 0x7f04f807b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:20.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.088+0000 7f0527fff640 10 monclient: get_auth_request con 0x7f04f80791c0 auth_method 0 2026-02-17T06:00:20.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.088+0000 7f0524ff9640 10 client.4869.objecter ms_handle_connect 0x7f04f80791c0 2026-02-17T06:00:20.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.088+0000 7f0527fff640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04f80791c0 0x7f04f807b680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0514004660 tx=0x7f0514009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:20.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.195+0000 7f0524ff9640 1 -- 10.20.193.16:0/12199050 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f051c0840a0 con 0x7f05281560c0 2026-02-17T06:00:20.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.195+0000 7f0524ff9640 10 client.4869.objecter ms_dispatch2 0x7f0528000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:20.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.195+0000 7f0524ff9640 3 client.4869.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:20.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.195+0000 7f0524ff9640 3 client.4869.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:20.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.195+0000 7f0524ff9640 20 client.4869.objecter dump_active .. 0 homeless 2026-02-17T06:00:20.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.195+0000 7f0524ff9640 1 -- 10.20.193.16:0/12199050 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f051c051020 con 0x7f05281560c0 2026-02-17T06:00:20.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.195+0000 7f0524ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:20.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.195+0000 7f0524ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:20.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.299+0000 7f052e2fa640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:20.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.299+0000 7f052e2fa640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:20.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.299+0000 7f052e2fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:20.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.299+0000 7f052e2fa640 1 -- 10.20.193.16:0/12199050 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f052806be30 con 0x7f05281560c0 2026-02-17T06:00:20.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.306+0000 7f0524ff9640 1 -- 10.20.193.16:0/12199050 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f051c088060 con 0x7f05281560c0 2026-02-17T06:00:20.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.306+0000 7f0524ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:20.307 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.306+0000 7f0524ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:20.308 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:20.308 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:20.309 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 10 librados: watch_flush enter 2026-02-17T06:00:20.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 10 librados: watch_flush exit 2026-02-17T06:00:20.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 20 client.4869.objecter shutdown clearing up homeless session... 2026-02-17T06:00:20.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 10 client.4869.objecter successfully canceled tick 2026-02-17T06:00:20.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 -- 10.20.193.16:0/12199050 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04f80791c0 msgr2=0x7f04f807b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:20.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04f80791c0 0x7f04f807b680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0514004660 tx=0x7f0514009110 comp rx=0 tx=0).stop 2026-02-17T06:00:20.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 10 monclient: shutdown 2026-02-17T06:00:20.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:20.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 -- 10.20.193.16:0/12199050 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f05281560c0 msgr2=0x7f05281539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:20.310 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f05281560c0 0x7f05281539b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f051c009e90 tx=0x7f051c004b40 comp rx=0 tx=0).stop 2026-02-17T06:00:20.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 -- 10.20.193.16:0/12199050 shutdown_connections 2026-02-17T06:00:20.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04f80791c0 0x7f04f807b680 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:20.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0528155ce0 0x7f0528153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:20.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f05281560c0 0x7f05281539b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:20.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 --2- 10.20.193.16:0/12199050 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f05280c9f80 0x7f0528157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:20.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 -- 10.20.193.16:0/12199050 >> 10.20.193.16:0/12199050 conn(0x7f05280cf600 msgr2=0x7f05280ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:20.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 -- 10.20.193.16:0/12199050 shutdown_connections 2026-02-17T06:00:20.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 -- 10.20.193.16:0/12199050 wait complete. 2026-02-17T06:00:20.311 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:20.307+0000 7f052e2fa640 1 librados: shutdown 2026-02-17T06:00:23.317 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:23.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.387+0000 7f078b16a640 10 monclient: get_monmap_and_config 2026-02-17T06:00:23.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.387+0000 7f078b16a640 10 monclient: build_initial_monmap 2026-02-17T06:00:23.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.387+0000 7f078b16a640 10 monclient: monmap: 2026-02-17T06:00:23.388 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:23.388 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:23.388 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:23.388176+0000 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:23.388176+0000 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.387+0000 7f078b16a640 1 Processor -- start 2026-02-17T06:00:23.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.387+0000 7f078b16a640 1 -- start start 2026-02-17T06:00:23.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.387+0000 7f078b16a640 10 monclient: init 2026-02-17T06:00:23.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:23.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 10 monclient: _add_conns ranks=[5,0,2,3,4,1] 2026-02-17T06:00:23.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f07840700f0 0x7f07840704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:23.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 10 monclient(hunting): picked mon.noname-c con 0x7f07840700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:00:23.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f07840c7fe0 addr v1:10.20.193.16:6789/0 2026-02-17T06:00:23.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f07841536f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07840c4820 con 0x7f07840c7fe0 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07840c4b00 con 0x7f07841536f0 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f07840c4de0 con 0x7f07840700f0 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f078b16a640 10 monclient(hunting): authenticate will time out at 1275.957167s 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f0788edf640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f07840700f0 0x7f07840704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f077bfff640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f07840c7fe0 0x7f07840c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:34718/0 (socket says 10.20.193.16:34718) 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f077bfff640 1 -- 10.20.193.16:0/791986554 learned_addr learned my addr 10.20.193.16:0/791986554 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f0788edf640 10 monclient(hunting): get_auth_request con 0x7f07840700f0 auth_method 0 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f0788edf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:23.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f0788edf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.388+0000 7f0788edf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.389+0000 7f0788edf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.389+0000 7f0788edf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.389+0000 7f0788edf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.389+0000 7f077b7fe640 1 -- 10.20.193.16:0/791986554 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 490475198 0 0) 0x7f07840c4820 con 0x7f07840c7fe0 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.389+0000 7f077b7fe640 10 monclient(hunting): _negotiate 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.389+0000 7f077b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.389+0000 7f077b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.389+0000 7f077b7fe640 10 monclient(hunting): my global_id is 4872 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.389+0000 7f077b7fe640 1 -- 10.20.193.16:0/791986554 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0770003620 con 0x7f07840c7fe0 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.390+0000 7f0788edf640 10 monclient(hunting): handle_auth_done global_id 5006 payload 274 2026-02-17T06:00:23.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.390+0000 7f0788edf640 1 -- 10.20.193.16:0/791986554 >> v1:10.20.193.148:6789/0 conn(0x7f07841536f0 legacy=0x7f0784153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:23.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.390+0000 7f0788edf640 1 -- 10.20.193.16:0/791986554 >> v1:10.20.193.16:6789/0 conn(0x7f07840c7fe0 legacy=0x7f07840c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:23.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.390+0000 7f0788edf640 10 monclient: _finish_hunting 0 2026-02-17T06:00:23.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.390+0000 7f0788edf640 1 monclient: found mon.noname-c 2026-02-17T06:00:23.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.390+0000 7f0788edf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:23.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.390+0000 7f0788edf640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:00:23.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.390+0000 7f0788edf640 1 -- 10.20.193.16:0/791986554 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07840c50c0 con 0x7f07840700f0 2026-02-17T06:00:23.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.392+0000 7f0788edf640 1 --2- 10.20.193.16:0/791986554 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f07840700f0 0x7f07840704d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07740098a0 tx=0x7f077402edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa026865804c821 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:23.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 1 -- 10.20.193.16:0/791986554 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077403c070 con 0x7f07840700f0 2026-02-17T06:00:23.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:23.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:23.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 10 monclient: dump: 2026-02-17T06:00:23.403 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:23.403 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:23.403 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:23.403 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:23.403 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:23.403 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:23.403 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:23.403 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:23.403 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 1 -- 10.20.193.16:0/791986554 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077402fae0 con 0x7f07840700f0 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 5 monclient: authenticate success, global_id 5006 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 1 -- 10.20.193.16:0/791986554 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774037440 con 0x7f07840700f0 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:23.404 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 4 set_mon_vals no callback set 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f077b7fe640 10 monclient: dump: 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:23.405 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 10 monclient: shutdown 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 1 -- 10.20.193.16:0/791986554 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f07840700f0 msgr2=0x7f07840704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 1 --2- 10.20.193.16:0/791986554 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f07840700f0 0x7f07840704d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07740098a0 tx=0x7f077402edd0 comp rx=0 tx=0).stop 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 1 -- 10.20.193.16:0/791986554 shutdown_connections 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 1 --2- 10.20.193.16:0/791986554 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f07840700f0 0x7f07840704d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 1 -- 10.20.193.16:0/791986554 >> 10.20.193.16:0/791986554 conn(0x7f07840cd770 msgr2=0x7f07840cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 1 -- 10.20.193.16:0/791986554 shutdown_connections 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.394+0000 7f078b16a640 1 -- 10.20.193.16:0/791986554 wait complete. 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient: build_initial_monmap 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient: monmap: 2026-02-17T06:00:23.406 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 librados: starting msgr at 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 librados: starting objecter 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 Processor -- start 2026-02-17T06:00:23.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 -- start start 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 librados: setting wanted keys 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 librados: calling monclient init 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient: init 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f07840700f0 0x7f07841f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient(hunting): picked mon.noname-a con 0x7f07840700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f07840c7fe0 0x7f0784072f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient(hunting): picked mon.noname-b con 0x7f07840c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f07841536f0 0x7f07840cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient(hunting): picked mon.noname-c con 0x7f07841536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:23.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0784074c00 con 0x7f07840700f0 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f07840c5680 con 0x7f07840c7fe0 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0784074d80 con 0x7f07841536f0 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.395+0000 7f078b16a640 10 monclient(hunting): authenticate will time out at 1275.964869s 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f0788edf640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f07840700f0 0x7f07841f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f07896e0640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f07841536f0 0x7f07840cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f0788edf640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f07840700f0 0x7f07841f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44348/0 (socket says 10.20.193.16:44348) 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f0788edf640 1 -- 10.20.193.16:0/1092760336 learned_addr learned my addr 10.20.193.16:0/1092760336 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:23.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f0788edf640 10 monclient(hunting): get_auth_request con 0x7f07840700f0 auth_method 0 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f0788edf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f0788edf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f0788edf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f077bfff640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f07840c7fe0 0x7f0784072f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f07896e0640 10 monclient(hunting): get_auth_request con 0x7f07841536f0 auth_method 0 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f07896e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f07896e0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f07896e0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f0788edf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f0788edf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f0788edf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f07896e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:23.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f07896e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f07896e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f077bfff640 10 monclient(hunting): get_auth_request con 0x7f07840c7fe0 auth_method 0 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f077bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f077bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f077bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f077bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f077bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.396+0000 7f077bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.398+0000 7f07896e0640 10 monclient(hunting): handle_auth_done global_id 5012 payload 824 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.398+0000 7f07896e0640 1 -- 10.20.193.16:0/1092760336 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f07840c7fe0 msgr2=0x7f0784072f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.398+0000 7f07896e0640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f07840c7fe0 0x7f0784072f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:23.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.398+0000 7f07896e0640 1 -- 10.20.193.16:0/1092760336 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f07840700f0 msgr2=0x7f07841f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.398+0000 7f07896e0640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f07840700f0 0x7f07841f4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.398+0000 7f07896e0640 10 monclient: _finish_hunting 0 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.398+0000 7f07896e0640 1 monclient: found mon.noname-c 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.398+0000 7f07896e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.398+0000 7f07896e0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.398+0000 7f07896e0640 1 -- 10.20.193.16:0/1092760336 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07840cbac0 con 0x7f07841536f0 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.399+0000 7f07896e0640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f07841536f0 0x7f07840cb180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f078000cb10 tx=0x7f078000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.399+0000 7f07797fa640 10 client.?.objecter ms_handle_connect 0x7f07841536f0 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.399+0000 7f07797fa640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 1 -- 10.20.193.16:0/1092760336 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078000acb0 con 0x7f07841536f0 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:23.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 10 monclient: dump: 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 10 monclient: _finish_auth 0 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 10 monclient: _check_auth_tickets 2026-02-17T06:00:23.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 1 -- 10.20.193.16:0/1092760336 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078000ae50 con 0x7f07841536f0 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 1 -- 10.20.193.16:0/1092760336 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780003c70 con 0x7f07841536f0 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f07797fa640 10 monclient: dump: 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:23.414 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.400+0000 7f077affd640 4 set_mon_vals no callback set 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 5 monclient: authenticate success, global_id 5012 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 10 monclient: _renew_subs 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 1 -- 10.20.193.16:0/1092760336 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07840cbda0 con 0x7f07841536f0 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 10 client.5012.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 10 monclient: _renew_subs 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:23.415 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 1 -- 10.20.193.16:0/1092760336 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0784072780 con 0x7f07841536f0 2026-02-17T06:00:23.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 1 librados: init done 2026-02-17T06:00:23.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:23.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:23.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:23.416 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.401+0000 7f078b16a640 1 -- 10.20.193.16:0/1092760336 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07840704d0 con 0x7f07841536f0 2026-02-17T06:00:23.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.412+0000 7f07797fa640 1 -- 10.20.193.16:0/1092760336 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f07800050d0 con 0x7f07841536f0 2026-02-17T06:00:23.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.413+0000 7f07797fa640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0754079140 0x7f075407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:23.417 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.413+0000 7f0788edf640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0754079140 0x7f075407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:23.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.413+0000 7f0788edf640 10 monclient: get_auth_request con 0x7f0754079140 auth_method 0 2026-02-17T06:00:23.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.413+0000 7f07797fa640 10 client.5012.objecter ms_handle_connect 0x7f0754079140 2026-02-17T06:00:23.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.413+0000 7f0788edf640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0754079140 0x7f075407b600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0774016da0 tx=0x7f0774033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:23.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.414+0000 7f07797fa640 1 -- 10.20.193.16:0/1092760336 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f078005e830 con 0x7f07841536f0 2026-02-17T06:00:23.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.414+0000 7f07797fa640 10 client.5012.objecter ms_dispatch2 0x7f0784000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:23.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.414+0000 7f07797fa640 3 client.5012.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:23.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.414+0000 7f07797fa640 3 client.5012.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:23.418 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.415+0000 7f07797fa640 20 client.5012.objecter dump_active .. 0 homeless 2026-02-17T06:00:23.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.519+0000 7f07797fa640 1 -- 10.20.193.16:0/1092760336 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f07800627f0 con 0x7f07841536f0 2026-02-17T06:00:23.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.519+0000 7f07797fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:23.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.519+0000 7f07797fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:23.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.620+0000 7f078b16a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:23.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.620+0000 7f078b16a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:23.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.620+0000 7f078b16a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:23.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.620+0000 7f078b16a640 1 -- 10.20.193.16:0/1092760336 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0784153cc0 con 0x7f07841536f0 2026-02-17T06:00:23.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.622+0000 7f07797fa640 1 -- 10.20.193.16:0/1092760336 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f0780066af0 con 0x7f07841536f0 2026-02-17T06:00:23.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.622+0000 7f07797fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:23.623 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.622+0000 7f07797fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:23.623 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:23.623 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 10 librados: watch_flush enter 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 10 librados: watch_flush exit 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 20 client.5012.objecter shutdown clearing up homeless session... 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 10 client.5012.objecter successfully canceled tick 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 -- 10.20.193.16:0/1092760336 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0754079140 msgr2=0x7f075407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0754079140 0x7f075407b600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0774016da0 tx=0x7f0774033000 comp rx=0 tx=0).stop 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 10 monclient: shutdown 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 -- 10.20.193.16:0/1092760336 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f07841536f0 msgr2=0x7f07840cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f07841536f0 0x7f07840cb180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f078000cb10 tx=0x7f078000a060 comp rx=0 tx=0).stop 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 -- 10.20.193.16:0/1092760336 shutdown_connections 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0754079140 0x7f075407b600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f07840c7fe0 0x7f0784072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:23.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f07841536f0 0x7f07840cb180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:23.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 --2- 10.20.193.16:0/1092760336 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f07840700f0 0x7f07841f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:23.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 -- 10.20.193.16:0/1092760336 >> 10.20.193.16:0/1092760336 conn(0x7f07840cd770 msgr2=0x7f078414e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:23.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 -- 10.20.193.16:0/1092760336 shutdown_connections 2026-02-17T06:00:23.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.623+0000 7f078b16a640 1 -- 10.20.193.16:0/1092760336 wait complete. 2026-02-17T06:00:23.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:23.624+0000 7f078b16a640 1 librados: shutdown 2026-02-17T06:00:24.129 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:06:02.956 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:24.131 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:06:02.958 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:24.139 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:06:02.966 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:24.155 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:06:02.981 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:24.156 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:06:02.982 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:24.157 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:06:02.946 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:24.158 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:06:02.947 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:24.165 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:06:02.991 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:24.166 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:06:02.955 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:26.633 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:26.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.703+0000 7f09e7c76640 10 monclient: get_monmap_and_config 2026-02-17T06:00:26.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.703+0000 7f09e7c76640 10 monclient: build_initial_monmap 2026-02-17T06:00:26.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.703+0000 7f09e7c76640 10 monclient: monmap: 2026-02-17T06:00:26.703 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:26.704424+0000 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:26.704424+0000 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.703+0000 7f09e7c76640 1 Processor -- start 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.703+0000 7f09e7c76640 1 -- start start 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.703+0000 7f09e7c76640 10 monclient: init 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 10 monclient: _add_conns ranks=[3,4,0,2,1,5] 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f09e00700f0 0x7f09e00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 10 monclient(hunting): picked mon.noname-a con 0x7f09e00700f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f09e00c7fe0 0x7f09e00c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 10 monclient(hunting): picked mon.noname-b con 0x7f09e00c7fe0 addr v2:10.20.193.128:3300/0 2026-02-17T06:00:26.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f09e01536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e00c8970 con 0x7f09e01536f0 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f09e00c4820 con 0x7f09e00700f0 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f09e00c4b00 con 0x7f09e00c7fe0 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.704+0000 7f09e7c76640 10 monclient(hunting): authenticate will time out at 1279.273478s 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e59eb640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f09e00700f0 0x7f09e00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e51ea640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f09e00c7fe0 0x7f09e00c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e61ec640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f09e01536f0 0x7f09e0153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:34732/0 (socket says 10.20.193.16:34732) 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e61ec640 1 -- 10.20.193.16:0/1892616050 learned_addr learned my addr 10.20.193.16:0/1892616050 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e51ea640 10 monclient(hunting): get_auth_request con 0x7f09e00c7fe0 auth_method 0 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e51ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e51ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e51ea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e59eb640 10 monclient(hunting): get_auth_request con 0x7f09e00700f0 auth_method 0 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e59eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e59eb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e59eb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e51ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e51ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:26.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e51ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:26.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e59eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:26.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e59eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:26.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.705+0000 7f09e59eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:26.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.706+0000 7f09e51ea640 10 monclient(hunting): handle_auth_done global_id 4951 payload 274 2026-02-17T06:00:26.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.706+0000 7f09e51ea640 1 -- 10.20.193.16:0/1892616050 >> v2:10.20.193.16:3300/0 conn(0x7f09e00700f0 msgr2=0x7f09e00704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:26.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.706+0000 7f09e51ea640 1 --2- 10.20.193.16:0/1892616050 >> v2:10.20.193.16:3300/0 conn(0x7f09e00700f0 0x7f09e00704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:26.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.706+0000 7f09e51ea640 1 -- 10.20.193.16:0/1892616050 >> v1:10.20.193.16:6789/0 conn(0x7f09e01536f0 legacy=0x7f09e0153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:26.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.706+0000 7f09e51ea640 10 monclient: _finish_hunting 0 2026-02-17T06:00:26.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.706+0000 7f09e51ea640 1 monclient: found mon.noname-b 2026-02-17T06:00:26.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.706+0000 7f09e51ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:26.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.706+0000 7f09e51ea640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:00:26.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.706+0000 7f09e51ea640 1 -- 10.20.193.16:0/1892616050 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e00c4de0 con 0x7f09e00c7fe0 2026-02-17T06:00:26.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.707+0000 7f09e51ea640 1 --2- 10.20.193.16:0/1892616050 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f09e00c7fe0 0x7f09e00c8430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f09e00c5b80 tx=0x7f09d402ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f60603a03345ad server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 1 -- 10.20.193.16:0/1892616050 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d403c070 con 0x7f09e00c7fe0 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 10 monclient: dump: 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 10 monclient: _finish_auth 0 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 1 -- 10.20.193.16:0/1892616050 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d402fa60 con 0x7f09e00c7fe0 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 5 monclient: authenticate success, global_id 4951 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 1 -- 10.20.193.16:0/1892616050 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d402fe30 con 0x7f09e00c7fe0 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 4 set_mon_vals no callback set 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e49e9640 10 monclient: dump: 2026-02-17T06:00:26.710 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 10 monclient: shutdown 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 1 -- 10.20.193.16:0/1892616050 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f09e00c7fe0 msgr2=0x7f09e00c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 1 --2- 10.20.193.16:0/1892616050 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f09e00c7fe0 0x7f09e00c8430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f09e00c5b80 tx=0x7f09d402ed50 comp rx=0 tx=0).stop 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 1 -- 10.20.193.16:0/1892616050 shutdown_connections 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 1 --2- 10.20.193.16:0/1892616050 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f09e00c7fe0 0x7f09e00c8430 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 1 --2- 10.20.193.16:0/1892616050 >> v2:10.20.193.16:3300/0 conn(0x7f09e00700f0 0x7f09e00704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 1 -- 10.20.193.16:0/1892616050 >> 10.20.193.16:0/1892616050 conn(0x7f09e00cd770 msgr2=0x7f09e00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 1 -- 10.20.193.16:0/1892616050 shutdown_connections 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.710+0000 7f09e7c76640 1 -- 10.20.193.16:0/1892616050 wait complete. 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient: build_initial_monmap 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient: monmap: 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:26.711 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 librados: starting msgr at 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 librados: starting objecter 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 Processor -- start 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 -- start start 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 librados: setting wanted keys 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 librados: calling monclient init 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient: init 2026-02-17T06:00:26.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f09e00700f0 0x7f09e01f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient(hunting): picked mon.noname-a con 0x7f09e00700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f09e00c7fe0 0x7f09e00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient(hunting): picked mon.noname-c con 0x7f09e00c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f09e01536f0 0x7f09e01e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient(hunting): picked mon.noname-b con 0x7f09e01536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09e00c5680 con 0x7f09e00700f0 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f09e00c53a0 con 0x7f09e01536f0 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0074c00 con 0x7f09e00c7fe0 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.711+0000 7f09e7c76640 10 monclient(hunting): authenticate will time out at 1279.280762s 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e59eb640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f09e00700f0 0x7f09e01f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e61ec640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f09e01536f0 0x7f09e01e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e59eb640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f09e00700f0 0x7f09e01f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:44364/0 (socket says 10.20.193.16:44364) 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e59eb640 1 -- 10.20.193.16:0/3737760729 learned_addr learned my addr 10.20.193.16:0/3737760729 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e59eb640 10 monclient(hunting): get_auth_request con 0x7f09e00700f0 auth_method 0 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e59eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e59eb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e59eb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e61ec640 10 monclient(hunting): get_auth_request con 0x7f09e01536f0 auth_method 0 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e61ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e61ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e61ec640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e51ea640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f09e00c7fe0 0x7f09e00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e59eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e59eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e59eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e61ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e61ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e61ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e51ea640 10 monclient(hunting): get_auth_request con 0x7f09e00c7fe0 auth_method 0 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e51ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e51ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:26.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.712+0000 7f09e51ea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e51ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e51ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e51ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e61ec640 10 monclient(hunting): handle_auth_done global_id 4954 payload 824 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e61ec640 1 -- 10.20.193.16:0/3737760729 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f09e00c7fe0 msgr2=0x7f09e00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e61ec640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f09e00c7fe0 0x7f09e00cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e61ec640 1 -- 10.20.193.16:0/3737760729 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f09e00700f0 msgr2=0x7f09e01f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e61ec640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f09e00700f0 0x7f09e01f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e61ec640 10 monclient: _finish_hunting 0 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e61ec640 1 monclient: found mon.noname-b 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e61ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e61ec640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:00:26.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.713+0000 7f09e61ec640 1 -- 10.20.193.16:0/3737760729 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e01e5a60 con 0x7f09e01536f0 2026-02-17T06:00:26.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.715+0000 7f09e61ec640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f09e01536f0 0x7f09e01e4f70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00cad0 tx=0x7f09dc00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:26.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.715+0000 7f09ce7fc640 10 client.?.objecter ms_handle_connect 0x7f09e01536f0 2026-02-17T06:00:26.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.715+0000 7f09ce7fc640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 1 -- 10.20.193.16:0/3737760729 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc017070 con 0x7f09e01536f0 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 10 monclient: dump: 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 10 monclient: _finish_auth 0 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 1 -- 10.20.193.16:0/3737760729 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09dc00ac10 con 0x7f09e01536f0 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09e7c76640 5 monclient: authenticate success, global_id 4954 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09e7c76640 10 monclient: _renew_subs 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09e7c76640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09e7c76640 1 -- 10.20.193.16:0/3737760729 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e01e5d40 con 0x7f09e01536f0 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 1 -- 10.20.193.16:0/3737760729 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc012400 con 0x7f09e01536f0 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:26.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09cffff640 4 set_mon_vals no callback set 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09ce7fc640 10 monclient: dump: 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09e7c76640 10 client.4954.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09e7c76640 10 monclient: _renew_subs 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09e7c76640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09e7c76640 1 -- 10.20.193.16:0/3737760729 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e01515e0 con 0x7f09e01536f0 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.716+0000 7f09e7c76640 1 librados: init done 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.717+0000 7f09e7c76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.717+0000 7f09e7c76640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.717+0000 7f09e7c76640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:26.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.717+0000 7f09e7c76640 1 -- 10.20.193.16:0/3737760729 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e00704d0 con 0x7f09e01536f0 2026-02-17T06:00:26.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.729+0000 7f09ce7fc640 1 -- 10.20.193.16:0/3737760729 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f09dc01b3b0 con 0x7f09e01536f0 2026-02-17T06:00:26.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.729+0000 7f09ce7fc640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f09b0079190 0x7f09b007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:26.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.729+0000 7f09e59eb640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f09b0079190 0x7f09b007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:26.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.729+0000 7f09e59eb640 10 monclient: get_auth_request con 0x7f09b0079190 auth_method 0 2026-02-17T06:00:26.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.730+0000 7f09ce7fc640 10 client.4954.objecter ms_handle_connect 0x7f09b0079190 2026-02-17T06:00:26.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.730+0000 7f09e59eb640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f09b0079190 0x7f09b007b650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009c60 tx=0x7f09d0009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:26.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.830+0000 7f09ce7fc640 1 -- 10.20.193.16:0/3737760729 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f09dc05de70 con 0x7f09e01536f0 2026-02-17T06:00:26.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.830+0000 7f09ce7fc640 10 client.4954.objecter ms_dispatch2 0x7f09e0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:26.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.830+0000 7f09ce7fc640 3 client.4954.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:26.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.830+0000 7f09ce7fc640 3 client.4954.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:26.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.830+0000 7f09ce7fc640 20 client.4954.objecter dump_active .. 0 homeless 2026-02-17T06:00:26.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.836+0000 7f09ce7fc640 1 -- 10.20.193.16:0/3737760729 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f09dc021020 con 0x7f09e01536f0 2026-02-17T06:00:26.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.836+0000 7f09ce7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:26.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.836+0000 7f09ce7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:26.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.939+0000 7f09e7c76640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:26.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.939+0000 7f09e7c76640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:26.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.939+0000 7f09e7c76640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:26.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.939+0000 7f09e7c76640 1 -- 10.20.193.16:0/3737760729 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f09e01573f0 con 0x7f09e01536f0 2026-02-17T06:00:26.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.941+0000 7f09ce7fc640 1 -- 10.20.193.16:0/3737760729 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f09dc061e30 con 0x7f09e01536f0 2026-02-17T06:00:26.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.941+0000 7f09ce7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:26.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.941+0000 7f09ce7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:26.942 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:26.942 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:26.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 10 librados: watch_flush enter 2026-02-17T06:00:26.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 10 librados: watch_flush exit 2026-02-17T06:00:26.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 20 client.4954.objecter shutdown clearing up homeless session... 2026-02-17T06:00:26.944 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 10 client.4954.objecter successfully canceled tick 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 -- 10.20.193.16:0/3737760729 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f09b0079190 msgr2=0x7f09b007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f09b0079190 0x7f09b007b650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009c60 tx=0x7f09d0009290 comp rx=0 tx=0).stop 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 10 monclient: shutdown 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 -- 10.20.193.16:0/3737760729 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f09e01536f0 msgr2=0x7f09e01e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f09e01536f0 0x7f09e01e4f70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00cad0 tx=0x7f09dc00a060 comp rx=0 tx=0).stop 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 -- 10.20.193.16:0/3737760729 shutdown_connections 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f09b0079190 0x7f09b007b650 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f09e01536f0 0x7f09e01e4f70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f09e00c7fe0 0x7f09e00cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:26.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 --2- 10.20.193.16:0/3737760729 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f09e00700f0 0x7f09e01f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:26.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 -- 10.20.193.16:0/3737760729 >> 10.20.193.16:0/3737760729 conn(0x7f09e00cd770 msgr2=0x7f09e01446f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:26.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 -- 10.20.193.16:0/3737760729 shutdown_connections 2026-02-17T06:00:26.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 -- 10.20.193.16:0/3737760729 wait complete. 2026-02-17T06:00:26.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:26.942+0000 7f09e7c76640 1 librados: shutdown 2026-02-17T06:00:29.952 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.019+0000 7f779940e640 10 monclient: get_monmap_and_config 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.019+0000 7f779940e640 10 monclient: build_initial_monmap 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.019+0000 7f779940e640 10 monclient: monmap: 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:30.020286+0000 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:30.020286+0000 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:30.020 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:30.021 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:30.021 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:30.021 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:30.021 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:30.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.019+0000 7f779940e640 1 Processor -- start 2026-02-17T06:00:30.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.019+0000 7f779940e640 1 -- start start 2026-02-17T06:00:30.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.019+0000 7f779940e640 10 monclient: init 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 10 monclient: _add_conns ranks=[5,1,4,0,3,2] 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f77940700f0 0x7f77940704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 10 monclient(hunting): picked mon.noname-c con 0x7f77940700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f77940c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f77941536f0 0x7f7794153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 10 monclient(hunting): picked mon.noname-b con 0x7f77941536f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77940c4820 con 0x7f77940c7fe0 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f77940c4b00 con 0x7f77941536f0 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:30.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f77940c4de0 con 0x7f77940700f0 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f779940e640 10 monclient(hunting): authenticate will time out at 1282.589235s 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f7792ffd640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f77940700f0 0x7f77940704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f7792ffd640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f77940700f0 0x7f77940704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:51500/0 (socket says 10.20.193.16:51500) 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f7792ffd640 1 -- 10.20.193.16:0/3322906349 learned_addr learned my addr 10.20.193.16:0/3322906349 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f7792ffd640 10 monclient(hunting): get_auth_request con 0x7f77940700f0 auth_method 0 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f7792ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f7792ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f7792ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.020+0000 7f77937fe640 1 --2- 10.20.193.16:0/3322906349 >> v2:10.20.193.128:3300/0 conn(0x7f77941536f0 0x7f7794153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.021+0000 7f77937fe640 10 monclient(hunting): get_auth_request con 0x7f77941536f0 auth_method 0 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.021+0000 7f77937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.021+0000 7f77937fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.021+0000 7f77937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.021+0000 7f7792ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.021+0000 7f7792ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.021+0000 7f7792ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.021+0000 7f77937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.021+0000 7f77937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:30.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.021+0000 7f77937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:30.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.022+0000 7f7792ffd640 10 monclient(hunting): handle_auth_done global_id 5018 payload 274 2026-02-17T06:00:30.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.022+0000 7f7792ffd640 1 -- 10.20.193.16:0/3322906349 >> v2:10.20.193.128:3300/0 conn(0x7f77941536f0 msgr2=0x7f7794153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:30.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.022+0000 7f7792ffd640 1 --2- 10.20.193.16:0/3322906349 >> v2:10.20.193.128:3300/0 conn(0x7f77941536f0 0x7f7794153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:30.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.022+0000 7f7792ffd640 1 -- 10.20.193.16:0/3322906349 >> v1:10.20.193.128:6789/0 conn(0x7f77940c7fe0 legacy=0x7f77940c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:30.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.022+0000 7f7792ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:00:30.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.022+0000 7f7792ffd640 1 monclient: found mon.noname-c 2026-02-17T06:00:30.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.022+0000 7f7792ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:30.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.022+0000 7f7792ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:00:30.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.022+0000 7f7792ffd640 1 -- 10.20.193.16:0/3322906349 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77940c50c0 con 0x7f77940700f0 2026-02-17T06:00:30.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.023+0000 7f7792ffd640 1 --2- 10.20.193.16:0/3322906349 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f77940700f0 0x7f77940704d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7794070b90 tx=0x7f778802ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99e961a4ea9bec0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:30.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 1 -- 10.20.193.16:0/3322906349 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778803c070 con 0x7f77940700f0 2026-02-17T06:00:30.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 10 monclient: dump: 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 10 monclient: _finish_auth 0 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 5 monclient: authenticate success, global_id 5018 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 1 -- 10.20.193.16:0/3322906349 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778802fb30 con 0x7f77940700f0 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 1 -- 10.20.193.16:0/3322906349 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788037440 con 0x7f77940700f0 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f7791ffb640 10 monclient: dump: 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:30.027 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 4 set_mon_vals no callback set 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 10 monclient: shutdown 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 1 -- 10.20.193.16:0/3322906349 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f77940700f0 msgr2=0x7f77940704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 1 --2- 10.20.193.16:0/3322906349 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f77940700f0 0x7f77940704d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7794070b90 tx=0x7f778802ee60 comp rx=0 tx=0).stop 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 1 -- 10.20.193.16:0/3322906349 shutdown_connections 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 1 --2- 10.20.193.16:0/3322906349 >> v2:10.20.193.128:3300/0 conn(0x7f77941536f0 0x7f7794153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 1 --2- 10.20.193.16:0/3322906349 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f77940700f0 0x7f77940704d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 1 -- 10.20.193.16:0/3322906349 >> 10.20.193.16:0/3322906349 conn(0x7f77940cd770 msgr2=0x7f77940cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 1 -- 10.20.193.16:0/3322906349 shutdown_connections 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.025+0000 7f779940e640 1 -- 10.20.193.16:0/3322906349 wait complete. 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient: build_initial_monmap 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient: monmap: 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:30.028 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:30.029 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:30.029 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:30.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 librados: starting msgr at 2026-02-17T06:00:30.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 librados: starting objecter 2026-02-17T06:00:30.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 Processor -- start 2026-02-17T06:00:30.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 -- start start 2026-02-17T06:00:30.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 librados: setting wanted keys 2026-02-17T06:00:30.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 librados: calling monclient init 2026-02-17T06:00:30.029 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient: init 2026-02-17T06:00:30.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:30.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:00:30.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f77940700f0 0x7f77941f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:30.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient(hunting): picked mon.noname-b con 0x7f77940700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:30.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f77940c7fe0 0x7f779406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:30.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient(hunting): picked mon.noname-c con 0x7f77940c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:30.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f77941536f0 0x7f77940cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:30.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient(hunting): picked mon.noname-a con 0x7f77941536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:30.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:30.030 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7794154060 con 0x7f77941536f0 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f77940c5680 con 0x7f77940700f0 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7794074c00 con 0x7f77940c7fe0 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.026+0000 7f779940e640 10 monclient(hunting): authenticate will time out at 1282.595740s 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f7792ffd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f77940700f0 0x7f77941f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77937fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f77941536f0 0x7f77940cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77927fc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f77940c7fe0 0x7f779406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f7792ffd640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f77940700f0 0x7f77941f4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:41418/0 (socket says 10.20.193.16:41418) 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f7792ffd640 1 -- 10.20.193.16:0/939835998 learned_addr learned my addr 10.20.193.16:0/939835998 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f7792ffd640 10 monclient(hunting): get_auth_request con 0x7f77940700f0 auth_method 0 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f7792ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f7792ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f7792ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77937fe640 10 monclient(hunting): get_auth_request con 0x7f77941536f0 auth_method 0 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77937fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77927fc640 10 monclient(hunting): get_auth_request con 0x7f77940c7fe0 auth_method 0 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77927fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77927fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f7792ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f7792ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f7792ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77927fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77927fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:30.031 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77927fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.027+0000 7f77937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.028+0000 7f77927fc640 10 monclient(hunting): handle_auth_done global_id 5021 payload 824 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.028+0000 7f77927fc640 1 -- 10.20.193.16:0/939835998 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f77940700f0 msgr2=0x7f77941f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.028+0000 7f77927fc640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f77940700f0 0x7f77941f4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.028+0000 7f77927fc640 1 -- 10.20.193.16:0/939835998 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f77941536f0 msgr2=0x7f77940cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.028+0000 7f77927fc640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f77941536f0 0x7f77940cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.028+0000 7f77927fc640 10 monclient: _finish_hunting 0 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.028+0000 7f77927fc640 1 monclient: found mon.noname-c 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.028+0000 7f77927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.028+0000 7f77927fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.028+0000 7f77927fc640 1 -- 10.20.193.16:0/939835998 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779406d760 con 0x7f77940c7fe0 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.029+0000 7f77927fc640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f77940c7fe0 0x7f779406ebc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f777c00c480 tx=0x7f777c00c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.029+0000 7f77737fe640 10 client.?.objecter ms_handle_connect 0x7f77940c7fe0 2026-02-17T06:00:30.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.029+0000 7f77737fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 1 -- 10.20.193.16:0/939835998 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c017070 con 0x7f77940c7fe0 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 10 monclient: dump: 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 1 -- 10.20.193.16:0/939835998 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f777c0052d0 con 0x7f77940c7fe0 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 1 -- 10.20.193.16:0/939835998 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c0036f0 con 0x7f77940c7fe0 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77737fe640 10 monclient: dump: 2026-02-17T06:00:30.033 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f77917fa640 4 set_mon_vals no callback set 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 5 monclient: authenticate success, global_id 5021 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 10 monclient: _renew_subs 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 1 -- 10.20.193.16:0/939835998 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779406da40 con 0x7f77940c7fe0 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 10 client.5021.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 10 monclient: _renew_subs 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 1 -- 10.20.193.16:0/939835998 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779406e000 con 0x7f77940c7fe0 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 1 librados: init done 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:30.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.031+0000 7f779940e640 1 -- 10.20.193.16:0/939835998 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77940704d0 con 0x7f77940c7fe0 2026-02-17T06:00:30.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.043+0000 7f77737fe640 1 -- 10.20.193.16:0/939835998 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f777c003890 con 0x7f77940c7fe0 2026-02-17T06:00:30.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.043+0000 7f77737fe640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7760079140 0x7f776007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:30.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.043+0000 7f7792ffd640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7760079140 0x7f776007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:30.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.043+0000 7f7792ffd640 10 monclient: get_auth_request con 0x7f7760079140 auth_method 0 2026-02-17T06:00:30.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.044+0000 7f7792ffd640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7760079140 0x7f776007b600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f77880091b0 tx=0x7f7788039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:30.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.044+0000 7f77737fe640 10 client.5021.objecter ms_handle_connect 0x7f7760079140 2026-02-17T06:00:30.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.044+0000 7f77737fe640 1 -- 10.20.193.16:0/939835998 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f777c05d3c0 con 0x7f77940c7fe0 2026-02-17T06:00:30.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.044+0000 7f77737fe640 10 client.5021.objecter ms_dispatch2 0x7f7794000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:30.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.044+0000 7f77737fe640 3 client.5021.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:30.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.044+0000 7f77737fe640 3 client.5021.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:30.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.044+0000 7f77737fe640 20 client.5021.objecter dump_active .. 0 homeless 2026-02-17T06:00:30.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.143+0000 7f77737fe640 1 -- 10.20.193.16:0/939835998 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f777c021080 con 0x7f77940c7fe0 2026-02-17T06:00:30.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.143+0000 7f77737fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:30.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.143+0000 7f77737fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:30.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.245+0000 7f779940e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:30.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.245+0000 7f779940e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:30.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.245+0000 7f779940e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:30.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.245+0000 7f779940e640 1 -- 10.20.193.16:0/939835998 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f77941573f0 con 0x7f77940c7fe0 2026-02-17T06:00:30.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.248+0000 7f77737fe640 1 -- 10.20.193.16:0/939835998 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f777c061380 con 0x7f77940c7fe0 2026-02-17T06:00:30.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.248+0000 7f77737fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:30.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.248+0000 7f77737fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:30.249 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:30.249 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 10 librados: watch_flush enter 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 10 librados: watch_flush exit 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 20 client.5021.objecter shutdown clearing up homeless session... 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 10 client.5021.objecter successfully canceled tick 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 -- 10.20.193.16:0/939835998 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7760079140 msgr2=0x7f776007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7760079140 0x7f776007b600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f77880091b0 tx=0x7f7788039040 comp rx=0 tx=0).stop 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 10 monclient: shutdown 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 -- 10.20.193.16:0/939835998 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f77940c7fe0 msgr2=0x7f779406ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f77940c7fe0 0x7f779406ebc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f777c00c480 tx=0x7f777c00c950 comp rx=0 tx=0).stop 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 -- 10.20.193.16:0/939835998 shutdown_connections 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7760079140 0x7f776007b600 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f77941536f0 0x7f77940cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f77940c7fe0 0x7f779406ebc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:30.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 --2- 10.20.193.16:0/939835998 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f77940700f0 0x7f77941f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:30.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 -- 10.20.193.16:0/939835998 >> 10.20.193.16:0/939835998 conn(0x7f77940cd770 msgr2=0x7f77940cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:30.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 -- 10.20.193.16:0/939835998 shutdown_connections 2026-02-17T06:00:30.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 -- 10.20.193.16:0/939835998 wait complete. 2026-02-17T06:00:30.251 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:30.249+0000 7f779940e640 1 librados: shutdown 2026-02-17T06:00:33.259 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:33.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.326+0000 7fcb6a5de640 10 monclient: get_monmap_and_config 2026-02-17T06:00:33.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.326+0000 7fcb6a5de640 10 monclient: build_initial_monmap 2026-02-17T06:00:33.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.326+0000 7fcb6a5de640 10 monclient: monmap: 2026-02-17T06:00:33.327 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:33.327 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:33.327 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:33.327382+0000 2026-02-17T06:00:33.327 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:33.327382+0000 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.326+0000 7fcb6a5de640 1 Processor -- start 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.326+0000 7fcb6a5de640 1 -- start start 2026-02-17T06:00:33.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.326+0000 7fcb6a5de640 10 monclient: init 2026-02-17T06:00:33.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb6a5de640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:33.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb6a5de640 10 monclient: _add_conns ranks=[1,0,2,5,3,4] 2026-02-17T06:00:33.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb6a5de640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fcb640700f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb6a5de640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcb640c86f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb6a5de640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcb64153e00 addr v1:10.20.193.148:6789/0 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb6a5de640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb640c4820 con 0x7fcb640c86f0 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb6a5de640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb640c4b00 con 0x7fcb640700f0 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb6a5de640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb640c4de0 con 0x7fcb64153e00 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb6a5de640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb6a5de640 10 monclient(hunting): authenticate will time out at 1285.896338s 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb637fe640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fcb640c86f0 0x7fcb641536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:42952/0 (socket says 10.20.193.16:42952) 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.327+0000 7fcb637fe640 1 -- 10.20.193.16:0/334156816 learned_addr learned my addr 10.20.193.16:0/334156816 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.328+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701500091 0 0) 0x7fcb640c4820 con 0x7fcb640c86f0 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.328+0000 7fcb62ffd640 10 monclient(hunting): _negotiate 2026-02-17T06:00:33.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.328+0000 7fcb62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.328+0000 7fcb62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.328+0000 7fcb62ffd640 10 monclient(hunting): my global_id is 4890 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.328+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb48003620 con 0x7fcb640c86f0 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3129089686 0 0) 0x7fcb640c4de0 con 0x7fcb64153e00 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 10 monclient(hunting): _negotiate 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 10 monclient(hunting): my global_id is 5024 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb640c4820 con 0x7fcb64153e00 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1636647694 0 0) 0x7fcb640c4b00 con 0x7fcb640700f0 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 10 monclient(hunting): _negotiate 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:33.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:33.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 10 monclient(hunting): my global_id is 4966 2026-02-17T06:00:33.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.329+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb640c4de0 con 0x7fcb640700f0 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 <== mon.0 v1:10.20.193.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 939330270 0 0) 0x7fcb48003620 con 0x7fcb640c86f0 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 >> v1:10.20.193.148:6789/0 conn(0x7fcb64153e00 legacy=0x7fcb64164350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 >> v1:10.20.193.128:6789/0 conn(0x7fcb640700f0 legacy=0x7fcb640c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.16:6789/0 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 --> v1:10.20.193.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb640c50c0 con 0x7fcb640c86f0 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 <== mon.0 v1:10.20.193.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7fcb54002f00 con 0x7fcb640c86f0 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient: dump: 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:33.340 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] but i'm connected to v1:10.20.193.16:6789/0, reconnecting 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 >> v1:10.20.193.16:6789/0 conn(0x7fcb640c86f0 legacy=0x7fcb641536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48003fe0 0x7fcb48004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient(hunting): picked mon.a con 0x7fcb48003fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb48004e40 0x7fcb480052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient(hunting): picked mon.c con 0x7fcb48004e40 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb48009ab0 0x7fcb4800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient(hunting): picked mon.b con 0x7fcb48009ab0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4800c5c0 con 0x7fcb48003fe0 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4800c770 con 0x7fcb48009ab0 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:33.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4800c970 con 0x7fcb48004e40 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.330+0000 7fcb62ffd640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.331+0000 7fcb637fe640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb48004e40 0x7fcb480052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.331+0000 7fcb637fe640 10 monclient(hunting): get_auth_request con 0x7fcb48004e40 auth_method 0 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.331+0000 7fcb637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.331+0000 7fcb637fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.331+0000 7fcb637fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.331+0000 7fcb68b54640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb48009ab0 0x7fcb4800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb68b54640 10 monclient(hunting): get_auth_request con 0x7fcb48009ab0 auth_method 0 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb68b54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb68b54640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb68b54640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb63fff640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48003fe0 0x7fcb48004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb63fff640 10 monclient(hunting): get_auth_request con 0x7fcb48003fe0 auth_method 0 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb63fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb63fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb68b54640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb68b54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb68b54640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb63fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:00:33.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 10 monclient(hunting): handle_auth_done global_id 4890 payload 293 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 1 -- 10.20.193.16:0/334156816 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb48009ab0 msgr2=0x7fcb4800bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb48009ab0 0x7fcb4800bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 1 -- 10.20.193.16:0/334156816 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48003fe0 msgr2=0x7fcb48004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48003fe0 0x7fcb48004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 10 monclient: _finish_hunting 0 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 1 monclient: found mon.c 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.332+0000 7fcb637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.333+0000 7fcb637fe640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.333+0000 7fcb637fe640 1 -- 10.20.193.16:0/334156816 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb48003620 con 0x7fcb48004e40 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.333+0000 7fcb637fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.333+0000 7fcb637fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.333+0000 7fcb637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.333+0000 7fcb68b54640 1 -- 10.20.193.16:0/334156816 reap_dead start 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.333+0000 7fcb6a5de640 5 monclient: authenticate success, global_id 4890 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.333+0000 7fcb6a5de640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.333+0000 7fcb637fe640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb48004e40 0x7fcb480052c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c240 tx=0x7fcb5402f530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce7b79455ec302e1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54007d20 con 0x7fcb48004e40 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb62ffd640 10 monclient: dump: 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:33.343 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb54005bf0 con 0x7fcb48004e40 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb6a5de640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb62ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb62ffd640 1 -- 10.20.193.16:0/334156816 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54007070 con 0x7fcb48004e40 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb62ffd640 10 monclient: dump: 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb6a5de640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.335+0000 7fcb6a5de640 4 set_mon_vals no callback set 2026-02-17T06:00:33.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 10 monclient: shutdown 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 1 -- 10.20.193.16:0/334156816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb48004e40 msgr2=0x7fcb480052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb48004e40 0x7fcb480052c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c240 tx=0x7fcb5402f530 comp rx=0 tx=0).stop 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 1 -- 10.20.193.16:0/334156816 shutdown_connections 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 1 --2- 10.20.193.16:0/334156816 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb48004e40 0x7fcb480052c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 1 -- 10.20.193.16:0/334156816 >> 10.20.193.16:0/334156816 conn(0x7fcb640cd770 msgr2=0x7fcb640cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 1 -- 10.20.193.16:0/334156816 shutdown_connections 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 1 -- 10.20.193.16:0/334156816 wait complete. 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 10 monclient: build_initial_monmap 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 10 monclient: monmap: 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 1 librados: starting msgr at 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 1 librados: starting objecter 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.336+0000 7fcb6a5de640 1 Processor -- start 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 1 -- start start 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 1 librados: setting wanted keys 2026-02-17T06:00:33.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 1 librados: calling monclient init 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient: init 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48004e40 0x7fcb6413e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient(hunting): picked mon.noname-a con 0x7fcb48004e40 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb6413cad0 0x7fcb640cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient(hunting): picked mon.noname-b con 0x7fcb6413cad0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb6413cf30 0x7fcb640cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient(hunting): picked mon.noname-c con 0x7fcb6413cf30 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb64074c00 con 0x7fcb48004e40 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fcb640c5680 con 0x7fcb6413cad0 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcb64074d80 con 0x7fcb6413cf30 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb6a5de640 10 monclient(hunting): authenticate will time out at 1285.906424s 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb63fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48004e40 0x7fcb6413e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb63fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48004e40 0x7fcb6413e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:48850/0 (socket says 10.20.193.16:48850) 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb63fff640 1 -- 10.20.193.16:0/1850574953 learned_addr learned my addr 10.20.193.16:0/1850574953 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb63fff640 10 monclient(hunting): get_auth_request con 0x7fcb48004e40 auth_method 0 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb63fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb63fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:33.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb68b54640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb6413cf30 0x7fcb640cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb637fe640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb6413cad0 0x7fcb640cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb68b54640 10 monclient(hunting): get_auth_request con 0x7fcb6413cf30 auth_method 0 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb68b54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb68b54640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb68b54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.337+0000 7fcb63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb637fe640 10 monclient(hunting): get_auth_request con 0x7fcb6413cad0 auth_method 0 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb637fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb68b54640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb68b54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb68b54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.338+0000 7fcb637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.339+0000 7fcb63fff640 10 monclient(hunting): handle_auth_done global_id 4893 payload 824 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.339+0000 7fcb63fff640 1 -- 10.20.193.16:0/1850574953 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb6413cf30 msgr2=0x7fcb640cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.339+0000 7fcb63fff640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb6413cf30 0x7fcb640cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.339+0000 7fcb63fff640 1 -- 10.20.193.16:0/1850574953 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb6413cad0 msgr2=0x7fcb640cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.339+0000 7fcb63fff640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb6413cad0 0x7fcb640cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.339+0000 7fcb63fff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.339+0000 7fcb63fff640 1 monclient: found mon.noname-a 2026-02-17T06:00:33.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.339+0000 7fcb63fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:33.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.339+0000 7fcb63fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:33.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.339+0000 7fcb63fff640 1 -- 10.20.193.16:0/1850574953 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb640cb2e0 con 0x7fcb48004e40 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.340+0000 7fcb63fff640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48004e40 0x7fcb6413e9d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000eb10 tx=0x7fcb5000c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.340+0000 7fcb60ff9640 10 client.?.objecter ms_handle_connect 0x7fcb48004e40 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.340+0000 7fcb60ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 1 -- 10.20.193.16:0/1850574953 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000ca30 con 0x7fcb48004e40 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 10 monclient: dump: 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:33.349 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 10 monclient: _finish_auth 0 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 5 monclient: authenticate success, global_id 4893 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 10 monclient: _renew_subs 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 1 -- 10.20.193.16:0/1850574953 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6413f0c0 con 0x7fcb48004e40 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 10 client.4893.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 10 monclient: _renew_subs 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 1 -- 10.20.193.16:0/1850574953 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6413f680 con 0x7fcb48004e40 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 1 librados: init done 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb6a5de640 1 -- 10.20.193.16:0/1850574953 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb640704d0 con 0x7fcb48004e40 2026-02-17T06:00:33.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 1 -- 10.20.193.16:0/1850574953 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5001a070 con 0x7fcb48004e40 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 1 -- 10.20.193.16:0/1850574953 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000cdd0 con 0x7fcb48004e40 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb627fc640 4 set_mon_vals no callback set 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.341+0000 7fcb60ff9640 10 monclient: dump: 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:33.351 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:33.450 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.450+0000 7fcb60ff9640 1 -- 10.20.193.16:0/1850574953 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fcb50014070 con 0x7fcb48004e40 2026-02-17T06:00:33.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.450+0000 7fcb60ff9640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb34079140 0x7fcb3407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:33.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.450+0000 7fcb637fe640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb34079140 0x7fcb3407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:33.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.450+0000 7fcb637fe640 10 monclient: get_auth_request con 0x7fcb34079140 auth_method 0 2026-02-17T06:00:33.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.451+0000 7fcb60ff9640 10 client.4893.objecter ms_handle_connect 0x7fcb34079140 2026-02-17T06:00:33.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.451+0000 7fcb637fe640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb34079140 0x7fcb3407b600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcb54036000 tx=0x7fcb54009070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:33.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.458+0000 7fcb60ff9640 1 -- 10.20.193.16:0/1850574953 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fcb5005ea60 con 0x7fcb48004e40 2026-02-17T06:00:33.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.458+0000 7fcb60ff9640 10 client.4893.objecter ms_dispatch2 0x7fcb64000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:33.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.458+0000 7fcb60ff9640 3 client.4893.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:33.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.458+0000 7fcb60ff9640 3 client.4893.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:33.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.458+0000 7fcb60ff9640 20 client.4893.objecter dump_active .. 0 homeless 2026-02-17T06:00:33.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.458+0000 7fcb60ff9640 1 -- 10.20.193.16:0/1850574953 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcb50062a20 con 0x7fcb48004e40 2026-02-17T06:00:33.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.458+0000 7fcb60ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:33.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.458+0000 7fcb60ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:33.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.557+0000 7fcb6a5de640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:33.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.557+0000 7fcb6a5de640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:33.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.557+0000 7fcb6a5de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:33.558 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.557+0000 7fcb6a5de640 1 -- 10.20.193.16:0/1850574953 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcb64146860 con 0x7fcb48004e40 2026-02-17T06:00:33.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.561+0000 7fcb60ff9640 1 -- 10.20.193.16:0/1850574953 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fcb50066d20 con 0x7fcb48004e40 2026-02-17T06:00:33.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.561+0000 7fcb60ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:33.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.561+0000 7fcb60ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:33.562 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:33.562 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:33.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.562+0000 7fcb6a5de640 10 librados: watch_flush enter 2026-02-17T06:00:33.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.562+0000 7fcb6a5de640 10 librados: watch_flush exit 2026-02-17T06:00:33.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.562+0000 7fcb6a5de640 20 client.4893.objecter shutdown clearing up homeless session... 2026-02-17T06:00:33.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.562+0000 7fcb6a5de640 10 client.4893.objecter successfully canceled tick 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 -- 10.20.193.16:0/1850574953 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb34079140 msgr2=0x7fcb3407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb34079140 0x7fcb3407b600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcb54036000 tx=0x7fcb54009070 comp rx=0 tx=0).stop 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 10 monclient: shutdown 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 -- 10.20.193.16:0/1850574953 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48004e40 msgr2=0x7fcb6413e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48004e40 0x7fcb6413e9d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000eb10 tx=0x7fcb5000c0e0 comp rx=0 tx=0).stop 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 -- 10.20.193.16:0/1850574953 shutdown_connections 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcb34079140 0x7fcb3407b600 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcb6413cad0 0x7fcb640cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcb6413cf30 0x7fcb640cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 --2- 10.20.193.16:0/1850574953 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcb48004e40 0x7fcb6413e9d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 -- 10.20.193.16:0/1850574953 >> 10.20.193.16:0/1850574953 conn(0x7fcb640cd770 msgr2=0x7fcb640cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 -- 10.20.193.16:0/1850574953 shutdown_connections 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 -- 10.20.193.16:0/1850574953 wait complete. 2026-02-17T06:00:33.564 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:33.563+0000 7fcb6a5de640 1 librados: shutdown 2026-02-17T06:00:36.573 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:36.643 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.642+0000 7fcc42afa640 10 monclient: get_monmap_and_config 2026-02-17T06:00:36.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.642+0000 7fcc42afa640 10 monclient: build_initial_monmap 2026-02-17T06:00:36.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.642+0000 7fcc42afa640 10 monclient: monmap: 2026-02-17T06:00:36.644 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:36.644 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:36.644 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:36.643662+0000 2026-02-17T06:00:36.644 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:36.643662+0000 2026-02-17T06:00:36.644 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:36.644 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:36.645 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:36.645 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:36.645 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:36.645 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:36.645 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:36.645 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:36.645 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:36.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 1 Processor -- start 2026-02-17T06:00:36.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 1 -- start start 2026-02-17T06:00:36.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient: init 2026-02-17T06:00:36.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:36.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient: _add_conns ranks=[5,4,3,0,2,1] 2026-02-17T06:00:36.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fcc3c0700f0 0x7fcc3c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:36.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient(hunting): picked mon.noname-c con 0x7fcc3c0700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:00:36.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fcc3c0c7fe0 0x7fcc3c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:36.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient(hunting): picked mon.noname-b con 0x7fcc3c0c7fe0 addr v2:10.20.193.128:3300/0 2026-02-17T06:00:36.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fcc3c1536f0 0x7fcc3c153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient(hunting): picked mon.noname-a con 0x7fcc3c1536f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3c0c8970 con 0x7fcc3c1536f0 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3c0c4820 con 0x7fcc3c0c7fe0 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3c0c4b00 con 0x7fcc3c0700f0 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.643+0000 7fcc42afa640 10 monclient(hunting): authenticate will time out at 1289.212749s 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc4086f640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fcc3c0700f0 0x7fcc3c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:36.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc33fff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fcc3c0c7fe0 0x7fcc3c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc4086f640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fcc3c0700f0 0x7fcc3c0704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:50380/0 (socket says 10.20.193.16:50380) 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc4086f640 1 -- 10.20.193.16:0/2676463055 learned_addr learned my addr 10.20.193.16:0/2676463055 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc4086f640 10 monclient(hunting): get_auth_request con 0x7fcc3c0700f0 auth_method 0 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc4086f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc4086f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc4086f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc33fff640 10 monclient(hunting): get_auth_request con 0x7fcc3c0c7fe0 auth_method 0 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc33fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc4086f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:36.651 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc4086f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc4086f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.644+0000 7fcc33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.645+0000 7fcc33fff640 10 monclient(hunting): handle_auth_done global_id 4972 payload 274 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.645+0000 7fcc33fff640 1 -- 10.20.193.16:0/2676463055 >> v2:10.20.193.148:3300/0 conn(0x7fcc3c0700f0 msgr2=0x7fcc3c0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.645+0000 7fcc33fff640 1 --2- 10.20.193.16:0/2676463055 >> v2:10.20.193.148:3300/0 conn(0x7fcc3c0700f0 0x7fcc3c0704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.645+0000 7fcc33fff640 1 -- 10.20.193.16:0/2676463055 >> v2:10.20.193.16:3300/0 conn(0x7fcc3c1536f0 msgr2=0x7fcc3c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.645+0000 7fcc33fff640 1 --2- 10.20.193.16:0/2676463055 >> v2:10.20.193.16:3300/0 conn(0x7fcc3c1536f0 0x7fcc3c153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.645+0000 7fcc33fff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.645+0000 7fcc33fff640 1 monclient: found mon.noname-b 2026-02-17T06:00:36.652 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.645+0000 7fcc33fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.645+0000 7fcc33fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.645+0000 7fcc33fff640 1 -- 10.20.193.16:0/2676463055 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c0c4de0 con 0x7fcc3c0c7fe0 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.647+0000 7fcc33fff640 1 --2- 10.20.193.16:0/2676463055 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcc3c0c7fe0 0x7fcc3c0c8430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0c5b80 tx=0x7fcc2402ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85324afe3cebfb97 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.648+0000 7fcc337fe640 1 -- 10.20.193.16:0/2676463055 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2403c070 con 0x7fcc3c0c7fe0 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.648+0000 7fcc337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.648+0000 7fcc337fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.648+0000 7fcc337fe640 10 monclient: dump: 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:36.653 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:36.654 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:36.654 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:36.654 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:36.654 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:36.654 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:36.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.648+0000 7fcc337fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:36.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.648+0000 7fcc337fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:36.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.648+0000 7fcc337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:36.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.648+0000 7fcc42afa640 5 monclient: authenticate success, global_id 4972 2026-02-17T06:00:36.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.648+0000 7fcc42afa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:36.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc337fe640 1 -- 10.20.193.16:0/2676463055 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2402f9f0 con 0x7fcc3c0c7fe0 2026-02-17T06:00:36.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc337fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:36.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc337fe640 1 -- 10.20.193.16:0/2676463055 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2402fdc0 con 0x7fcc3c0c7fe0 2026-02-17T06:00:36.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:36.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:36.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 4 set_mon_vals no callback set 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc337fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc337fe640 10 monclient: dump: 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:36.662 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 10 monclient: shutdown 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 1 -- 10.20.193.16:0/2676463055 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcc3c0c7fe0 msgr2=0x7fcc3c0c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 1 --2- 10.20.193.16:0/2676463055 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcc3c0c7fe0 0x7fcc3c0c8430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0c5b80 tx=0x7fcc2402ed30 comp rx=0 tx=0).stop 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 1 -- 10.20.193.16:0/2676463055 shutdown_connections 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 1 --2- 10.20.193.16:0/2676463055 >> v2:10.20.193.16:3300/0 conn(0x7fcc3c1536f0 0x7fcc3c153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 1 --2- 10.20.193.16:0/2676463055 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcc3c0c7fe0 0x7fcc3c0c8430 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 1 --2- 10.20.193.16:0/2676463055 >> v2:10.20.193.148:3300/0 conn(0x7fcc3c0700f0 0x7fcc3c0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 1 -- 10.20.193.16:0/2676463055 >> 10.20.193.16:0/2676463055 conn(0x7fcc3c0cd770 msgr2=0x7fcc3c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 1 -- 10.20.193.16:0/2676463055 shutdown_connections 2026-02-17T06:00:36.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 1 -- 10.20.193.16:0/2676463055 wait complete. 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 10 monclient: build_initial_monmap 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.649+0000 7fcc42afa640 10 monclient: monmap: 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:36.664 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 librados: starting msgr at 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 librados: starting objecter 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 Processor -- start 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 -- start start 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 librados: setting wanted keys 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 librados: calling monclient init 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient: init 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcc3c0700f0 0x7fcc3c0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient(hunting): picked mon.noname-b con 0x7fcc3c0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcc3c0c7fe0 0x7fcc3c1f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:36.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient(hunting): picked mon.noname-c con 0x7fcc3c0c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcc3c1536f0 0x7fcc3c1f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient(hunting): picked mon.noname-a con 0x7fcc3c1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c0c5680 con 0x7fcc3c1536f0 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c0c53a0 con 0x7fcc3c0700f0 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c154040 con 0x7fcc3c0c7fe0 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc42afa640 10 monclient(hunting): authenticate will time out at 1289.219597s 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc41070640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcc3c1536f0 0x7fcc3c1f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:36.666 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.650+0000 7fcc4086f640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcc3c0700f0 0x7fcc3c0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc33fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcc3c0c7fe0 0x7fcc3c1f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc41070640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcc3c1536f0 0x7fcc3c1f5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:48876/0 (socket says 10.20.193.16:48876) 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc41070640 1 -- 10.20.193.16:0/1513373386 learned_addr learned my addr 10.20.193.16:0/1513373386 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc41070640 10 monclient(hunting): get_auth_request con 0x7fcc3c1536f0 auth_method 0 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc41070640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc41070640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc41070640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc4086f640 10 monclient(hunting): get_auth_request con 0x7fcc3c0700f0 auth_method 0 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc4086f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc4086f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc4086f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:36.667 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc33fff640 10 monclient(hunting): get_auth_request con 0x7fcc3c0c7fe0 auth_method 0 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc33fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc4086f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc4086f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc4086f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc41070640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc41070640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc41070640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:36.668 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.651+0000 7fcc33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.652+0000 7fcc4086f640 10 monclient(hunting): handle_auth_done global_id 4975 payload 824 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.652+0000 7fcc4086f640 1 -- 10.20.193.16:0/1513373386 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcc3c0c7fe0 msgr2=0x7fcc3c1f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.652+0000 7fcc4086f640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcc3c0c7fe0 0x7fcc3c1f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.652+0000 7fcc4086f640 1 -- 10.20.193.16:0/1513373386 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcc3c1536f0 msgr2=0x7fcc3c1f5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.652+0000 7fcc4086f640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcc3c1536f0 0x7fcc3c1f5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.652+0000 7fcc4086f640 10 monclient: _finish_hunting 0 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.652+0000 7fcc4086f640 1 monclient: found mon.noname-b 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.652+0000 7fcc4086f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.652+0000 7fcc4086f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.652+0000 7fcc4086f640 1 -- 10.20.193.16:0/1513373386 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c1f41c0 con 0x7fcc3c0700f0 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.653+0000 7fcc4086f640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcc3c0700f0 0x7fcc3c0c3ef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00e7d0 tx=0x7fcc2c00eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:36.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.653+0000 7fcc317fa640 10 client.?.objecter ms_handle_connect 0x7fcc3c0700f0 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.653+0000 7fcc317fa640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc317fa640 1 -- 10.20.193.16:0/1513373386 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c017070 con 0x7fcc3c0700f0 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc317fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc317fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc317fa640 10 monclient: dump: 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:36.670 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc317fa640 10 monclient: _finish_auth 0 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc317fa640 10 monclient: _check_auth_tickets 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 5 monclient: authenticate success, global_id 4975 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 10 monclient: _renew_subs 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 1 -- 10.20.193.16:0/1513373386 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c1f46f0 con 0x7fcc3c0700f0 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 10 client.4975.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:36.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 10 monclient: _renew_subs 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 1 -- 10.20.193.16:0/1513373386 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c1554c0 con 0x7fcc3c0700f0 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 1 librados: init done 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.654+0000 7fcc42afa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.655+0000 7fcc42afa640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.655+0000 7fcc42afa640 1 -- 10.20.193.16:0/1513373386 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc3c074c00 con 0x7fcc3c0700f0 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.655+0000 7fcc317fa640 1 -- 10.20.193.16:0/1513373386 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2c005a90 con 0x7fcc3c0700f0 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.655+0000 7fcc317fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.655+0000 7fcc317fa640 1 -- 10.20.193.16:0/1513373386 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c021060 con 0x7fcc3c0700f0 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.655+0000 7fcc317fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.655+0000 7fcc317fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:36.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.655+0000 7fcc32ffd640 4 set_mon_vals no callback set 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.655+0000 7fcc317fa640 10 monclient: dump: 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:36.673 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:36.762 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.761+0000 7fcc317fa640 1 -- 10.20.193.16:0/1513373386 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fcc2c00c940 con 0x7fcc3c0700f0 2026-02-17T06:00:36.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.761+0000 7fcc317fa640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcc0c079140 0x7fcc0c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:36.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.761+0000 7fcc33fff640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcc0c079140 0x7fcc0c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:36.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.761+0000 7fcc33fff640 10 monclient: get_auth_request con 0x7fcc0c079140 auth_method 0 2026-02-17T06:00:36.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.762+0000 7fcc317fa640 10 client.4975.objecter ms_handle_connect 0x7fcc0c079140 2026-02-17T06:00:36.763 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.762+0000 7fcc33fff640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcc0c079140 0x7fcc0c07b600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcc24039000 tx=0x7fcc24033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:36.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.763+0000 7fcc317fa640 1 -- 10.20.193.16:0/1513373386 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fcc2c05e520 con 0x7fcc3c0700f0 2026-02-17T06:00:36.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.763+0000 7fcc317fa640 10 client.4975.objecter ms_dispatch2 0x7fcc3c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:36.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.763+0000 7fcc317fa640 3 client.4975.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:36.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.763+0000 7fcc317fa640 3 client.4975.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:36.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.763+0000 7fcc317fa640 20 client.4975.objecter dump_active .. 0 homeless 2026-02-17T06:00:36.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.769+0000 7fcc317fa640 1 -- 10.20.193.16:0/1513373386 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcc2c0624e0 con 0x7fcc3c0700f0 2026-02-17T06:00:36.769 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.769+0000 7fcc317fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:36.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.769+0000 7fcc317fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:36.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.870+0000 7fcc42afa640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:36.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.870+0000 7fcc42afa640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:36.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.870+0000 7fcc42afa640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:00:36.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.870+0000 7fcc42afa640 1 -- 10.20.193.16:0/1513373386 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcc3c1573f0 con 0x7fcc3c0700f0 2026-02-17T06:00:36.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.872+0000 7fcc317fa640 1 -- 10.20.193.16:0/1513373386 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fcc2c0667e0 con 0x7fcc3c0700f0 2026-02-17T06:00:36.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.872+0000 7fcc317fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:36.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.872+0000 7fcc317fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:36.873 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:36.873 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:36.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 10 librados: watch_flush enter 2026-02-17T06:00:36.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 10 librados: watch_flush exit 2026-02-17T06:00:36.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 20 client.4975.objecter shutdown clearing up homeless session... 2026-02-17T06:00:36.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 10 client.4975.objecter successfully canceled tick 2026-02-17T06:00:36.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 -- 10.20.193.16:0/1513373386 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcc0c079140 msgr2=0x7fcc0c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:36.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcc0c079140 0x7fcc0c07b600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcc24039000 tx=0x7fcc24033000 comp rx=0 tx=0).stop 2026-02-17T06:00:36.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 10 monclient: shutdown 2026-02-17T06:00:36.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:36.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 -- 10.20.193.16:0/1513373386 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcc3c0700f0 msgr2=0x7fcc3c0c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:36.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcc3c0700f0 0x7fcc3c0c3ef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00e7d0 tx=0x7fcc2c00eca0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 -- 10.20.193.16:0/1513373386 shutdown_connections 2026-02-17T06:00:36.876 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcc0c079140 0x7fcc0c07b600 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcc3c1536f0 0x7fcc3c1f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcc3c0c7fe0 0x7fcc3c1f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 --2- 10.20.193.16:0/1513373386 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcc3c0700f0 0x7fcc3c0c3ef0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:36.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 -- 10.20.193.16:0/1513373386 >> 10.20.193.16:0/1513373386 conn(0x7fcc3c0cd770 msgr2=0x7fcc3c074410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:36.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 -- 10.20.193.16:0/1513373386 shutdown_connections 2026-02-17T06:00:36.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 -- 10.20.193.16:0/1513373386 wait complete. 2026-02-17T06:00:36.877 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:36.873+0000 7fcc42afa640 1 librados: shutdown 2026-02-17T06:00:39.883 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:39.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.950+0000 7f2bf1349640 10 monclient: get_monmap_and_config 2026-02-17T06:00:39.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.950+0000 7f2bf1349640 10 monclient: build_initial_monmap 2026-02-17T06:00:39.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.950+0000 7f2bf1349640 10 monclient: monmap: 2026-02-17T06:00:39.952 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:39.952 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:39.952 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:39.951718+0000 2026-02-17T06:00:39.952 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:39.951718+0000 2026-02-17T06:00:39.952 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:39.952 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:39.952 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:39.953 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:39.953 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:39.953 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:39.953 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:39.953 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:39.953 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:39.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 1 Processor -- start 2026-02-17T06:00:39.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 1 -- start start 2026-02-17T06:00:39.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 10 monclient: init 2026-02-17T06:00:39.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:39.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 10 monclient: _add_conns ranks=[0,2,4,3,5,1] 2026-02-17T06:00:39.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2bec14f020 addr v1:10.20.193.16:6789/0 2026-02-17T06:00:39.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2bec14f400 addr v1:10.20.193.148:6789/0 2026-02-17T06:00:39.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f2bec15caf0 0x7f2bec0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:39.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 10 monclient(hunting): picked mon.noname-b con 0x7f2bec15caf0 addr v2:10.20.193.128:3300/0 2026-02-17T06:00:39.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bec0c5b10 con 0x7f2bec14f020 2026-02-17T06:00:39.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bec06e620 con 0x7f2bec14f400 2026-02-17T06:00:39.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:39.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f2bec06e900 con 0x7f2bec15caf0 2026-02-17T06:00:39.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:39.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.951+0000 7f2bf1349640 10 monclient(hunting): authenticate will time out at 1292.520796s 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.952+0000 7f2beaffd640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f2bec14f020 0x7f2bec06dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:42964/0 (socket says 10.20.193.16:42964) 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.952+0000 7f2beaffd640 1 -- 10.20.193.16:0/3454135469 learned_addr learned my addr 10.20.193.16:0/3454135469 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.952+0000 7f2beb7fe640 1 --2- 10.20.193.16:0/3454135469 >> v2:10.20.193.128:3300/0 conn(0x7f2bec15caf0 0x7f2bec0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.952+0000 7f2beb7fe640 10 monclient(hunting): get_auth_request con 0x7f2bec15caf0 auth_method 0 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.952+0000 7f2beb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.952+0000 7f2beb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.952+0000 7f2beb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2beb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2beb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2beb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:39.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 1 -- 10.20.193.16:0/3454135469 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447205680 0 0) 0x7f2bec0c5b10 con 0x7f2bec14f020 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 10 monclient(hunting): _negotiate 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 10 monclient(hunting): my global_id is 4899 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 1 -- 10.20.193.16:0/3454135469 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bd8003620 con 0x7f2bec14f020 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 1 -- 10.20.193.16:0/3454135469 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453270098 0 0) 0x7f2bec06e620 con 0x7f2bec14f400 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 10 monclient(hunting): _negotiate 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 10 monclient(hunting): my global_id is 5036 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.953+0000 7f2be9ffb640 1 -- 10.20.193.16:0/3454135469 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bec0c5b10 con 0x7f2bec14f400 2026-02-17T06:00:39.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.954+0000 7f2beb7fe640 10 monclient(hunting): handle_auth_done global_id 4978 payload 274 2026-02-17T06:00:39.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.954+0000 7f2beb7fe640 1 -- 10.20.193.16:0/3454135469 >> v1:10.20.193.148:6789/0 conn(0x7f2bec14f400 legacy=0x7f2bec15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:39.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.954+0000 7f2beb7fe640 1 -- 10.20.193.16:0/3454135469 >> v1:10.20.193.16:6789/0 conn(0x7f2bec14f020 legacy=0x7f2bec06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:39.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.954+0000 7f2beb7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:00:39.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.954+0000 7f2beb7fe640 1 monclient: found mon.noname-b 2026-02-17T06:00:39.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.954+0000 7f2beb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:39.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.954+0000 7f2beb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:00:39.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.954+0000 7f2beb7fe640 1 -- 10.20.193.16:0/3454135469 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec06ebe0 con 0x7f2bec15caf0 2026-02-17T06:00:39.960 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.955+0000 7f2beb7fe640 1 --2- 10.20.193.16:0/3454135469 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2bec15caf0 0x7f2bec0c6010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009d90 tx=0x7f2bdc02f2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6c57cc483f75d5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:39.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.956+0000 7f2be9ffb640 1 -- 10.20.193.16:0/3454135469 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc03c070 con 0x7f2bec15caf0 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.956+0000 7f2be9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.956+0000 7f2be9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.956+0000 7f2be9ffb640 10 monclient: dump: 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:39.967 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.956+0000 7f2be9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.956+0000 7f2be9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.956+0000 7f2be9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.956+0000 7f2bf1349640 5 monclient: authenticate success, global_id 4978 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.956+0000 7f2bf1349640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2be9ffb640 1 -- 10.20.193.16:0/3454135469 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bdc037440 con 0x7f2bec15caf0 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2be9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2be9ffb640 1 -- 10.20.193.16:0/3454135469 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0377f0 con 0x7f2bec15caf0 2026-02-17T06:00:39.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 4 set_mon_vals no callback set 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2be9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2be9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2be9ffb640 10 monclient: dump: 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:39.969 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 10 monclient: shutdown 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 1 -- 10.20.193.16:0/3454135469 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2bec15caf0 msgr2=0x7f2bec0c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 1 --2- 10.20.193.16:0/3454135469 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2bec15caf0 0x7f2bec0c6010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009d90 tx=0x7f2bdc02f2c0 comp rx=0 tx=0).stop 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 1 -- 10.20.193.16:0/3454135469 shutdown_connections 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 1 --2- 10.20.193.16:0/3454135469 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2bec15caf0 0x7f2bec0c6010 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 1 -- 10.20.193.16:0/3454135469 >> 10.20.193.16:0/3454135469 conn(0x7f2bec0cd770 msgr2=0x7f2bec0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 1 -- 10.20.193.16:0/3454135469 shutdown_connections 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.957+0000 7f2bf1349640 1 -- 10.20.193.16:0/3454135469 wait complete. 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient: build_initial_monmap 2026-02-17T06:00:39.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient: monmap: 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 librados: starting msgr at 2026-02-17T06:00:39.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 librados: starting objecter 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 Processor -- start 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 -- start start 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 librados: setting wanted keys 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 librados: calling monclient init 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient: init 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2bec14f020 0x7f2bec14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient(hunting): picked mon.noname-c con 0x7f2bec14f020 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2bec14f400 0x7f2bec15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient(hunting): picked mon.noname-a con 0x7f2bec14f400 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2bec15caf0 0x7f2bec15b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:39.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient(hunting): picked mon.noname-b con 0x7f2bec15caf0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec06e2f0 con 0x7f2bec14f400 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec06f1a0 con 0x7f2bec15caf0 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec0c6550 con 0x7f2bec14f020 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.958+0000 7f2bf1349640 10 monclient(hunting): authenticate will time out at 1292.527845s 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2bea7fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2bec14f400 0x7f2bec15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2bea7fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2bec14f400 0x7f2bec15ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:48888/0 (socket says 10.20.193.16:48888) 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2bea7fc640 1 -- 10.20.193.16:0/1486533798 learned_addr learned my addr 10.20.193.16:0/1486533798 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:39.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2bea7fc640 10 monclient(hunting): get_auth_request con 0x7f2bec14f400 auth_method 0 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2bea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2bea7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2bea7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beb7fe640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2bec15caf0 0x7f2bec15b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beaffd640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2bec14f020 0x7f2bec14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beb7fe640 10 monclient(hunting): get_auth_request con 0x7f2bec15caf0 auth_method 0 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2bea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2bea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:39.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2bea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beaffd640 10 monclient(hunting): get_auth_request con 0x7f2bec14f020 auth_method 0 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.959+0000 7f2beaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.960+0000 7f2bea7fc640 10 monclient(hunting): handle_auth_done global_id 4902 payload 824 2026-02-17T06:00:39.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.960+0000 7f2bea7fc640 1 -- 10.20.193.16:0/1486533798 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2bec14f020 msgr2=0x7f2bec14bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.960+0000 7f2bea7fc640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2bec14f020 0x7f2bec14bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.960+0000 7f2bea7fc640 1 -- 10.20.193.16:0/1486533798 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2bec15caf0 msgr2=0x7f2bec15b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.960+0000 7f2bea7fc640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2bec15caf0 0x7f2bec15b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.960+0000 7f2bea7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.960+0000 7f2bea7fc640 1 monclient: found mon.noname-a 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.960+0000 7f2bea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.960+0000 7f2bea7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.960+0000 7f2bea7fc640 1 -- 10.20.193.16:0/1486533798 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec15b660 con 0x7f2bec14f400 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.961+0000 7f2bea7fc640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2bec14f400 0x7f2bec15ab20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400c730 tx=0x7f2bd400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.961+0000 7f2bcb7fe640 10 client.?.objecter ms_handle_connect 0x7f2bec14f400 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.961+0000 7f2bcb7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:39.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 1 -- 10.20.193.16:0/1486533798 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40052d0 con 0x7f2bec14f400 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 10 monclient: dump: 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:39.977 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 1 -- 10.20.193.16:0/1486533798 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd4005490 con 0x7f2bec14f400 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 1 -- 10.20.193.16:0/1486533798 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40038d0 con 0x7f2bec14f400 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bcb7fe640 10 monclient: dump: 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:39.978 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2be97fa640 4 set_mon_vals no callback set 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bf1349640 5 monclient: authenticate success, global_id 4902 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bf1349640 10 monclient: _renew_subs 2026-02-17T06:00:39.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bf1349640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:39.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bf1349640 1 -- 10.20.193.16:0/1486533798 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bec15b940 con 0x7f2bec14f400 2026-02-17T06:00:39.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bf1349640 10 client.4902.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:39.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bf1349640 10 monclient: _renew_subs 2026-02-17T06:00:39.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bf1349640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:39.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bf1349640 1 -- 10.20.193.16:0/1486533798 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bec0736d0 con 0x7f2bec14f400 2026-02-17T06:00:39.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.963+0000 7f2bf1349640 1 librados: init done 2026-02-17T06:00:39.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.964+0000 7f2bf1349640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:39.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.964+0000 7f2bf1349640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:39.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.964+0000 7f2bf1349640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:39.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.964+0000 7f2bf1349640 1 -- 10.20.193.16:0/1486533798 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bec0c6930 con 0x7f2bec14f400 2026-02-17T06:00:39.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.974+0000 7f2bcb7fe640 1 -- 10.20.193.16:0/1486533798 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f2bd4003a70 con 0x7f2bec14f400 2026-02-17T06:00:39.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.975+0000 7f2bcb7fe640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2bb8079140 0x7f2bb807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:39.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.975+0000 7f2beaffd640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2bb8079140 0x7f2bb807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:39.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.975+0000 7f2beaffd640 10 monclient: get_auth_request con 0x7f2bb8079140 auth_method 0 2026-02-17T06:00:39.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.975+0000 7f2beaffd640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2bb8079140 0x7f2bb807b600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2be0002410 tx=0x7f2be00075a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:39.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.975+0000 7f2bcb7fe640 10 client.4902.objecter ms_handle_connect 0x7f2bb8079140 2026-02-17T06:00:39.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.976+0000 7f2bcb7fe640 1 -- 10.20.193.16:0/1486533798 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f2bd405e3c0 con 0x7f2bec14f400 2026-02-17T06:00:39.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.976+0000 7f2bcb7fe640 10 client.4902.objecter ms_dispatch2 0x7f2bec000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:39.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.976+0000 7f2bcb7fe640 3 client.4902.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:39.982 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.976+0000 7f2bcb7fe640 3 client.4902.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:39.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:39.976+0000 7f2bcb7fe640 20 client.4902.objecter dump_active .. 0 homeless 2026-02-17T06:00:40.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.077+0000 7f2bcb7fe640 1 -- 10.20.193.16:0/1486533798 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2bd4062380 con 0x7f2bec14f400 2026-02-17T06:00:40.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.078+0000 7f2bcb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:40.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.078+0000 7f2bcb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:40.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.180+0000 7f2bf1349640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:40.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.180+0000 7f2bf1349640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:40.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.180+0000 7f2bf1349640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:40.181 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.180+0000 7f2bf1349640 1 -- 10.20.193.16:0/1486533798 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2bec06d290 con 0x7f2bec14f400 2026-02-17T06:00:40.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.183+0000 7f2bcb7fe640 1 -- 10.20.193.16:0/1486533798 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f2bd4066680 con 0x7f2bec14f400 2026-02-17T06:00:40.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.183+0000 7f2bcb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:40.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.183+0000 7f2bcb7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:40.184 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:40.184 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:40.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 10 librados: watch_flush enter 2026-02-17T06:00:40.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 10 librados: watch_flush exit 2026-02-17T06:00:40.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 20 client.4902.objecter shutdown clearing up homeless session... 2026-02-17T06:00:40.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 10 client.4902.objecter successfully canceled tick 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 -- 10.20.193.16:0/1486533798 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2bb8079140 msgr2=0x7f2bb807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2bb8079140 0x7f2bb807b600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2be0002410 tx=0x7f2be00075a0 comp rx=0 tx=0).stop 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 10 monclient: shutdown 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 -- 10.20.193.16:0/1486533798 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2bec14f400 msgr2=0x7f2bec15ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2bec14f400 0x7f2bec15ab20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400c730 tx=0x7f2bd400cc00 comp rx=0 tx=0).stop 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 -- 10.20.193.16:0/1486533798 shutdown_connections 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2bb8079140 0x7f2bb807b600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2bec15caf0 0x7f2bec15b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2bec14f400 0x7f2bec15ab20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 --2- 10.20.193.16:0/1486533798 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2bec14f020 0x7f2bec14bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:40.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 -- 10.20.193.16:0/1486533798 >> 10.20.193.16:0/1486533798 conn(0x7f2bec0cd770 msgr2=0x7f2bec0c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:40.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 -- 10.20.193.16:0/1486533798 shutdown_connections 2026-02-17T06:00:40.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.184+0000 7f2bf1349640 1 -- 10.20.193.16:0/1486533798 wait complete. 2026-02-17T06:00:40.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:40.185+0000 7f2bf1349640 1 librados: shutdown 2026-02-17T06:00:43.194 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:43.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.263+0000 7f8be3ce1640 10 monclient: get_monmap_and_config 2026-02-17T06:00:43.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.263+0000 7f8be3ce1640 10 monclient: build_initial_monmap 2026-02-17T06:00:43.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.263+0000 7f8be3ce1640 10 monclient: monmap: 2026-02-17T06:00:43.264 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:43.264 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:43.264 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:43.264186+0000 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:43.264186+0000 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.263+0000 7f8be3ce1640 1 Processor -- start 2026-02-17T06:00:43.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.263+0000 7f8be3ce1640 1 -- start start 2026-02-17T06:00:43.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.263+0000 7f8be3ce1640 10 monclient: init 2026-02-17T06:00:43.268 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:43.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 10 monclient: _add_conns ranks=[3,5,0,2,4,1] 2026-02-17T06:00:43.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f8bdc0700f0 0x7f8bdc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:43.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 10 monclient(hunting): picked mon.noname-a con 0x7f8bdc0700f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:00:43.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f8bdc0c7fe0 0x7f8bdc0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:43.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 10 monclient(hunting): picked mon.noname-c con 0x7f8bdc0c7fe0 addr v2:10.20.193.148:3300/0 2026-02-17T06:00:43.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8bdc1536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:00:43.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bdc0c8970 con 0x7f8bdc1536f0 2026-02-17T06:00:43.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:43.269 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8bdc0c4820 con 0x7f8bdc0700f0 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f8bdc0c4b00 con 0x7f8bdc0c7fe0 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be3ce1640 10 monclient(hunting): authenticate will time out at 1295.833225s 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be1a56640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f8bdc0700f0 0x7f8bdc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be1255640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f8bdc0c7fe0 0x7f8bdc0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be2257640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f8bdc1536f0 0x7f8bdc153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:39216/0 (socket says 10.20.193.16:39216) 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be2257640 1 -- 10.20.193.16:0/482490578 learned_addr learned my addr 10.20.193.16:0/482490578 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be1a56640 10 monclient(hunting): get_auth_request con 0x7f8bdc0700f0 auth_method 0 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be1a56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:43.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be1a56640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be1a56640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be1255640 10 monclient(hunting): get_auth_request con 0x7f8bdc0c7fe0 auth_method 0 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be1255640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be1255640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.264+0000 7f8be1255640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.265+0000 7f8be1255640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.265+0000 7f8be1255640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.265+0000 7f8be1255640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.265+0000 7f8be1a56640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.265+0000 7f8be1a56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.265+0000 7f8be1a56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:43.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.266+0000 7f8be1255640 10 monclient(hunting): handle_auth_done global_id 5042 payload 274 2026-02-17T06:00:43.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.266+0000 7f8be1255640 1 -- 10.20.193.16:0/482490578 >> v2:10.20.193.16:3300/0 conn(0x7f8bdc0700f0 msgr2=0x7f8bdc0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:43.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.266+0000 7f8be1255640 1 --2- 10.20.193.16:0/482490578 >> v2:10.20.193.16:3300/0 conn(0x7f8bdc0700f0 0x7f8bdc0704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:43.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.266+0000 7f8be1255640 1 -- 10.20.193.16:0/482490578 >> v1:10.20.193.16:6789/0 conn(0x7f8bdc1536f0 legacy=0x7f8bdc153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:43.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.266+0000 7f8be1255640 10 monclient: _finish_hunting 0 2026-02-17T06:00:43.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.266+0000 7f8be1255640 1 monclient: found mon.noname-c 2026-02-17T06:00:43.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.266+0000 7f8be1255640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:43.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.266+0000 7f8be1255640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:00:43.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.266+0000 7f8be1255640 1 -- 10.20.193.16:0/482490578 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc0c4de0 con 0x7f8bdc0c7fe0 2026-02-17T06:00:43.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.267+0000 7f8be1255640 1 --2- 10.20.193.16:0/482490578 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8bdc0c7fe0 0x7f8bdc0c8430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0c5b80 tx=0x7f8bd002ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6108adb6fd6887ad server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:43.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 1 -- 10.20.193.16:0/482490578 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd003c070 con 0x7f8bdc0c7fe0 2026-02-17T06:00:43.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:43.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:43.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 10 monclient: dump: 2026-02-17T06:00:43.279 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 10 monclient: _finish_auth 0 2026-02-17T06:00:43.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 10 monclient: _check_auth_tickets 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 1 -- 10.20.193.16:0/482490578 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd002fa60 con 0x7f8bdc0c7fe0 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 1 -- 10.20.193.16:0/482490578 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd002fe30 con 0x7f8bdc0c7fe0 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 5 monclient: authenticate success, global_id 5042 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 4 set_mon_vals no callback set 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be0a54640 10 monclient: dump: 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:43.281 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:43.282 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:43.282 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:43.282 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:43.282 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:43.282 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:43.282 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:43.282 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:43.282 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:43.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 10 monclient: shutdown 2026-02-17T06:00:43.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 1 -- 10.20.193.16:0/482490578 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8bdc0c7fe0 msgr2=0x7f8bdc0c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 1 --2- 10.20.193.16:0/482490578 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8bdc0c7fe0 0x7f8bdc0c8430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0c5b80 tx=0x7f8bd002ed50 comp rx=0 tx=0).stop 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 1 -- 10.20.193.16:0/482490578 shutdown_connections 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 1 --2- 10.20.193.16:0/482490578 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8bdc0c7fe0 0x7f8bdc0c8430 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 1 --2- 10.20.193.16:0/482490578 >> v2:10.20.193.16:3300/0 conn(0x7f8bdc0700f0 0x7f8bdc0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 1 -- 10.20.193.16:0/482490578 >> 10.20.193.16:0/482490578 conn(0x7f8bdc0cd770 msgr2=0x7f8bdc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 1 -- 10.20.193.16:0/482490578 shutdown_connections 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.269+0000 7f8be3ce1640 1 -- 10.20.193.16:0/482490578 wait complete. 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient: build_initial_monmap 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient: monmap: 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:43.283 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 librados: starting msgr at 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 librados: starting objecter 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 Processor -- start 2026-02-17T06:00:43.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 -- start start 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 librados: setting wanted keys 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 librados: calling monclient init 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient: init 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8bdc0700f0 0x7f8bdc144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient(hunting): picked mon.noname-b con 0x7f8bdc0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8bdc0c7fe0 0x7f8bdc06e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient(hunting): picked mon.noname-c con 0x7f8bdc0c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8bdc1536f0 0x7f8bdc06e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient(hunting): picked mon.noname-a con 0x7f8bdc1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:43.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc0c5680 con 0x7f8bdc1536f0 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc0c53a0 con 0x7f8bdc0700f0 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc074c00 con 0x7f8bdc0c7fe0 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.270+0000 7f8be3ce1640 10 monclient(hunting): authenticate will time out at 1295.839831s 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be2257640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8bdc1536f0 0x7f8bdc06e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be2257640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8bdc1536f0 0x7f8bdc06e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51766/0 (socket says 10.20.193.16:51766) 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be2257640 1 -- 10.20.193.16:0/240788493 learned_addr learned my addr 10.20.193.16:0/240788493 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be2257640 10 monclient(hunting): get_auth_request con 0x7f8bdc1536f0 auth_method 0 2026-02-17T06:00:43.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be2257640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be2257640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be2257640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1255640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8bdc0c7fe0 0x7f8bdc06e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1a56640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8bdc0700f0 0x7f8bdc144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be2257640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be2257640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be2257640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1255640 10 monclient(hunting): get_auth_request con 0x7f8bdc0c7fe0 auth_method 0 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1255640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1255640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1255640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:43.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1a56640 10 monclient(hunting): get_auth_request con 0x7f8bdc0700f0 auth_method 0 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1a56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1a56640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1a56640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1255640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1255640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1255640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1a56640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1a56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.271+0000 7f8be1a56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.272+0000 7f8be2257640 10 monclient(hunting): handle_auth_done global_id 4911 payload 824 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.272+0000 7f8be2257640 1 -- 10.20.193.16:0/240788493 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8bdc0c7fe0 msgr2=0x7f8bdc06e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:43.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.272+0000 7f8be2257640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8bdc0c7fe0 0x7f8bdc06e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.272+0000 7f8be2257640 1 -- 10.20.193.16:0/240788493 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8bdc0700f0 msgr2=0x7f8bdc144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.272+0000 7f8be2257640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8bdc0700f0 0x7f8bdc144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.272+0000 7f8be2257640 10 monclient: _finish_hunting 0 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.272+0000 7f8be2257640 1 monclient: found mon.noname-a 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.272+0000 7f8be2257640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.272+0000 7f8be2257640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.272+0000 7f8be2257640 1 -- 10.20.193.16:0/240788493 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc06ecb0 con 0x7f8bdc1536f0 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.273+0000 7f8be2257640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8bdc1536f0 0x7f8bdc06e5b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800cb50 tx=0x7f8bd800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.273+0000 7f8bca7fc640 10 client.?.objecter ms_handle_connect 0x7f8bdc1536f0 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.273+0000 7f8bca7fc640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 1 -- 10.20.193.16:0/240788493 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8017070 con 0x7f8bdc1536f0 2026-02-17T06:00:43.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 10 monclient: dump: 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 10 monclient: _finish_auth 0 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 1 -- 10.20.193.16:0/240788493 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd800ad50 con 0x7f8bdc1536f0 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8be3ce1640 5 monclient: authenticate success, global_id 4911 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8be3ce1640 10 monclient: _renew_subs 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8be3ce1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8be3ce1640 1 -- 10.20.193.16:0/240788493 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bdc06ef90 con 0x7f8bdc1536f0 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 1 -- 10.20.193.16:0/240788493 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8012600 con 0x7f8bdc1536f0 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bca7fc640 10 monclient: dump: 2026-02-17T06:00:43.290 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8bcbfff640 4 set_mon_vals no callback set 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8be3ce1640 10 client.4911.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:43.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8be3ce1640 10 monclient: _renew_subs 2026-02-17T06:00:43.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8be3ce1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:43.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8be3ce1640 1 -- 10.20.193.16:0/240788493 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bdc1f5410 con 0x7f8bdc1536f0 2026-02-17T06:00:43.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.275+0000 7f8be3ce1640 1 librados: init done 2026-02-17T06:00:43.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.276+0000 7f8be3ce1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:43.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.276+0000 7f8be3ce1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:43.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.276+0000 7f8be3ce1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:43.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.276+0000 7f8be3ce1640 1 -- 10.20.193.16:0/240788493 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bdc0704d0 con 0x7f8bdc1536f0 2026-02-17T06:00:43.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.386+0000 7f8bca7fc640 1 -- 10.20.193.16:0/240788493 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f8bd801b3b0 con 0x7f8bdc1536f0 2026-02-17T06:00:43.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.386+0000 7f8bca7fc640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8bac079190 0x7f8bac07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:43.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.386+0000 7f8be1a56640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8bac079190 0x7f8bac07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:43.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.386+0000 7f8be1a56640 10 monclient: get_auth_request con 0x7f8bac079190 auth_method 0 2026-02-17T06:00:43.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.387+0000 7f8be1a56640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8bac079190 0x7f8bac07b650 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009ba0 tx=0x7f8bcc009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:43.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.387+0000 7f8bca7fc640 10 client.4911.objecter ms_handle_connect 0x7f8bac079190 2026-02-17T06:00:43.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.394+0000 7f8bca7fc640 1 -- 10.20.193.16:0/240788493 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f8bd805ddd0 con 0x7f8bdc1536f0 2026-02-17T06:00:43.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.394+0000 7f8bca7fc640 10 client.4911.objecter ms_dispatch2 0x7f8bdc000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:43.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.394+0000 7f8bca7fc640 3 client.4911.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:43.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.394+0000 7f8bca7fc640 3 client.4911.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:43.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.394+0000 7f8bca7fc640 20 client.4911.objecter dump_active .. 0 homeless 2026-02-17T06:00:43.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.394+0000 7f8bca7fc640 1 -- 10.20.193.16:0/240788493 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8bd8022080 con 0x7f8bdc1536f0 2026-02-17T06:00:43.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.394+0000 7f8bca7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:43.396 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.394+0000 7f8bca7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:43.496 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.495+0000 7f8be3ce1640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:43.496 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.495+0000 7f8be3ce1640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:43.496 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.495+0000 7f8be3ce1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:43.497 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.495+0000 7f8be3ce1640 1 -- 10.20.193.16:0/240788493 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8bdc1573f0 con 0x7f8bdc1536f0 2026-02-17T06:00:43.498 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.497+0000 7f8bca7fc640 1 -- 10.20.193.16:0/240788493 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8bd8061d90 con 0x7f8bdc1536f0 2026-02-17T06:00:43.498 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.497+0000 7f8bca7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:43.498 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.497+0000 7f8bca7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:43.499 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:43.499 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:43.500 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 10 librados: watch_flush enter 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 10 librados: watch_flush exit 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 20 client.4911.objecter shutdown clearing up homeless session... 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 10 client.4911.objecter successfully canceled tick 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 -- 10.20.193.16:0/240788493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8bac079190 msgr2=0x7f8bac07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8bac079190 0x7f8bac07b650 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009ba0 tx=0x7f8bcc009290 comp rx=0 tx=0).stop 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 10 monclient: shutdown 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 -- 10.20.193.16:0/240788493 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8bdc1536f0 msgr2=0x7f8bdc06e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8bdc1536f0 0x7f8bdc06e5b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800cb50 tx=0x7f8bd800a430 comp rx=0 tx=0).stop 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 -- 10.20.193.16:0/240788493 shutdown_connections 2026-02-17T06:00:43.501 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8bac079190 0x7f8bac07b650 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:43.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f8bdc1536f0 0x7f8bdc06e5b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:43.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f8bdc0c7fe0 0x7f8bdc06e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:43.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 --2- 10.20.193.16:0/240788493 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f8bdc0700f0 0x7f8bdc144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:43.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 -- 10.20.193.16:0/240788493 >> 10.20.193.16:0/240788493 conn(0x7f8bdc0cd770 msgr2=0x7f8bdc0c6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:43.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 -- 10.20.193.16:0/240788493 shutdown_connections 2026-02-17T06:00:43.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 -- 10.20.193.16:0/240788493 wait complete. 2026-02-17T06:00:43.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:43.499+0000 7f8be3ce1640 1 librados: shutdown 2026-02-17T06:00:46.509 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:46.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.578+0000 7fb5f0432640 10 monclient: get_monmap_and_config 2026-02-17T06:00:46.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.578+0000 7fb5f0432640 10 monclient: build_initial_monmap 2026-02-17T06:00:46.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.578+0000 7fb5f0432640 10 monclient: monmap: 2026-02-17T06:00:46.578 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:46.578 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:46.579294+0000 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:46.579294+0000 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:46.579 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.578+0000 7fb5f0432640 1 Processor -- start 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.578+0000 7fb5f0432640 1 -- start start 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.578+0000 7fb5f0432640 10 monclient: init 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 10 monclient: _add_conns ranks=[3,0,2,5,4,1] 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb5e814f020 0x7fb5e806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 10 monclient(hunting): picked mon.noname-a con 0x7fb5e814f020 addr v2:10.20.193.16:3300/0 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb5e806e120 addr v1:10.20.193.16:6789/0 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb5e815caf0 addr v1:10.20.193.148:6789/0 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e80c5b10 con 0x7fb5e806e120 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e806e620 con 0x7fb5e815caf0 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:46.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e806e900 con 0x7fb5e814f020 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5f0432640 10 monclient(hunting): authenticate will time out at 1299.148341s 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5ee1a7640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb5e814f020 0x7fb5e806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5ee9a8640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fb5e815caf0 0x7fb5e80c6010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:57720/0 (socket says 10.20.193.16:57720) 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5ee9a8640 1 -- 10.20.193.16:0/2580270325 learned_addr learned my addr 10.20.193.16:0/2580270325 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5ee1a7640 10 monclient(hunting): get_auth_request con 0x7fb5e814f020 auth_method 0 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5ee1a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5ee1a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.579+0000 7fb5ee1a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.580+0000 7fb5ee1a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.580+0000 7fb5ee1a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:46.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.580+0000 7fb5ee1a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:46.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ed1a5640 1 -- 10.20.193.16:0/2580270325 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1465041913 0 0) 0x7fb5e806e620 con 0x7fb5e815caf0 2026-02-17T06:00:46.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ed1a5640 10 monclient(hunting): _negotiate 2026-02-17T06:00:46.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ed1a5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:46.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ed1a5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:46.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ed1a5640 10 monclient(hunting): my global_id is 5048 2026-02-17T06:00:46.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ed1a5640 1 -- 10.20.193.16:0/2580270325 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d0003620 con 0x7fb5e815caf0 2026-02-17T06:00:46.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ee1a7640 10 monclient(hunting): handle_auth_done global_id 4914 payload 274 2026-02-17T06:00:46.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ee1a7640 1 -- 10.20.193.16:0/2580270325 >> v1:10.20.193.148:6789/0 conn(0x7fb5e815caf0 legacy=0x7fb5e80c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:46.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ee1a7640 1 -- 10.20.193.16:0/2580270325 >> v1:10.20.193.16:6789/0 conn(0x7fb5e806e120 legacy=0x7fb5e815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:46.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ee1a7640 10 monclient: _finish_hunting 0 2026-02-17T06:00:46.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ee1a7640 1 monclient: found mon.noname-a 2026-02-17T06:00:46.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ee1a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:46.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ee1a7640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:46.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.581+0000 7fb5ee1a7640 1 -- 10.20.193.16:0/2580270325 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e806ebe0 con 0x7fb5e814f020 2026-02-17T06:00:46.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.582+0000 7fb5ee1a7640 1 --2- 10.20.193.16:0/2580270325 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e814f020 0x7fb5e806dbe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80098a0 tx=0x7fb5d802eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a815164d61ac82 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 1 -- 10.20.193.16:0/2580270325 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d803c070 con 0x7fb5e814f020 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 10 monclient: dump: 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 10 monclient: _finish_auth 0 2026-02-17T06:00:46.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 10 monclient: _check_auth_tickets 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 1 -- 10.20.193.16:0/2580270325 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d802fb70 con 0x7fb5e814f020 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 5 monclient: authenticate success, global_id 4914 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 1 -- 10.20.193.16:0/2580270325 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8037440 con 0x7fb5e814f020 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5ed1a5640 10 monclient: dump: 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:46.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 4 set_mon_vals no callback set 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 10 monclient: shutdown 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 1 -- 10.20.193.16:0/2580270325 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e814f020 msgr2=0x7fb5e806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 1 --2- 10.20.193.16:0/2580270325 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e814f020 0x7fb5e806dbe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80098a0 tx=0x7fb5d802eeb0 comp rx=0 tx=0).stop 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 1 -- 10.20.193.16:0/2580270325 shutdown_connections 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 1 --2- 10.20.193.16:0/2580270325 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e814f020 0x7fb5e806dbe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 1 -- 10.20.193.16:0/2580270325 >> 10.20.193.16:0/2580270325 conn(0x7fb5e80cd770 msgr2=0x7fb5e80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 1 -- 10.20.193.16:0/2580270325 shutdown_connections 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.584+0000 7fb5f0432640 1 -- 10.20.193.16:0/2580270325 wait complete. 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.585+0000 7fb5f0432640 10 monclient: build_initial_monmap 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.585+0000 7fb5f0432640 10 monclient: monmap: 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:46.588 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:46.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.585+0000 7fb5f0432640 1 librados: starting msgr at 2026-02-17T06:00:46.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.585+0000 7fb5f0432640 1 librados: starting objecter 2026-02-17T06:00:46.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.585+0000 7fb5f0432640 1 Processor -- start 2026-02-17T06:00:46.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.585+0000 7fb5f0432640 1 -- start start 2026-02-17T06:00:46.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.585+0000 7fb5f0432640 1 librados: setting wanted keys 2026-02-17T06:00:46.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.585+0000 7fb5f0432640 1 librados: calling monclient init 2026-02-17T06:00:46.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.585+0000 7fb5f0432640 10 monclient: init 2026-02-17T06:00:46.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5e806e120 0x7fb5e815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 10 monclient(hunting): picked mon.noname-c con 0x7fb5e806e120 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5e814f020 0x7fb5e814f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 10 monclient(hunting): picked mon.noname-b con 0x7fb5e814f020 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e815caf0 0x7fb5e80c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 10 monclient(hunting): picked mon.noname-a con 0x7fb5e815caf0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e80c6720 con 0x7fb5e815caf0 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e806f1a0 con 0x7fb5e814f020 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e80c68a0 con 0x7fb5e806e120 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5f0432640 10 monclient(hunting): authenticate will time out at 1299.155230s 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee9a8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e815caf0 0x7fb5e80c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee9a8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e815caf0 0x7fb5e80c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51784/0 (socket says 10.20.193.16:51784) 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee9a8640 1 -- 10.20.193.16:0/2356819020 learned_addr learned my addr 10.20.193.16:0/2356819020 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee9a8640 10 monclient(hunting): get_auth_request con 0x7fb5e815caf0 auth_method 0 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee9a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:46.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee9a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee9a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee1a7640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5e806e120 0x7fb5e815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ed9a6640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5e814f020 0x7fb5e814f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee9a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee9a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee9a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ed9a6640 10 monclient(hunting): get_auth_request con 0x7fb5e814f020 auth_method 0 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ed9a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ed9a6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ed9a6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee1a7640 10 monclient(hunting): get_auth_request con 0x7fb5e806e120 auth_method 0 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee1a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee1a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.586+0000 7fb5ee1a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.587+0000 7fb5ee1a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.587+0000 7fb5ee1a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.587+0000 7fb5ee1a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.587+0000 7fb5ed9a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.587+0000 7fb5ed9a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.587+0000 7fb5ed9a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.588+0000 7fb5ee9a8640 10 monclient(hunting): handle_auth_done global_id 4917 payload 824 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.588+0000 7fb5ee9a8640 1 -- 10.20.193.16:0/2356819020 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5e806e120 msgr2=0x7fb5e815a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:46.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.588+0000 7fb5ee9a8640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5e806e120 0x7fb5e815a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.588+0000 7fb5ee9a8640 1 -- 10.20.193.16:0/2356819020 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5e814f020 msgr2=0x7fb5e814f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.588+0000 7fb5ee9a8640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5e814f020 0x7fb5e814f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.588+0000 7fb5ee9a8640 10 monclient: _finish_hunting 0 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.588+0000 7fb5ee9a8640 1 monclient: found mon.noname-a 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.588+0000 7fb5ee9a8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.588+0000 7fb5ee9a8640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.588+0000 7fb5ee9a8640 1 -- 10.20.193.16:0/2356819020 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e81f42e0 con 0x7fb5e815caf0 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.589+0000 7fb5ee9a8640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e815caf0 0x7fb5e80c6dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400caa0 tx=0x7fb5e400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.589+0000 7fb5ceffd640 10 client.?.objecter ms_handle_connect 0x7fb5e815caf0 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.589+0000 7fb5ceffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 1 -- 10.20.193.16:0/2356819020 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400ac40 con 0x7fb5e815caf0 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 10 monclient: dump: 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:46.594 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 1 -- 10.20.193.16:0/2356819020 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e400ade0 con 0x7fb5e815caf0 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5f0432640 5 monclient: authenticate success, global_id 4917 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5f0432640 10 monclient: _renew_subs 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5f0432640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5f0432640 1 -- 10.20.193.16:0/2356819020 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e8151590 con 0x7fb5e815caf0 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 1 -- 10.20.193.16:0/2356819020 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4012600 con 0x7fb5e815caf0 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ceffd640 10 monclient: dump: 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:46.595 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5ec9a4640 4 set_mon_vals no callback set 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5f0432640 10 client.4917.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5f0432640 10 monclient: _renew_subs 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5f0432640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5f0432640 1 -- 10.20.193.16:0/2356819020 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e8151b50 con 0x7fb5e815caf0 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.590+0000 7fb5f0432640 1 librados: init done 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.591+0000 7fb5f0432640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.591+0000 7fb5f0432640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.591+0000 7fb5f0432640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:46.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.591+0000 7fb5f0432640 1 -- 10.20.193.16:0/2356819020 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e814f400 con 0x7fb5e815caf0 2026-02-17T06:00:46.603 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.602+0000 7fb5ceffd640 1 -- 10.20.193.16:0/2356819020 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fb5e40127a0 con 0x7fb5e815caf0 2026-02-17T06:00:46.603 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.603+0000 7fb5ceffd640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb5b8079140 0x7fb5b807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:46.604 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.603+0000 7fb5ee1a7640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb5b8079140 0x7fb5b807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:46.604 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.603+0000 7fb5ee1a7640 10 monclient: get_auth_request con 0x7fb5b8079140 auth_method 0 2026-02-17T06:00:46.604 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.603+0000 7fb5ee1a7640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb5b8079140 0x7fb5b807b600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8039000 tx=0x7fb5d8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:46.604 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.603+0000 7fb5ceffd640 10 client.4917.objecter ms_handle_connect 0x7fb5b8079140 2026-02-17T06:00:46.605 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.604+0000 7fb5ceffd640 1 -- 10.20.193.16:0/2356819020 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fb5e405ead0 con 0x7fb5e815caf0 2026-02-17T06:00:46.605 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.604+0000 7fb5ceffd640 10 client.4917.objecter ms_dispatch2 0x7fb5e8000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:46.605 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.604+0000 7fb5ceffd640 3 client.4917.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:46.605 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.604+0000 7fb5ceffd640 3 client.4917.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:46.605 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.604+0000 7fb5ceffd640 20 client.4917.objecter dump_active .. 0 homeless 2026-02-17T06:00:46.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.706+0000 7fb5ceffd640 1 -- 10.20.193.16:0/2356819020 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb5e4062a90 con 0x7fb5e815caf0 2026-02-17T06:00:46.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.706+0000 7fb5ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:46.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.706+0000 7fb5ceffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:46.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.806+0000 7fb5f0432640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:46.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.807+0000 7fb5f0432640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:46.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.807+0000 7fb5f0432640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:46.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.807+0000 7fb5f0432640 1 -- 10.20.193.16:0/2356819020 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb5e80ca5a0 con 0x7fb5e815caf0 2026-02-17T06:00:46.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.809+0000 7fb5ceffd640 1 -- 10.20.193.16:0/2356819020 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fb5e4066d90 con 0x7fb5e815caf0 2026-02-17T06:00:46.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.809+0000 7fb5ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:46.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.809+0000 7fb5ceffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:46.810 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:46.810 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:46.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 10 librados: watch_flush enter 2026-02-17T06:00:46.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 10 librados: watch_flush exit 2026-02-17T06:00:46.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 20 client.4917.objecter shutdown clearing up homeless session... 2026-02-17T06:00:46.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 10 client.4917.objecter successfully canceled tick 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 -- 10.20.193.16:0/2356819020 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb5b8079140 msgr2=0x7fb5b807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb5b8079140 0x7fb5b807b600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8039000 tx=0x7fb5d8033000 comp rx=0 tx=0).stop 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 10 monclient: shutdown 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 -- 10.20.193.16:0/2356819020 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e815caf0 msgr2=0x7fb5e80c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e815caf0 0x7fb5e80c6dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400caa0 tx=0x7fb5e400cf70 comp rx=0 tx=0).stop 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 -- 10.20.193.16:0/2356819020 shutdown_connections 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb5b8079140 0x7fb5b807b600 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb5e814f020 0x7fb5e814f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb5e815caf0 0x7fb5e80c6dd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 --2- 10.20.193.16:0/2356819020 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb5e806e120 0x7fb5e815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:46.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 -- 10.20.193.16:0/2356819020 >> 10.20.193.16:0/2356819020 conn(0x7fb5e80cd770 msgr2=0x7fb5e80c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:46.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 -- 10.20.193.16:0/2356819020 shutdown_connections 2026-02-17T06:00:46.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 -- 10.20.193.16:0/2356819020 wait complete. 2026-02-17T06:00:46.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:46.811+0000 7fb5f0432640 1 librados: shutdown 2026-02-17T06:00:49.138 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:06:27.965 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:49.165 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:06:27.993 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:49.167 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:06:27.993 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:49.169 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:06:27.995 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:49.173 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:06:28.001 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:49.178 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:06:28.004 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:49.192 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:06:27.981 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:49.194 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:06:27.983 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:49.202 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:06:27.991 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:00:49.821 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:49.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.892+0000 7fb7c68e8640 10 monclient: get_monmap_and_config 2026-02-17T06:00:49.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.892+0000 7fb7c68e8640 10 monclient: build_initial_monmap 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.892+0000 7fb7c68e8640 10 monclient: monmap: 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:49.893495+0000 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:49.893495+0000 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.892+0000 7fb7c68e8640 1 Processor -- start 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.892+0000 7fb7c68e8640 1 -- start start 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.892+0000 7fb7c68e8640 10 monclient: init 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 10 monclient: _add_conns ranks=[1,4,3,2,5,0] 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb7c00700f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fb7c00c86f0 0x7fb7c00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 10 monclient(hunting): picked mon.noname-b con 0x7fb7c00c86f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb7c01536f0 0x7fb7c0153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 10 monclient(hunting): picked mon.noname-a con 0x7fb7c01536f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:00:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7c00c4820 con 0x7fb7c00700f0 2026-02-17T06:00:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb7c00c4b00 con 0x7fb7c01536f0 2026-02-17T06:00:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fb7c00c4de0 con 0x7fb7c00c86f0 2026-02-17T06:00:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c68e8640 10 monclient(hunting): authenticate will time out at 1302.462449s 2026-02-17T06:00:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.893+0000 7fb7c4e5e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb7c01536f0 0x7fb7c0153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7c4e5e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb7c01536f0 0x7fb7c0153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51786/0 (socket says 10.20.193.16:51786) 2026-02-17T06:00:49.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7c4e5e640 1 -- 10.20.193.16:0/2601590345 learned_addr learned my addr 10.20.193.16:0/2601590345 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7c4e5e640 10 monclient(hunting): get_auth_request con 0x7fb7c01536f0 auth_method 0 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7c4e5e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7c4e5e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7c4e5e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7bf7fe640 1 --2- 10.20.193.16:0/2601590345 >> v2:10.20.193.128:3300/0 conn(0x7fb7c00c86f0 0x7fb7c00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7c4e5e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7c4e5e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7c4e5e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7bf7fe640 10 monclient(hunting): get_auth_request con 0x7fb7c00c86f0 auth_method 0 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7bf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.894+0000 7fb7bf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.895+0000 7fb7c4e5e640 10 monclient(hunting): handle_auth_done global_id 4920 payload 274 2026-02-17T06:00:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.895+0000 7fb7c4e5e640 1 -- 10.20.193.16:0/2601590345 >> v2:10.20.193.128:3300/0 conn(0x7fb7c00c86f0 msgr2=0x7fb7c00704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.895+0000 7fb7c4e5e640 1 --2- 10.20.193.16:0/2601590345 >> v2:10.20.193.128:3300/0 conn(0x7fb7c00c86f0 0x7fb7c00704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.895+0000 7fb7c4e5e640 1 -- 10.20.193.16:0/2601590345 >> v1:10.20.193.128:6789/0 conn(0x7fb7c00700f0 legacy=0x7fb7c00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.895+0000 7fb7c4e5e640 10 monclient: _finish_hunting 0 2026-02-17T06:00:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.895+0000 7fb7c4e5e640 1 monclient: found mon.noname-a 2026-02-17T06:00:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.895+0000 7fb7c4e5e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:49.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.895+0000 7fb7c4e5e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:49.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.895+0000 7fb7c4e5e640 1 -- 10.20.193.16:0/2601590345 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c00c50c0 con 0x7fb7c01536f0 2026-02-17T06:00:49.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.896+0000 7fb7c4e5e640 1 --2- 10.20.193.16:0/2601590345 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c01536f0 0x7fb7c0153b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00c5b80 tx=0x7fb7b402f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa1afc2e1b10d3f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 1 -- 10.20.193.16:0/2601590345 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b402fcf0 con 0x7fb7c01536f0 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 10 monclient: dump: 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 1 -- 10.20.193.16:0/2601590345 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b40373f0 con 0x7fb7c01536f0 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 1 -- 10.20.193.16:0/2601590345 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40377c0 con 0x7fb7c01536f0 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7beffd640 10 monclient: dump: 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 5 monclient: authenticate success, global_id 4920 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 4 set_mon_vals no callback set 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 10 monclient: shutdown 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 1 -- 10.20.193.16:0/2601590345 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c01536f0 msgr2=0x7fb7c0153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 1 --2- 10.20.193.16:0/2601590345 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c01536f0 0x7fb7c0153b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00c5b80 tx=0x7fb7b402f230 comp rx=0 tx=0).stop 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 1 -- 10.20.193.16:0/2601590345 shutdown_connections 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 1 --2- 10.20.193.16:0/2601590345 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c01536f0 0x7fb7c0153b20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 1 --2- 10.20.193.16:0/2601590345 >> v2:10.20.193.128:3300/0 conn(0x7fb7c00c86f0 0x7fb7c00704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 1 -- 10.20.193.16:0/2601590345 >> 10.20.193.16:0/2601590345 conn(0x7fb7c00cd770 msgr2=0x7fb7c00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 1 -- 10.20.193.16:0/2601590345 shutdown_connections 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.898+0000 7fb7c68e8640 1 -- 10.20.193.16:0/2601590345 wait complete. 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient: build_initial_monmap 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient: monmap: 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 librados: starting msgr at 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 librados: starting objecter 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 Processor -- start 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 -- start start 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 librados: setting wanted keys 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 librados: calling monclient init 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient: init 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb7c00700f0 0x7fb7c0145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient(hunting): picked mon.noname-b con 0x7fb7c00700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c00c86f0 0x7fb7c01f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient(hunting): picked mon.noname-a con 0x7fb7c00c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb7c01536f0 0x7fb7c0140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient(hunting): picked mon.noname-c con 0x7fb7c01536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0154060 con 0x7fb7c00c86f0 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c00c5680 con 0x7fb7c00700f0 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0074c00 con 0x7fb7c01536f0 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.899+0000 7fb7c68e8640 10 monclient(hunting): authenticate will time out at 1302.468723s 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bf7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c00c86f0 0x7fb7c01f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bffff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb7c00700f0 0x7fb7c0145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bf7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c00c86f0 0x7fb7c01f73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51796/0 (socket says 10.20.193.16:51796) 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bf7fe640 1 -- 10.20.193.16:0/1259277300 learned_addr learned my addr 10.20.193.16:0/1259277300 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bf7fe640 10 monclient(hunting): get_auth_request con 0x7fb7c00c86f0 auth_method 0 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7c4e5e640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb7c01536f0 0x7fb7c0140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7c4e5e640 10 monclient(hunting): get_auth_request con 0x7fb7c01536f0 auth_method 0 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7c4e5e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7c4e5e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7c4e5e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7c4e5e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7c4e5e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.900+0000 7fb7c4e5e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bffff640 10 monclient(hunting): get_auth_request con 0x7fb7c00700f0 auth_method 0 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bf7fe640 10 monclient(hunting): handle_auth_done global_id 4923 payload 824 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bf7fe640 1 -- 10.20.193.16:0/1259277300 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb7c01536f0 msgr2=0x7fb7c0140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bf7fe640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb7c01536f0 0x7fb7c0140d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bf7fe640 1 -- 10.20.193.16:0/1259277300 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb7c00700f0 msgr2=0x7fb7c0145890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bf7fe640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb7c00700f0 0x7fb7c0145890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bf7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bf7fe640 1 monclient: found mon.noname-a 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.901+0000 7fb7bf7fe640 1 -- 10.20.193.16:0/1259277300 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c0141810 con 0x7fb7c00c86f0 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.902+0000 7fb7bf7fe640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c00c86f0 0x7fb7c01f73b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e7f0 tx=0x7fb7b000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.902+0000 7fb7bcff9640 10 client.?.objecter ms_handle_connect 0x7fb7c00c86f0 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.902+0000 7fb7bcff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 1 -- 10.20.193.16:0/1259277300 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0017070 con 0x7fb7c00c86f0 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 10 monclient: dump: 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:49.920 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 10 monclient: _finish_auth 0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 1 -- 10.20.193.16:0/1259277300 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b000c8d0 con 0x7fb7c00c86f0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7c68e8640 5 monclient: authenticate success, global_id 4923 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 1 -- 10.20.193.16:0/1259277300 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000f040 con 0x7fb7c00c86f0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7bcff9640 10 monclient: dump: 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7be7fc640 4 set_mon_vals no callback set 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7c68e8640 10 monclient: _renew_subs 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7c68e8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7c68e8640 1 -- 10.20.193.16:0/1259277300 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c0141af0 con 0x7fb7c00c86f0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7c68e8640 10 client.4923.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7c68e8640 10 monclient: _renew_subs 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7c68e8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7c68e8640 1 -- 10.20.193.16:0/1259277300 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c01420b0 con 0x7fb7c00c86f0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.904+0000 7fb7c68e8640 1 librados: init done 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.905+0000 7fb7c68e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.905+0000 7fb7c68e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.905+0000 7fb7c68e8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:49.905+0000 7fb7c68e8640 1 -- 10.20.193.16:0/1259277300 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c00698c0 con 0x7fb7c00c86f0 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.013+0000 7fb7bcff9640 1 -- 10.20.193.16:0/1259277300 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fb7b0012050 con 0x7fb7c00c86f0 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.013+0000 7fb7bcff9640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb790079170 0x7fb79007b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.014+0000 7fb7bffff640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb790079170 0x7fb79007b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.014+0000 7fb7bffff640 10 monclient: get_auth_request con 0x7fb790079170 auth_method 0 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.014+0000 7fb7bffff640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb790079170 0x7fb79007b630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac0051f0 tx=0x7fb7ac007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.014+0000 7fb7bcff9640 10 client.4923.objecter ms_handle_connect 0x7fb790079170 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.022+0000 7fb7bcff9640 1 -- 10.20.193.16:0/1259277300 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fb7b005ddc0 con 0x7fb7c00c86f0 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.022+0000 7fb7bcff9640 10 client.4923.objecter ms_dispatch2 0x7fb7c0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.022+0000 7fb7bcff9640 3 client.4923.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.022+0000 7fb7bcff9640 3 client.4923.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.022+0000 7fb7bcff9640 20 client.4923.objecter dump_active .. 0 homeless 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.022+0000 7fb7bcff9640 1 -- 10.20.193.16:0/1259277300 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb7c00698c0 con 0x7fb7c00c86f0 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.022+0000 7fb7bcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.022+0000 7fb7bcff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:50.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.124+0000 7fb7c68e8640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.124+0000 7fb7c68e8640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.124+0000 7fb7c68e8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:50.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.124+0000 7fb7c68e8640 1 -- 10.20.193.16:0/1259277300 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb7c00704d0 con 0x7fb7c00c86f0 2026-02-17T06:00:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.126+0000 7fb7bcff9640 1 -- 10.20.193.16:0/1259277300 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fb7b0061d80 con 0x7fb7c00c86f0 2026-02-17T06:00:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.126+0000 7fb7bcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:50.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.126+0000 7fb7bcff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:50.127 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:50.127 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 10 librados: watch_flush enter 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 10 librados: watch_flush exit 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 20 client.4923.objecter shutdown clearing up homeless session... 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 10 client.4923.objecter successfully canceled tick 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 -- 10.20.193.16:0/1259277300 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb790079170 msgr2=0x7fb79007b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb790079170 0x7fb79007b630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac0051f0 tx=0x7fb7ac007f30 comp rx=0 tx=0).stop 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 10 monclient: shutdown 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 -- 10.20.193.16:0/1259277300 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c00c86f0 msgr2=0x7fb7c01f73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c00c86f0 0x7fb7c01f73b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e7f0 tx=0x7fb7b000ecc0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 -- 10.20.193.16:0/1259277300 shutdown_connections 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb790079170 0x7fb79007b630 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb7c01536f0 0x7fb7c0140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb7c00c86f0 0x7fb7c01f73b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 --2- 10.20.193.16:0/1259277300 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb7c00700f0 0x7fb7c0145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 -- 10.20.193.16:0/1259277300 >> 10.20.193.16:0/1259277300 conn(0x7fb7c00cd770 msgr2=0x7fb7c0151180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:50.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 -- 10.20.193.16:0/1259277300 shutdown_connections 2026-02-17T06:00:50.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 -- 10.20.193.16:0/1259277300 wait complete. 2026-02-17T06:00:50.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:50.128+0000 7fb7c68e8640 1 librados: shutdown 2026-02-17T06:00:53.138 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:53.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.206+0000 7f3802787640 10 monclient: get_monmap_and_config 2026-02-17T06:00:53.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.206+0000 7f3802787640 10 monclient: build_initial_monmap 2026-02-17T06:00:53.206 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.206+0000 7f3802787640 10 monclient: monmap: 2026-02-17T06:00:53.206 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:53.206 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:53.206 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:53.207221+0000 2026-02-17T06:00:53.206 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:53.207221+0000 2026-02-17T06:00:53.206 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.206+0000 7f3802787640 1 Processor -- start 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.206+0000 7f3802787640 1 -- start start 2026-02-17T06:00:53.207 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.206+0000 7f3802787640 10 monclient: init 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 10 monclient: _add_conns ranks=[1,3,2,5,0,4] 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f37fc0700f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f37fc0c86f0 0x7f37fc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 10 monclient(hunting): picked mon.noname-a con 0x7f37fc0c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f37fc1536f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37fc0c4820 con 0x7f37fc0700f0 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37fc0c4b00 con 0x7f37fc1536f0 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f37fc0c4de0 con 0x7f37fc0c86f0 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f3802787640 10 monclient(hunting): authenticate will time out at 1305.776176s 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f37fb7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f37fc0c86f0 0x7f37fc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f37fbfff640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f37fc0700f0 0x7f37fc0c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:59812/0 (socket says 10.20.193.16:59812) 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f37fbfff640 1 -- 10.20.193.16:0/716863465 learned_addr learned my addr 10.20.193.16:0/716863465 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f37fb7fe640 10 monclient(hunting): get_auth_request con 0x7f37fc0c86f0 auth_method 0 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f37fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f37fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f37fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f37fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f37fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:53.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.207+0000 7f37fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:53.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.208+0000 7f37fb7fe640 10 monclient(hunting): handle_auth_done global_id 4926 payload 274 2026-02-17T06:00:53.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.208+0000 7f37fb7fe640 1 -- 10.20.193.16:0/716863465 >> v1:10.20.193.148:6789/0 conn(0x7f37fc1536f0 legacy=0x7f37fc153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.208+0000 7f37fb7fe640 1 -- 10.20.193.16:0/716863465 >> v1:10.20.193.128:6789/0 conn(0x7f37fc0700f0 legacy=0x7f37fc0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.208+0000 7f37fb7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:00:53.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.208+0000 7f37fb7fe640 1 monclient: found mon.noname-a 2026-02-17T06:00:53.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.208+0000 7f37fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:53.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.208+0000 7f37fb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:53.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.208+0000 7f37fb7fe640 1 -- 10.20.193.16:0/716863465 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37fc0c50c0 con 0x7f37fc0c86f0 2026-02-17T06:00:53.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.209+0000 7f37fb7fe640 1 --2- 10.20.193.16:0/716863465 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0c86f0 0x7f37fc0704d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0098a0 tx=0x7f37ec02edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c7b2f0be4d7f2e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 1 -- 10.20.193.16:0/716863465 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec03c070 con 0x7f37fc0c86f0 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 10 monclient: dump: 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 1 -- 10.20.193.16:0/716863465 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ec02fae0 con 0x7f37fc0c86f0 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 5 monclient: authenticate success, global_id 4926 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 1 -- 10.20.193.16:0/716863465 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec037440 con 0x7f37fc0c86f0 2026-02-17T06:00:53.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 4 set_mon_vals no callback set 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f37faffd640 10 monclient: dump: 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 10 monclient: shutdown 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 1 -- 10.20.193.16:0/716863465 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0c86f0 msgr2=0x7f37fc0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 1 --2- 10.20.193.16:0/716863465 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0c86f0 0x7f37fc0704d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0098a0 tx=0x7f37ec02edd0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 1 -- 10.20.193.16:0/716863465 shutdown_connections 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 1 --2- 10.20.193.16:0/716863465 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0c86f0 0x7f37fc0704d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 1 -- 10.20.193.16:0/716863465 >> 10.20.193.16:0/716863465 conn(0x7f37fc0cd770 msgr2=0x7f37fc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 1 -- 10.20.193.16:0/716863465 shutdown_connections 2026-02-17T06:00:53.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.211+0000 7f3802787640 1 -- 10.20.193.16:0/716863465 wait complete. 2026-02-17T06:00:53.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.212+0000 7f3802787640 10 monclient: build_initial_monmap 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.212+0000 7f3802787640 10 monclient: monmap: 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.212+0000 7f3802787640 1 librados: starting msgr at 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.212+0000 7f3802787640 1 librados: starting objecter 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.212+0000 7f3802787640 1 Processor -- start 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.212+0000 7f3802787640 1 -- start start 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.212+0000 7f3802787640 1 librados: setting wanted keys 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.212+0000 7f3802787640 1 librados: calling monclient init 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.212+0000 7f3802787640 10 monclient: init 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0700f0 0x7f37fc152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 10 monclient(hunting): picked mon.noname-a con 0x7f37fc0700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f37fc0c86f0 0x7f37fc142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 10 monclient(hunting): picked mon.noname-c con 0x7f37fc0c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f37fc1536f0 0x7f37fc1f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 10 monclient(hunting): picked mon.noname-b con 0x7f37fc1536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc074c00 con 0x7f37fc0700f0 2026-02-17T06:00:53.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc0c5680 con 0x7f37fc1536f0 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc074d80 con 0x7f37fc0c86f0 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3802787640 10 monclient(hunting): authenticate will time out at 1305.782153s 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fbfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0700f0 0x7f37fc152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fbfff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0700f0 0x7f37fc152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:34840/0 (socket says 10.20.193.16:34840) 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fbfff640 1 -- 10.20.193.16:0/1997078797 learned_addr learned my addr 10.20.193.16:0/1997078797 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fbfff640 10 monclient(hunting): get_auth_request con 0x7f37fc0700f0 auth_method 0 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fb7fe640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f37fc0c86f0 0x7f37fc142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3800cfd640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f37fc1536f0 0x7f37fc1f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fb7fe640 10 monclient(hunting): get_auth_request con 0x7f37fc0c86f0 auth_method 0 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f37fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3800cfd640 10 monclient(hunting): get_auth_request con 0x7f37fc1536f0 auth_method 0 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3800cfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3800cfd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.213+0000 7f3800cfd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.214+0000 7f3800cfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.214+0000 7f3800cfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.214+0000 7f3800cfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.214+0000 7f37fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.214+0000 7f37fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:53.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.214+0000 7f37fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.215+0000 7f37fbfff640 10 monclient(hunting): handle_auth_done global_id 4929 payload 824 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.215+0000 7f37fbfff640 1 -- 10.20.193.16:0/1997078797 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f37fc0c86f0 msgr2=0x7f37fc142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.215+0000 7f37fbfff640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f37fc0c86f0 0x7f37fc142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.215+0000 7f37fbfff640 1 -- 10.20.193.16:0/1997078797 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f37fc1536f0 msgr2=0x7f37fc1f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.215+0000 7f37fbfff640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f37fc1536f0 0x7f37fc1f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.215+0000 7f37fbfff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.215+0000 7f37fbfff640 1 monclient: found mon.noname-a 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.215+0000 7f37fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.215+0000 7f37fbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.215+0000 7f37fbfff640 1 -- 10.20.193.16:0/1997078797 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37fc1f5130 con 0x7f37fc0700f0 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.216+0000 7f37fbfff640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0700f0 0x7f37fc152c30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c500 tx=0x7f37e800c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.216+0000 7f37f8ff9640 10 client.?.objecter ms_handle_connect 0x7f37fc0700f0 2026-02-17T06:00:53.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.216+0000 7f37f8ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 1 -- 10.20.193.16:0/1997078797 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8017070 con 0x7f37fc0700f0 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 10 monclient: dump: 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 10 monclient: _finish_auth 0 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 1 -- 10.20.193.16:0/1997078797 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e8005e00 con 0x7f37fc0700f0 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 1 -- 10.20.193.16:0/1997078797 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80072b0 con 0x7f37fc0700f0 2026-02-17T06:00:53.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37f8ff9640 10 monclient: dump: 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f37fa7fc640 4 set_mon_vals no callback set 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f3802787640 5 monclient: authenticate success, global_id 4929 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f3802787640 10 monclient: _renew_subs 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f3802787640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f3802787640 1 -- 10.20.193.16:0/1997078797 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37fc1f5410 con 0x7f37fc0700f0 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f3802787640 10 client.4929.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f3802787640 10 monclient: _renew_subs 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f3802787640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f3802787640 1 -- 10.20.193.16:0/1997078797 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37fc1f59d0 con 0x7f37fc0700f0 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.219+0000 7f3802787640 1 librados: init done 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.220+0000 7f3802787640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:53.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.220+0000 7f3802787640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:53.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.220+0000 7f3802787640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:53.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.220+0000 7f3802787640 1 -- 10.20.193.16:0/1997078797 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37fc0704d0 con 0x7f37fc0700f0 2026-02-17T06:00:53.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.232+0000 7f37f8ff9640 1 -- 10.20.193.16:0/1997078797 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f37e8007510 con 0x7f37fc0700f0 2026-02-17T06:00:53.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.232+0000 7f37f8ff9640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37cc079140 0x7f37cc07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.232+0000 7f37fb7fe640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37cc079140 0x7f37cc07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:53.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.232+0000 7f37fb7fe640 10 monclient: get_auth_request con 0x7f37cc079140 auth_method 0 2026-02-17T06:00:53.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.232+0000 7f37fb7fe640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37cc079140 0x7f37cc07b600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f37ec039000 tx=0x7f37ec033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:53.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.232+0000 7f37f8ff9640 10 client.4929.objecter ms_handle_connect 0x7f37cc079140 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.335+0000 7f37f8ff9640 1 -- 10.20.193.16:0/1997078797 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f37e805d6d0 con 0x7f37fc0700f0 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.335+0000 7f37f8ff9640 10 client.4929.objecter ms_dispatch2 0x7f37fc000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.335+0000 7f37f8ff9640 3 client.4929.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.335+0000 7f37f8ff9640 3 client.4929.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.335+0000 7f37f8ff9640 20 client.4929.objecter dump_active .. 0 homeless 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.335+0000 7f37f8ff9640 1 -- 10.20.193.16:0/1997078797 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f37e8021020 con 0x7f37fc0700f0 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.335+0000 7f37f8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.335+0000 7f37f8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.435+0000 7f3802787640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.435+0000 7f3802787640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.435+0000 7f3802787640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.435+0000 7f3802787640 1 -- 10.20.193.16:0/1997078797 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f37fc1573f0 con 0x7f37fc0700f0 2026-02-17T06:00:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.437+0000 7f37f8ff9640 1 -- 10.20.193.16:0/1997078797 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f37e8061690 con 0x7f37fc0700f0 2026-02-17T06:00:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.437+0000 7f37f8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.437+0000 7f37f8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:53.438 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:53.438 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 10 librados: watch_flush enter 2026-02-17T06:00:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 10 librados: watch_flush exit 2026-02-17T06:00:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 20 client.4929.objecter shutdown clearing up homeless session... 2026-02-17T06:00:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 10 client.4929.objecter successfully canceled tick 2026-02-17T06:00:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 -- 10.20.193.16:0/1997078797 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37cc079140 msgr2=0x7f37cc07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37cc079140 0x7f37cc07b600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f37ec039000 tx=0x7f37ec033000 comp rx=0 tx=0).stop 2026-02-17T06:00:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 10 monclient: shutdown 2026-02-17T06:00:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 -- 10.20.193.16:0/1997078797 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0700f0 msgr2=0x7f37fc152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0700f0 0x7f37fc152c30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c500 tx=0x7f37e800c9d0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 -- 10.20.193.16:0/1997078797 shutdown_connections 2026-02-17T06:00:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f37cc079140 0x7f37cc07b600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f37fc1536f0 0x7f37fc1f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f37fc0c86f0 0x7f37fc142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 --2- 10.20.193.16:0/1997078797 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f37fc0700f0 0x7f37fc152c30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 -- 10.20.193.16:0/1997078797 >> 10.20.193.16:0/1997078797 conn(0x7f37fc0cd770 msgr2=0x7f37fc0c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 -- 10.20.193.16:0/1997078797 shutdown_connections 2026-02-17T06:00:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 -- 10.20.193.16:0/1997078797 wait complete. 2026-02-17T06:00:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:53.438+0000 7f3802787640 1 librados: shutdown 2026-02-17T06:00:56.448 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:56.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.517+0000 7f7b38e96640 10 monclient: get_monmap_and_config 2026-02-17T06:00:56.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.517+0000 7f7b38e96640 10 monclient: build_initial_monmap 2026-02-17T06:00:56.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.517+0000 7f7b38e96640 10 monclient: monmap: 2026-02-17T06:00:56.518 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:56.518 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:56.518 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:56.518618+0000 2026-02-17T06:00:56.518 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:56.518618+0000 2026-02-17T06:00:56.519 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:56.519 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:56.519 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:56.519 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:56.519 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:56.519 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:56.519 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:56.519 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:56.519 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:56.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 1 Processor -- start 2026-02-17T06:00:56.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 1 -- start start 2026-02-17T06:00:56.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 10 monclient: init 2026-02-17T06:00:56.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:56.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 10 monclient: _add_conns ranks=[3,0,1,4,5,2] 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f7b34155e00 0x7f7b340ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 10 monclient(hunting): picked mon.noname-a con 0x7f7b34155e00 addr v2:10.20.193.16:3300/0 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7b341561e0 addr v1:10.20.193.16:6789/0 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7b34074dc0 addr v1:10.20.193.128:6789/0 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b340c96d0 con 0x7f7b341561e0 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b34074860 con 0x7f7b34074dc0 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7b340c9940 con 0x7f7b34155e00 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.518+0000 7f7b38e96640 10 monclient(hunting): authenticate will time out at 1309.087680s 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.519+0000 7f7b32d76640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f7b34074dc0 0x7f7b3415f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:59822/0 (socket says 10.20.193.16:59822) 2026-02-17T06:00:56.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.519+0000 7f7b32d76640 1 -- 10.20.193.16:0/2041417438 learned_addr learned my addr 10.20.193.16:0/2041417438 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.519+0000 7f7b32575640 1 --2- 10.20.193.16:0/2041417438 >> v2:10.20.193.16:3300/0 conn(0x7f7b34155e00 0x7f7b340ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.519+0000 7f7b32575640 10 monclient(hunting): get_auth_request con 0x7f7b34155e00 auth_method 0 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.519+0000 7f7b32575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.519+0000 7f7b32575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.519+0000 7f7b32575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.519+0000 7f7b32575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.519+0000 7f7b32575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.519+0000 7f7b32575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b31573640 1 -- 10.20.193.16:0/2041417438 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61102166 0 0) 0x7f7b24002ca0 con 0x7f7b34074dc0 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b31573640 10 monclient(hunting): _negotiate 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b31573640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:56.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b31573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b31573640 10 monclient(hunting): my global_id is 5005 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b31573640 1 -- 10.20.193.16:0/2041417438 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b20003620 con 0x7f7b34074dc0 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b32575640 10 monclient(hunting): handle_auth_done global_id 4932 payload 274 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b32575640 1 -- 10.20.193.16:0/2041417438 >> v1:10.20.193.128:6789/0 conn(0x7f7b34074dc0 legacy=0x7f7b3415f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b32575640 1 -- 10.20.193.16:0/2041417438 >> v1:10.20.193.16:6789/0 conn(0x7f7b341561e0 legacy=0x7f7b34074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b32575640 10 monclient: _finish_hunting 0 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b32575640 1 monclient: found mon.noname-a 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b32575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b32575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.520+0000 7f7b32575640 1 -- 10.20.193.16:0/2041417438 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b340c9c20 con 0x7f7b34155e00 2026-02-17T06:00:56.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.521+0000 7f7b32575640 1 --2- 10.20.193.16:0/2041417438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b34155e00 0x7f7b340ca700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7b280098a0 tx=0x7f7b2802ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d6ce3c69a3039 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:56.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.523+0000 7f7b31573640 1 -- 10.20.193.16:0/2041417438 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2803c070 con 0x7f7b34155e00 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 10 monclient: dump: 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:56.532 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 10 monclient: _finish_auth 0 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 10 monclient: _check_auth_tickets 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 1 -- 10.20.193.16:0/2041417438 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b2802fad0 con 0x7f7b34155e00 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 1 -- 10.20.193.16:0/2041417438 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28037440 con 0x7f7b34155e00 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 5 monclient: authenticate success, global_id 4932 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:56.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b31573640 10 monclient: dump: 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:56.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 4 set_mon_vals no callback set 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 10 monclient: shutdown 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 1 -- 10.20.193.16:0/2041417438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b34155e00 msgr2=0x7f7b340ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 1 --2- 10.20.193.16:0/2041417438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b34155e00 0x7f7b340ca700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7b280098a0 tx=0x7f7b2802ee10 comp rx=0 tx=0).stop 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 1 -- 10.20.193.16:0/2041417438 shutdown_connections 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 1 --2- 10.20.193.16:0/2041417438 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b34155e00 0x7f7b340ca700 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 1 -- 10.20.193.16:0/2041417438 >> 10.20.193.16:0/2041417438 conn(0x7f7b340ce6c0 msgr2=0x7f7b340cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 1 -- 10.20.193.16:0/2041417438 shutdown_connections 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 1 -- 10.20.193.16:0/2041417438 wait complete. 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 10 monclient: build_initial_monmap 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.524+0000 7f7b38e96640 10 monclient: monmap: 2026-02-17T06:00:56.535 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 librados: starting msgr at 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 librados: starting objecter 2026-02-17T06:00:56.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 Processor -- start 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 -- start start 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 librados: setting wanted keys 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 librados: calling monclient init 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient: init 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7b34074dc0 0x7f7b3414c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient(hunting): picked mon.noname-b con 0x7f7b34074dc0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7b34155e00 0x7f7b3414e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient(hunting): picked mon.noname-c con 0x7f7b34155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b341561e0 0x7f7b3414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient(hunting): picked mon.noname-a con 0x7f7b341561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b340c9360 con 0x7f7b341561e0 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f7b340ca1e0 con 0x7f7b34074dc0 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3415fa10 con 0x7f7b34155e00 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b38e96640 10 monclient(hunting): authenticate will time out at 1309.094577s 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b32d76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b341561e0 0x7f7b3414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.525+0000 7f7b32575640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7b34074dc0 0x7f7b3414c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32d76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b341561e0 0x7f7b3414eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:34858/0 (socket says 10.20.193.16:34858) 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32d76640 1 -- 10.20.193.16:0/3667329339 learned_addr learned my addr 10.20.193.16:0/3667329339 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32d76640 10 monclient(hunting): get_auth_request con 0x7f7b341561e0 auth_method 0 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b31d74640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7b34155e00 0x7f7b3414e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32575640 10 monclient(hunting): get_auth_request con 0x7f7b34074dc0 auth_method 0 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b31d74640 10 monclient(hunting): get_auth_request con 0x7f7b34155e00 auth_method 0 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b31d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b31d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b31d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b32575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b31d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b31d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.526+0000 7f7b31d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.527+0000 7f7b32d76640 10 monclient(hunting): handle_auth_done global_id 4935 payload 824 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.527+0000 7f7b32d76640 1 -- 10.20.193.16:0/3667329339 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7b34155e00 msgr2=0x7f7b3414e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.527+0000 7f7b32d76640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7b34155e00 0x7f7b3414e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.527+0000 7f7b32d76640 1 -- 10.20.193.16:0/3667329339 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7b34074dc0 msgr2=0x7f7b3414c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.527+0000 7f7b32d76640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7b34074dc0 0x7f7b3414c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.527+0000 7f7b32d76640 10 monclient: _finish_hunting 0 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.527+0000 7f7b32d76640 1 monclient: found mon.noname-a 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.527+0000 7f7b32d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.527+0000 7f7b32d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.527+0000 7f7b32d76640 1 -- 10.20.193.16:0/3667329339 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b341f4af0 con 0x7f7b341561e0 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.528+0000 7f7b32d76640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b341561e0 0x7f7b3414eed0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400cb40 tx=0x7f7b2400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.528+0000 7f7b1affd640 10 client.?.objecter ms_handle_connect 0x7f7b341561e0 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.528+0000 7f7b1affd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 1 -- 10.20.193.16:0/3667329339 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2400ad50 con 0x7f7b341561e0 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 10 monclient: dump: 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 10 monclient: _finish_auth 0 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 1 -- 10.20.193.16:0/3667329339 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b24012400 con 0x7f7b341561e0 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 1 -- 10.20.193.16:0/3667329339 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240127f0 con 0x7f7b341561e0 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b1affd640 10 monclient: dump: 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 5 monclient: authenticate success, global_id 4935 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 10 monclient: _renew_subs 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 1 -- 10.20.193.16:0/3667329339 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3414e4b0 con 0x7f7b341561e0 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b30d72640 4 set_mon_vals no callback set 2026-02-17T06:00:56.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 10 client.4935.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:56.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 10 monclient: _renew_subs 2026-02-17T06:00:56.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:56.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 1 -- 10.20.193.16:0/3667329339 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b341f4250 con 0x7f7b341561e0 2026-02-17T06:00:56.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 1 librados: init done 2026-02-17T06:00:56.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:56.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:56.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:56.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.530+0000 7f7b38e96640 1 -- 10.20.193.16:0/3667329339 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b340ca700 con 0x7f7b341561e0 2026-02-17T06:00:56.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.542+0000 7f7b1affd640 1 -- 10.20.193.16:0/3667329339 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f7b24012990 con 0x7f7b341561e0 2026-02-17T06:00:56.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.543+0000 7f7b1affd640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7b00079140 0x7f7b0007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.543+0000 7f7b32575640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7b00079140 0x7f7b0007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:56.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.543+0000 7f7b32575640 10 monclient: get_auth_request con 0x7f7b00079140 auth_method 0 2026-02-17T06:00:56.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.543+0000 7f7b1affd640 10 client.4935.objecter ms_handle_connect 0x7f7b00079140 2026-02-17T06:00:56.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.543+0000 7f7b32575640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7b00079140 0x7f7b0007b600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b28039000 tx=0x7f7b28033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:56.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.543+0000 7f7b1affd640 1 -- 10.20.193.16:0/3667329339 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f7b2405eb40 con 0x7f7b341561e0 2026-02-17T06:00:56.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.543+0000 7f7b1affd640 10 client.4935.objecter ms_dispatch2 0x7f7b34000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:56.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.543+0000 7f7b1affd640 3 client.4935.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:56.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.543+0000 7f7b1affd640 3 client.4935.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:56.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.544+0000 7f7b1affd640 20 client.4935.objecter dump_active .. 0 homeless 2026-02-17T06:00:56.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.644+0000 7f7b1affd640 1 -- 10.20.193.16:0/3667329339 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7b34074860 con 0x7f7b341561e0 2026-02-17T06:00:56.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.644+0000 7f7b1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:56.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.644+0000 7f7b1affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:56.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.746+0000 7f7b38e96640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:00:56.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.746+0000 7f7b38e96640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:56.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.746+0000 7f7b38e96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:56.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.746+0000 7f7b38e96640 1 -- 10.20.193.16:0/3667329339 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7b34152fd0 con 0x7f7b341561e0 2026-02-17T06:00:56.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.750+0000 7f7b1affd640 1 -- 10.20.193.16:0/3667329339 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f7b24062b00 con 0x7f7b341561e0 2026-02-17T06:00:56.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.750+0000 7f7b1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:00:56.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.750+0000 7f7b1affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:56.751 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:00:56.751 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:00:56.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.751+0000 7f7b38e96640 10 librados: watch_flush enter 2026-02-17T06:00:56.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.751+0000 7f7b38e96640 10 librados: watch_flush exit 2026-02-17T06:00:56.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.751+0000 7f7b38e96640 20 client.4935.objecter shutdown clearing up homeless session... 2026-02-17T06:00:56.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.751+0000 7f7b38e96640 10 client.4935.objecter successfully canceled tick 2026-02-17T06:00:56.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 -- 10.20.193.16:0/3667329339 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7b00079140 msgr2=0x7f7b0007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:56.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7b00079140 0x7f7b0007b600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b28039000 tx=0x7f7b28033000 comp rx=0 tx=0).stop 2026-02-17T06:00:56.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 10 monclient: shutdown 2026-02-17T06:00:56.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:56.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 -- 10.20.193.16:0/3667329339 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b341561e0 msgr2=0x7f7b3414eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:56.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b341561e0 0x7f7b3414eed0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400cb40 tx=0x7f7b2400a430 comp rx=0 tx=0).stop 2026-02-17T06:00:56.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 -- 10.20.193.16:0/3667329339 shutdown_connections 2026-02-17T06:00:56.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7b00079140 0x7f7b0007b600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f7b341561e0 0x7f7b3414eed0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f7b34155e00 0x7f7b3414e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 --2- 10.20.193.16:0/3667329339 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f7b34074dc0 0x7f7b3414c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 -- 10.20.193.16:0/3667329339 >> 10.20.193.16:0/3667329339 conn(0x7f7b340ce6c0 msgr2=0x7f7b340ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:56.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 -- 10.20.193.16:0/3667329339 shutdown_connections 2026-02-17T06:00:56.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 -- 10.20.193.16:0/3667329339 wait complete. 2026-02-17T06:00:56.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:56.752+0000 7f7b38e96640 1 librados: shutdown 2026-02-17T06:00:59.762 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:00:59.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.829+0000 7f6e36ca7640 10 monclient: get_monmap_and_config 2026-02-17T06:00:59.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.829+0000 7f6e36ca7640 10 monclient: build_initial_monmap 2026-02-17T06:00:59.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.829+0000 7f6e36ca7640 10 monclient: monmap: 2026-02-17T06:00:59.830 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:59.830 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:59.830 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:00:59.830250+0000 2026-02-17T06:00:59.830 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:00:59.830250+0000 2026-02-17T06:00:59.830 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:59.830 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:59.830 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:00:59.830 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:00:59.831 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:00:59.831 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:00:59.831 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:00:59.831 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:00:59.831 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.829+0000 7f6e36ca7640 1 Processor -- start 2026-02-17T06:00:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.829+0000 7f6e36ca7640 1 -- start start 2026-02-17T06:00:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.829+0000 7f6e36ca7640 10 monclient: init 2026-02-17T06:00:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 10 monclient: _add_conns ranks=[4,3,5,2,1,0] 2026-02-17T06:00:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f6e300700f0 0x7f6e300704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 10 monclient(hunting): picked mon.noname-b con 0x7f6e300700f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f6e300c7fe0 0x7f6e300c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 10 monclient(hunting): picked mon.noname-a con 0x7f6e300c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6e301536f0 0x7f6e30153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 10 monclient(hunting): picked mon.noname-c con 0x7f6e301536f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6e300c8970 con 0x7f6e300c7fe0 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f6e300c4820 con 0x7f6e300700f0 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:59.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6e300c4b00 con 0x7f6e301536f0 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e36ca7640 10 monclient(hunting): authenticate will time out at 1312.399215s 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e27fff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f6e300c7fe0 0x7f6e300c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e27fff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f6e300c7fe0 0x7f6e300c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:34870/0 (socket says 10.20.193.16:34870) 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e34a1c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f6e300700f0 0x7f6e300704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e27fff640 1 -- 10.20.193.16:0/480680993 learned_addr learned my addr 10.20.193.16:0/480680993 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e27fff640 10 monclient(hunting): get_auth_request con 0x7f6e300c7fe0 auth_method 0 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e27fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e3521d640 1 --2- 10.20.193.16:0/480680993 >> v2:10.20.193.148:3300/0 conn(0x7f6e301536f0 0x7f6e30153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:59.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e34a1c640 10 monclient(hunting): get_auth_request con 0x7f6e300700f0 auth_method 0 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e34a1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e34a1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e34a1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e3521d640 10 monclient(hunting): get_auth_request con 0x7f6e301536f0 auth_method 0 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e3521d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e3521d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.830+0000 7f6e3521d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e34a1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:59.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e34a1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e34a1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e3521d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e3521d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e3521d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 10 monclient(hunting): handle_auth_done global_id 4938 payload 274 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 1 -- 10.20.193.16:0/480680993 >> v2:10.20.193.148:3300/0 conn(0x7f6e301536f0 msgr2=0x7f6e30153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 1 --2- 10.20.193.16:0/480680993 >> v2:10.20.193.148:3300/0 conn(0x7f6e301536f0 0x7f6e30153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 1 -- 10.20.193.16:0/480680993 >> v2:10.20.193.128:3300/0 conn(0x7f6e300700f0 msgr2=0x7f6e300704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 1 --2- 10.20.193.16:0/480680993 >> v2:10.20.193.128:3300/0 conn(0x7f6e300700f0 0x7f6e300704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 1 monclient: found mon.noname-a 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.831+0000 7f6e27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:59.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.832+0000 7f6e27fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:59.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.832+0000 7f6e27fff640 1 -- 10.20.193.16:0/480680993 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e300c4de0 con 0x7f6e300c7fe0 2026-02-17T06:00:59.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.832+0000 7f6e27fff640 1 --2- 10.20.193.16:0/480680993 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 0x7f6e300c8430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e30070b90 tx=0x7f6e1802edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd129843cb7eebc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:59.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 1 -- 10.20.193.16:0/480680993 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1803c070 con 0x7f6e300c7fe0 2026-02-17T06:00:59.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:59.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 10 monclient: dump: 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:59.846 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 1 -- 10.20.193.16:0/480680993 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1802fab0 con 0x7f6e300c7fe0 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e36ca7640 5 monclient: authenticate success, global_id 4938 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 1 -- 10.20.193.16:0/480680993 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18037440 con 0x7f6e300c7fe0 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e36ca7640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e36ca7640 4 set_mon_vals no callback set 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.834+0000 7f6e277fe640 10 monclient: dump: 2026-02-17T06:00:59.847 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 10 monclient: shutdown 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:59.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 -- 10.20.193.16:0/480680993 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 msgr2=0x7f6e300c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 --2- 10.20.193.16:0/480680993 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 0x7f6e300c8430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e30070b90 tx=0x7f6e1802edf0 comp rx=0 tx=0).stop 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 -- 10.20.193.16:0/480680993 shutdown_connections 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 --2- 10.20.193.16:0/480680993 >> v2:10.20.193.148:3300/0 conn(0x7f6e301536f0 0x7f6e30153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 --2- 10.20.193.16:0/480680993 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 0x7f6e300c8430 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 --2- 10.20.193.16:0/480680993 >> v2:10.20.193.128:3300/0 conn(0x7f6e300700f0 0x7f6e300704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 -- 10.20.193.16:0/480680993 >> 10.20.193.16:0/480680993 conn(0x7f6e300cd770 msgr2=0x7f6e300cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 -- 10.20.193.16:0/480680993 shutdown_connections 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 -- 10.20.193.16:0/480680993 wait complete. 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 10 monclient: build_initial_monmap 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 10 monclient: monmap: 2026-02-17T06:00:59.849 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 librados: starting msgr at 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 librados: starting objecter 2026-02-17T06:00:59.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.835+0000 7f6e36ca7640 1 Processor -- start 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 1 -- start start 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 1 librados: setting wanted keys 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 1 librados: calling monclient init 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient: init 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e300700f0 0x7f6e30148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient(hunting): picked mon.noname-b con 0x7f6e300700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 0x7f6e3014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient(hunting): picked mon.noname-a con 0x7f6e300c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e301536f0 0x7f6e301ed300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:59.851 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient(hunting): picked mon.noname-c con 0x7f6e301536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e300c5680 con 0x7f6e300c7fe0 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f6e300c53a0 con 0x7f6e300700f0 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30154040 con 0x7f6e301536f0 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e36ca7640 10 monclient(hunting): authenticate will time out at 1312.405424s 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e27fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 0x7f6e3014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e34a1c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e300700f0 0x7f6e30148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e27fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 0x7f6e3014e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:34884/0 (socket says 10.20.193.16:34884) 2026-02-17T06:00:59.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e27fff640 1 -- 10.20.193.16:0/1741108662 learned_addr learned my addr 10.20.193.16:0/1741108662 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e27fff640 10 monclient(hunting): get_auth_request con 0x7f6e300c7fe0 auth_method 0 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e27fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.836+0000 7f6e3521d640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e301536f0 0x7f6e301ed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e34a1c640 10 monclient(hunting): get_auth_request con 0x7f6e300700f0 auth_method 0 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e34a1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e34a1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e34a1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:59.853 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e3521d640 10 monclient(hunting): get_auth_request con 0x7f6e301536f0 auth_method 0 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e3521d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e3521d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e3521d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e34a1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e34a1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e34a1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e3521d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e3521d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.837+0000 7f6e3521d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.838+0000 7f6e27fff640 10 monclient(hunting): handle_auth_done global_id 4941 payload 824 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.838+0000 7f6e27fff640 1 -- 10.20.193.16:0/1741108662 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e301536f0 msgr2=0x7f6e301ed300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:59.854 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.838+0000 7f6e27fff640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e301536f0 0x7f6e301ed300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.838+0000 7f6e27fff640 1 -- 10.20.193.16:0/1741108662 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e300700f0 msgr2=0x7f6e30148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.838+0000 7f6e27fff640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e300700f0 0x7f6e30148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.838+0000 7f6e27fff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.838+0000 7f6e27fff640 1 monclient: found mon.noname-a 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.838+0000 7f6e27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.838+0000 7f6e27fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.838+0000 7f6e27fff640 1 -- 10.20.193.16:0/1741108662 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e30148a60 con 0x7f6e300c7fe0 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.839+0000 7f6e27fff640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 0x7f6e3014e800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009a40 tx=0x7f6e180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.839+0000 7f6e257fa640 10 client.?.objecter ms_handle_connect 0x7f6e300c7fe0 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.839+0000 7f6e257fa640 10 client.?.objecter resend_mon_ops 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 1 -- 10.20.193.16:0/1741108662 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1803c040 con 0x7f6e300c7fe0 2026-02-17T06:00:59.855 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 10 monclient: dump: 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:59.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 10 monclient: _finish_auth 0 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 10 monclient: _check_auth_tickets 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e36ca7640 5 monclient: authenticate success, global_id 4941 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e36ca7640 10 monclient: _renew_subs 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 1 -- 10.20.193.16:0/1741108662 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e18042430 con 0x7f6e300c7fe0 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e36ca7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e36ca7640 1 -- 10.20.193.16:0/1741108662 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e30148d10 con 0x7f6e300c7fe0 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 1 -- 10.20.193.16:0/1741108662 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18042800 con 0x7f6e300c7fe0 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:59.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e257fa640 10 monclient: dump: 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e26ffd640 4 set_mon_vals no callback set 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e36ca7640 10 client.4941.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e36ca7640 10 monclient: _renew_subs 2026-02-17T06:00:59.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e36ca7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:59.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e36ca7640 1 -- 10.20.193.16:0/1741108662 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e301495b0 con 0x7f6e300c7fe0 2026-02-17T06:00:59.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.840+0000 7f6e36ca7640 1 librados: init done 2026-02-17T06:00:59.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.841+0000 7f6e36ca7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:59.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.841+0000 7f6e36ca7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:59.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.841+0000 7f6e36ca7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:00:59.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.841+0000 7f6e36ca7640 1 -- 10.20.193.16:0/1741108662 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e30074c00 con 0x7f6e300c7fe0 2026-02-17T06:00:59.948 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.948+0000 7f6e257fa640 1 -- 10.20.193.16:0/1741108662 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f6e180429a0 con 0x7f6e300c7fe0 2026-02-17T06:00:59.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.949+0000 7f6e257fa640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e00079190 0x7f6e0007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:59.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.949+0000 7f6e34a1c640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e00079190 0x7f6e0007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:59.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.949+0000 7f6e34a1c640 10 monclient: get_auth_request con 0x7f6e00079190 auth_method 0 2026-02-17T06:00:59.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.949+0000 7f6e257fa640 10 client.4941.objecter ms_handle_connect 0x7f6e00079190 2026-02-17T06:00:59.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.949+0000 7f6e34a1c640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e00079190 0x7f6e0007b650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009ba0 tx=0x7f6e20009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.957+0000 7f6e257fa640 1 -- 10.20.193.16:0/1741108662 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f6e180838d0 con 0x7f6e300c7fe0 2026-02-17T06:00:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.957+0000 7f6e257fa640 10 client.4941.objecter ms_dispatch2 0x7f6e30000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:00:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.957+0000 7f6e257fa640 3 client.4941.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:00:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.957+0000 7f6e257fa640 3 client.4941.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:00:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.957+0000 7f6e257fa640 20 client.4941.objecter dump_active .. 0 homeless 2026-02-17T06:00:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.957+0000 7f6e257fa640 1 -- 10.20.193.16:0/1741108662 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6e18087890 con 0x7f6e300c7fe0 2026-02-17T06:00:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.957+0000 7f6e257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:00:59.957+0000 7f6e257fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:00.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.059+0000 7f6e36ca7640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.059+0000 7f6e36ca7640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.059+0000 7f6e36ca7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.059+0000 7f6e36ca7640 1 -- 10.20.193.16:0/1741108662 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6e30152270 con 0x7f6e300c7fe0 2026-02-17T06:01:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.071+0000 7f6e257fa640 1 -- 10.20.193.16:0/1741108662 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f6e1808bb90 con 0x7f6e300c7fe0 2026-02-17T06:01:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.071+0000 7f6e257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.071+0000 7f6e257fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:00.072 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:00.072 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:00.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 10 librados: watch_flush enter 2026-02-17T06:01:00.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 10 librados: watch_flush exit 2026-02-17T06:01:00.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 20 client.4941.objecter shutdown clearing up homeless session... 2026-02-17T06:01:00.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 10 client.4941.objecter successfully canceled tick 2026-02-17T06:01:00.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 -- 10.20.193.16:0/1741108662 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e00079190 msgr2=0x7f6e0007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:00.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e00079190 0x7f6e0007b650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009ba0 tx=0x7f6e20009290 comp rx=0 tx=0).stop 2026-02-17T06:01:00.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 10 monclient: shutdown 2026-02-17T06:01:00.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:00.074 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 -- 10.20.193.16:0/1741108662 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 msgr2=0x7f6e3014e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:00.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 0x7f6e3014e800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009a40 tx=0x7f6e180047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:00.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 -- 10.20.193.16:0/1741108662 shutdown_connections 2026-02-17T06:01:00.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6e00079190 0x7f6e0007b650 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:00.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e301536f0 0x7f6e301ed300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:00.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e300c7fe0 0x7f6e3014e800 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:00.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 --2- 10.20.193.16:0/1741108662 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e300700f0 0x7f6e30148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:00.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 -- 10.20.193.16:0/1741108662 >> 10.20.193.16:0/1741108662 conn(0x7f6e300cd770 msgr2=0x7f6e300cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:00.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 -- 10.20.193.16:0/1741108662 shutdown_connections 2026-02-17T06:01:00.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.072+0000 7f6e36ca7640 1 -- 10.20.193.16:0/1741108662 wait complete. 2026-02-17T06:01:00.075 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:00.073+0000 7f6e36ca7640 1 librados: shutdown 2026-02-17T06:01:03.083 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:03.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.151+0000 7f196a31c640 10 monclient: get_monmap_and_config 2026-02-17T06:01:03.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.151+0000 7f196a31c640 10 monclient: build_initial_monmap 2026-02-17T06:01:03.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.151+0000 7f196a31c640 10 monclient: monmap: 2026-02-17T06:01:03.152 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:03.152362+0000 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:03.152362+0000 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:03.153 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:03.154 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:03.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.151+0000 7f196a31c640 1 Processor -- start 2026-02-17T06:01:03.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.151+0000 7f196a31c640 1 -- start start 2026-02-17T06:01:03.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.151+0000 7f196a31c640 10 monclient: init 2026-02-17T06:01:03.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:03.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 10 monclient: _add_conns ranks=[0,5,3,1,2,4] 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f19640700f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f19640c86f0 0x7f19640704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 10 monclient(hunting): picked mon.noname-c con 0x7f19640c86f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f19641536f0 0x7f1964153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 10 monclient(hunting): picked mon.noname-a con 0x7f19641536f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19640c4820 con 0x7f19640700f0 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f19640c4b00 con 0x7f19641536f0 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f19640c4de0 con 0x7f19640c86f0 2026-02-17T06:01:03.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f196a31c640 10 monclient(hunting): authenticate will time out at 1315.721413s 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f19637fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f19640c86f0 0x7f19640704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.152+0000 7f1968892640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f19641536f0 0x7f1964153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f19637fe640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f19640c86f0 0x7f19640704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:53592/0 (socket says 10.20.193.16:53592) 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f19637fe640 1 -- 10.20.193.16:0/4073698843 learned_addr learned my addr 10.20.193.16:0/4073698843 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f19637fe640 10 monclient(hunting): get_auth_request con 0x7f19640c86f0 auth_method 0 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f19637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f19637fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f19637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f1968892640 10 monclient(hunting): get_auth_request con 0x7f19641536f0 auth_method 0 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f1968892640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:03.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f1968892640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f1968892640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f19637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f19637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f19637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f1968892640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f1968892640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.153+0000 7f1968892640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.154+0000 7f19637fe640 10 monclient(hunting): handle_auth_done global_id 5072 payload 274 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.154+0000 7f19637fe640 1 -- 10.20.193.16:0/4073698843 >> v2:10.20.193.16:3300/0 conn(0x7f19641536f0 msgr2=0x7f1964153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.154+0000 7f19637fe640 1 --2- 10.20.193.16:0/4073698843 >> v2:10.20.193.16:3300/0 conn(0x7f19641536f0 0x7f1964153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.154+0000 7f19637fe640 1 -- 10.20.193.16:0/4073698843 >> v1:10.20.193.16:6789/0 conn(0x7f19640700f0 legacy=0x7f19640c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.154+0000 7f19637fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:03.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.154+0000 7f19637fe640 1 monclient: found mon.noname-c 2026-02-17T06:01:03.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.154+0000 7f19637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:03.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.154+0000 7f19637fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:03.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.154+0000 7f19637fe640 1 -- 10.20.193.16:0/4073698843 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19640c50c0 con 0x7f19640c86f0 2026-02-17T06:01:03.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.155+0000 7f19637fe640 1 --2- 10.20.193.16:0/4073698843 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f19640c86f0 0x7f19640704d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f19640c5b80 tx=0x7f195402ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0696147ab411b6f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:03.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 1 -- 10.20.193.16:0/4073698843 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195403c070 con 0x7f19640c86f0 2026-02-17T06:01:03.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:03.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:03.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 10 monclient: dump: 2026-02-17T06:01:03.167 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:03.167 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:03.167 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:03.167 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:03.167 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 1 -- 10.20.193.16:0/4073698843 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195402fa90 con 0x7f19640c86f0 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 5 monclient: authenticate success, global_id 5072 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:03.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 1 -- 10.20.193.16:0/4073698843 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195402fe60 con 0x7f19640c86f0 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 4 set_mon_vals no callback set 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f1962ffd640 10 monclient: dump: 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:03.169 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 10 monclient: shutdown 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 1 -- 10.20.193.16:0/4073698843 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f19640c86f0 msgr2=0x7f19640704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 1 --2- 10.20.193.16:0/4073698843 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f19640c86f0 0x7f19640704d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f19640c5b80 tx=0x7f195402ee60 comp rx=0 tx=0).stop 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 1 -- 10.20.193.16:0/4073698843 shutdown_connections 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 1 --2- 10.20.193.16:0/4073698843 >> v2:10.20.193.16:3300/0 conn(0x7f19641536f0 0x7f1964153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:03.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 1 --2- 10.20.193.16:0/4073698843 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f19640c86f0 0x7f19640704d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 1 -- 10.20.193.16:0/4073698843 >> 10.20.193.16:0/4073698843 conn(0x7f19640cd770 msgr2=0x7f19640cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 1 -- 10.20.193.16:0/4073698843 shutdown_connections 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.157+0000 7f196a31c640 1 -- 10.20.193.16:0/4073698843 wait complete. 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient: build_initial_monmap 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient: monmap: 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:03.171 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 librados: starting msgr at 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 librados: starting objecter 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 Processor -- start 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 -- start start 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 librados: setting wanted keys 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 librados: calling monclient init 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient: init 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:01:03.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f19640700f0 0x7f19641f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient(hunting): picked mon.noname-a con 0x7f19640700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f19640c86f0 0x7f1964143050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient(hunting): picked mon.noname-c con 0x7f19640c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f19641536f0 0x7f196413c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient(hunting): picked mon.noname-b con 0x7f19641536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1964154060 con 0x7f19640700f0 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f19640c5680 con 0x7f19641536f0 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1964074c00 con 0x7f19640c86f0 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:03.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.158+0000 7f196a31c640 10 monclient(hunting): authenticate will time out at 1315.727841s 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1963fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f19640700f0 0x7f19641f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f19637fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f19640c86f0 0x7f1964143050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1968892640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f19641536f0 0x7f196413c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1963fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f19640700f0 0x7f19641f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:53006/0 (socket says 10.20.193.16:53006) 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1963fff640 1 -- 10.20.193.16:0/77219990 learned_addr learned my addr 10.20.193.16:0/77219990 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1963fff640 10 monclient(hunting): get_auth_request con 0x7f19640700f0 auth_method 0 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1963fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1963fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1963fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f19637fe640 10 monclient(hunting): get_auth_request con 0x7f19640c86f0 auth_method 0 2026-02-17T06:01:03.174 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f19637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f19637fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f19637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f19637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f19637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f19637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1968892640 10 monclient(hunting): get_auth_request con 0x7f19641536f0 auth_method 0 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1968892640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1968892640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1968892640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1963fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:03.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1963fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1963fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1968892640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1968892640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.159+0000 7f1968892640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.161+0000 7f19637fe640 10 monclient(hunting): handle_auth_done global_id 5075 payload 824 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.161+0000 7f19637fe640 1 -- 10.20.193.16:0/77219990 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f19641536f0 msgr2=0x7f196413c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.161+0000 7f19637fe640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f19641536f0 0x7f196413c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.161+0000 7f19637fe640 1 -- 10.20.193.16:0/77219990 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f19640700f0 msgr2=0x7f19641f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.161+0000 7f19637fe640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f19640700f0 0x7f19641f2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.161+0000 7f19637fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.161+0000 7f19637fe640 1 monclient: found mon.noname-c 2026-02-17T06:01:03.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.161+0000 7f19637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.161+0000 7f19637fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.161+0000 7f19637fe640 1 -- 10.20.193.16:0/77219990 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19641e5010 con 0x7f19640c86f0 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.162+0000 7f19637fe640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f19640c86f0 0x7f1964143050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f19640c9260 tx=0x7f19540047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.162+0000 7f1960ff9640 10 client.?.objecter ms_handle_connect 0x7f19640c86f0 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.162+0000 7f1960ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 1 -- 10.20.193.16:0/77219990 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195403c040 con 0x7f19640c86f0 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 10 monclient: dump: 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:03.177 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 10 monclient: _finish_auth 0 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 5 monclient: authenticate success, global_id 5075 2026-02-17T06:01:03.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 10 monclient: _renew_subs 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 1 -- 10.20.193.16:0/77219990 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19641e52f0 con 0x7f19640c86f0 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 10 client.5075.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 10 monclient: _renew_subs 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 1 -- 10.20.193.16:0/77219990 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19641e58b0 con 0x7f19640c86f0 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 1 librados: init done 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f196a31c640 1 -- 10.20.193.16:0/77219990 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19640704d0 con 0x7f19640c86f0 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 1 -- 10.20.193.16:0/77219990 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195402f950 con 0x7f19640c86f0 2026-02-17T06:01:03.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 1 -- 10.20.193.16:0/77219990 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954042870 con 0x7f19640c86f0 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f19627fc640 4 set_mon_vals no callback set 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.163+0000 7f1960ff9640 10 monclient: dump: 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:03.180 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:03.181 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:03.181 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:03.181 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:03.181 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:03.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.276+0000 7f1960ff9640 1 -- 10.20.193.16:0/77219990 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1954042a10 con 0x7f19640c86f0 2026-02-17T06:01:03.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.276+0000 7f1960ff9640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1934079170 0x7f193407b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:03.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.276+0000 7f1963fff640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1934079170 0x7f193407b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:03.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.276+0000 7f1963fff640 10 monclient: get_auth_request con 0x7f1934079170 auth_method 0 2026-02-17T06:01:03.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.276+0000 7f1963fff640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1934079170 0x7f193407b630 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1950004660 tx=0x7f1950009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:03.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.276+0000 7f1960ff9640 10 client.5075.objecter ms_handle_connect 0x7f1934079170 2026-02-17T06:01:03.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.278+0000 7f1960ff9640 1 -- 10.20.193.16:0/77219990 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1954042d30 con 0x7f19640c86f0 2026-02-17T06:01:03.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.278+0000 7f1960ff9640 10 client.5075.objecter ms_dispatch2 0x7f1964000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:03.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.278+0000 7f1960ff9640 3 client.5075.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:03.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.278+0000 7f1960ff9640 3 client.5075.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:03.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.278+0000 7f1960ff9640 20 client.5075.objecter dump_active .. 0 homeless 2026-02-17T06:01:03.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.285+0000 7f1960ff9640 1 -- 10.20.193.16:0/77219990 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1954087960 con 0x7f19640c86f0 2026-02-17T06:01:03.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.285+0000 7f1960ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:03.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.285+0000 7f1960ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:03.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.385+0000 7f196a31c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:03.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.385+0000 7f196a31c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:03.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.385+0000 7f196a31c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:03.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.385+0000 7f196a31c640 1 -- 10.20.193.16:0/77219990 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1964151b00 con 0x7f19640c86f0 2026-02-17T06:01:03.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f1960ff9640 1 -- 10.20.193.16:0/77219990 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f195408bc60 con 0x7f19640c86f0 2026-02-17T06:01:03.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f1960ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:03.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f1960ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:03.388 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:03.388 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:03.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 10 librados: watch_flush enter 2026-02-17T06:01:03.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 10 librados: watch_flush exit 2026-02-17T06:01:03.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 20 client.5075.objecter shutdown clearing up homeless session... 2026-02-17T06:01:03.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 10 client.5075.objecter successfully canceled tick 2026-02-17T06:01:03.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 -- 10.20.193.16:0/77219990 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1934079170 msgr2=0x7f193407b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1934079170 0x7f193407b630 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1950004660 tx=0x7f1950009210 comp rx=0 tx=0).stop 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 10 monclient: shutdown 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 -- 10.20.193.16:0/77219990 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f19640c86f0 msgr2=0x7f1964143050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f19640c86f0 0x7f1964143050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f19640c9260 tx=0x7f19540047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 -- 10.20.193.16:0/77219990 shutdown_connections 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1934079170 0x7f193407b630 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f19641536f0 0x7f196413c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f19640c86f0 0x7f1964143050 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 --2- 10.20.193.16:0/77219990 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f19640700f0 0x7f19641f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 -- 10.20.193.16:0/77219990 >> 10.20.193.16:0/77219990 conn(0x7f19640cd770 msgr2=0x7f1964151180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 -- 10.20.193.16:0/77219990 shutdown_connections 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 -- 10.20.193.16:0/77219990 wait complete. 2026-02-17T06:01:03.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:03.389+0000 7f196a31c640 1 librados: shutdown 2026-02-17T06:01:06.398 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:06.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.465+0000 7fd21740e640 10 monclient: get_monmap_and_config 2026-02-17T06:01:06.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.465+0000 7fd21740e640 10 monclient: build_initial_monmap 2026-02-17T06:01:06.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.465+0000 7fd21740e640 10 monclient: monmap: 2026-02-17T06:01:06.466 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:06.466 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:06.466 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:06.466044+0000 2026-02-17T06:01:06.466 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:06.466044+0000 2026-02-17T06:01:06.466 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:06.466 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:06.466 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:06.466 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:06.466 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:06.467 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:06.467 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:06.467 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:06.467 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:06.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.465+0000 7fd21740e640 1 Processor -- start 2026-02-17T06:01:06.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.465+0000 7fd21740e640 1 -- start start 2026-02-17T06:01:06.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.465+0000 7fd21740e640 10 monclient: init 2026-02-17T06:01:06.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 10 monclient: _add_conns ranks=[5,3,2,4,1,0] 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd21014f020 0x7fd21006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 10 monclient(hunting): picked mon.noname-c con 0x7fd21014f020 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fd21006e120 0x7fd21015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 10 monclient(hunting): picked mon.noname-a con 0x7fd21006e120 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd21014f400 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2100c5b10 con 0x7fd21014f400 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd21006e620 con 0x7fd21006e120 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:06.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd21006e900 con 0x7fd21014f020 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd21740e640 10 monclient(hunting): authenticate will time out at 1319.035108s 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd214982640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fd21006e120 0x7fd21015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd215183640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fd21014f020 0x7fd21006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd214982640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fd21006e120 0x7fd21015c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:53010/0 (socket says 10.20.193.16:53010) 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd214982640 1 -- 10.20.193.16:0/3381287284 learned_addr learned my addr 10.20.193.16:0/3381287284 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd214982640 10 monclient(hunting): get_auth_request con 0x7fd21006e120 auth_method 0 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd214982640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd214982640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd214982640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:06.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd215183640 10 monclient(hunting): get_auth_request con 0x7fd21014f020 auth_method 0 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd215183640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd215183640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.466+0000 7fd215183640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.467+0000 7fd214982640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.467+0000 7fd214982640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.467+0000 7fd214982640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.467+0000 7fd215183640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.467+0000 7fd215183640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.467+0000 7fd215183640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.467+0000 7fd214982640 10 monclient(hunting): handle_auth_done global_id 4950 payload 274 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.467+0000 7fd214982640 1 -- 10.20.193.16:0/3381287284 >> v2:10.20.193.148:3300/0 conn(0x7fd21014f020 msgr2=0x7fd21006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.467+0000 7fd214982640 1 --2- 10.20.193.16:0/3381287284 >> v2:10.20.193.148:3300/0 conn(0x7fd21014f020 0x7fd21006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:06.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.468+0000 7fd214982640 1 -- 10.20.193.16:0/3381287284 >> v1:10.20.193.148:6789/0 conn(0x7fd21014f400 legacy=0x7fd2100c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:06.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.468+0000 7fd214982640 10 monclient: _finish_hunting 0 2026-02-17T06:01:06.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.468+0000 7fd214982640 1 monclient: found mon.noname-a 2026-02-17T06:01:06.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.468+0000 7fd214982640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:06.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.468+0000 7fd214982640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:06.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.468+0000 7fd214982640 1 -- 10.20.193.16:0/3381287284 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21006ebe0 con 0x7fd21006e120 2026-02-17T06:01:06.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.468+0000 7fd214982640 1 --2- 10.20.193.16:0/3381287284 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21006e120 0x7fd21015c3e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd2100ce3c0 tx=0x7fd20402ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bb03add8f48919 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:06.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 1 -- 10.20.193.16:0/3381287284 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20403c070 con 0x7fd21006e120 2026-02-17T06:01:06.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:06.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:06.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 10 monclient: dump: 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:06.481 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 10 monclient: _finish_auth 0 2026-02-17T06:01:06.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:06.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:06.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 1 -- 10.20.193.16:0/3381287284 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20402fbd0 con 0x7fd21006e120 2026-02-17T06:01:06.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd21740e640 5 monclient: authenticate success, global_id 4950 2026-02-17T06:01:06.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd21740e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:06.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:06.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 1 -- 10.20.193.16:0/3381287284 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204037440 con 0x7fd21006e120 2026-02-17T06:01:06.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd21740e640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd21740e640 4 set_mon_vals no callback set 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.470+0000 7fd1fffff640 10 monclient: dump: 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:06.483 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 10 monclient: shutdown 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 1 -- 10.20.193.16:0/3381287284 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21006e120 msgr2=0x7fd21015c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 1 --2- 10.20.193.16:0/3381287284 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21006e120 0x7fd21015c3e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd2100ce3c0 tx=0x7fd20402ef60 comp rx=0 tx=0).stop 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 1 -- 10.20.193.16:0/3381287284 shutdown_connections 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 1 --2- 10.20.193.16:0/3381287284 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21006e120 0x7fd21015c3e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 1 --2- 10.20.193.16:0/3381287284 >> v2:10.20.193.148:3300/0 conn(0x7fd21014f020 0x7fd21006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:06.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 1 -- 10.20.193.16:0/3381287284 >> 10.20.193.16:0/3381287284 conn(0x7fd2100cd770 msgr2=0x7fd2100cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 1 -- 10.20.193.16:0/3381287284 shutdown_connections 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 1 -- 10.20.193.16:0/3381287284 wait complete. 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 10 monclient: build_initial_monmap 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.471+0000 7fd21740e640 10 monclient: monmap: 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:06.485 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 librados: starting msgr at 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 librados: starting objecter 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 Processor -- start 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 -- start start 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 librados: setting wanted keys 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 librados: calling monclient init 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient: init 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:01:06.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd21006e120 0x7fd2101495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient(hunting): picked mon.noname-c con 0x7fd21006e120 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd21014f020 0x7fd210072f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient(hunting): picked mon.noname-b con 0x7fd21014f020 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21014f400 0x7fd2101f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient(hunting): picked mon.noname-a con 0x7fd21014f400 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2100c6720 con 0x7fd21014f400 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fd21006f1a0 con 0x7fd21014f020 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2100c68a0 con 0x7fd21006e120 2026-02-17T06:01:06.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd21740e640 10 monclient(hunting): authenticate will time out at 1319.041549s 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd215984640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21014f400 0x7fd2101f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd215984640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21014f400 0x7fd2101f3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:53018/0 (socket says 10.20.193.16:53018) 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd215984640 1 -- 10.20.193.16:0/3373919434 learned_addr learned my addr 10.20.193.16:0/3373919434 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd215984640 10 monclient(hunting): get_auth_request con 0x7fd21014f400 auth_method 0 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd215984640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd215984640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.472+0000 7fd215984640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215183640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd21006e120 0x7fd2101495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd214982640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd21014f020 0x7fd210072f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:06.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215984640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215984640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215984640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215183640 10 monclient(hunting): get_auth_request con 0x7fd21006e120 auth_method 0 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215183640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215183640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215183640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd214982640 10 monclient(hunting): get_auth_request con 0x7fd21014f020 auth_method 0 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd214982640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd214982640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd214982640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215183640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215183640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:06.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd215183640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd214982640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd214982640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.473+0000 7fd214982640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.474+0000 7fd215984640 10 monclient(hunting): handle_auth_done global_id 4953 payload 824 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.474+0000 7fd215984640 1 -- 10.20.193.16:0/3373919434 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd21006e120 msgr2=0x7fd2101495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.474+0000 7fd215984640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd21006e120 0x7fd2101495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.474+0000 7fd215984640 1 -- 10.20.193.16:0/3373919434 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd21014f020 msgr2=0x7fd210072f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.474+0000 7fd215984640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd21014f020 0x7fd210072f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.474+0000 7fd215984640 10 monclient: _finish_hunting 0 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.474+0000 7fd215984640 1 monclient: found mon.noname-a 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.474+0000 7fd215984640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.474+0000 7fd215984640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:06.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.474+0000 7fd215984640 1 -- 10.20.193.16:0/3373919434 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2101f41e0 con 0x7fd21014f400 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.475+0000 7fd215984640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21014f400 0x7fd2101f3940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00b9c0 tx=0x7fd20c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.475+0000 7fd1fdffb640 10 client.?.objecter ms_handle_connect 0x7fd21014f400 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.475+0000 7fd1fdffb640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 1 -- 10.20.193.16:0/3373919434 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c006e60 con 0x7fd21014f400 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 10 monclient: dump: 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:06.491 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 5 monclient: authenticate success, global_id 4953 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 1 -- 10.20.193.16:0/3373919434 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c010070 con 0x7fd21014f400 2026-02-17T06:01:06.492 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 1 -- 10.20.193.16:0/3373919434 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00c9d0 con 0x7fd21014f400 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1fdffb640 10 monclient: dump: 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:06.493 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd1ff7fe640 4 set_mon_vals no callback set 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 10 monclient: _renew_subs 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 1 -- 10.20.193.16:0/3373919434 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2101f44c0 con 0x7fd21014f400 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 10 client.4953.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 10 monclient: _renew_subs 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 1 -- 10.20.193.16:0/3373919434 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2100724a0 con 0x7fd21014f400 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 1 librados: init done 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:06.494 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:06.495 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:06.495 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.477+0000 7fd21740e640 1 -- 10.20.193.16:0/3373919434 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21006dbe0 con 0x7fd21014f400 2026-02-17T06:01:06.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.585+0000 7fd1fdffb640 1 -- 10.20.193.16:0/3373919434 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fd20c00cb70 con 0x7fd21014f400 2026-02-17T06:01:06.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.586+0000 7fd1fdffb640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1e0079190 0x7fd1e007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:06.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.586+0000 7fd215183640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1e0079190 0x7fd1e007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:06.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.586+0000 7fd215183640 10 monclient: get_auth_request con 0x7fd1e0079190 auth_method 0 2026-02-17T06:01:06.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.586+0000 7fd1fdffb640 10 client.4953.objecter ms_handle_connect 0x7fd1e0079190 2026-02-17T06:01:06.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.586+0000 7fd215183640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1e0079190 0x7fd1e007b650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd200009ba0 tx=0x7fd200009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:06.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.593+0000 7fd1fdffb640 1 -- 10.20.193.16:0/3373919434 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fd20c05dc60 con 0x7fd21014f400 2026-02-17T06:01:06.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.593+0000 7fd1fdffb640 10 client.4953.objecter ms_dispatch2 0x7fd210000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:06.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.593+0000 7fd1fdffb640 3 client.4953.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:06.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.593+0000 7fd1fdffb640 3 client.4953.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:06.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.593+0000 7fd1fdffb640 20 client.4953.objecter dump_active .. 0 homeless 2026-02-17T06:01:06.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.593+0000 7fd1fdffb640 1 -- 10.20.193.16:0/3373919434 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd20c061c20 con 0x7fd21014f400 2026-02-17T06:01:06.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.593+0000 7fd1fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:06.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.593+0000 7fd1fdffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:06.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.694+0000 7fd21740e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:06.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.694+0000 7fd21740e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:06.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.694+0000 7fd21740e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:06.696 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.694+0000 7fd21740e640 1 -- 10.20.193.16:0/3373919434 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd2101562c0 con 0x7fd21014f400 2026-02-17T06:01:06.697 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.697+0000 7fd1fdffb640 1 -- 10.20.193.16:0/3373919434 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd20c065f20 con 0x7fd21014f400 2026-02-17T06:01:06.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.697+0000 7fd1fdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:06.698 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.697+0000 7fd1fdffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:06.698 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:06.698 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:06.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 10 librados: watch_flush enter 2026-02-17T06:01:06.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 10 librados: watch_flush exit 2026-02-17T06:01:06.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 20 client.4953.objecter shutdown clearing up homeless session... 2026-02-17T06:01:06.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 10 client.4953.objecter successfully canceled tick 2026-02-17T06:01:06.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 -- 10.20.193.16:0/3373919434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1e0079190 msgr2=0x7fd1e007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:06.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1e0079190 0x7fd1e007b650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd200009ba0 tx=0x7fd200009290 comp rx=0 tx=0).stop 2026-02-17T06:01:06.701 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 10 monclient: shutdown 2026-02-17T06:01:06.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:06.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 -- 10.20.193.16:0/3373919434 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21014f400 msgr2=0x7fd2101f3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:06.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21014f400 0x7fd2101f3940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00b9c0 tx=0x7fd20c00be90 comp rx=0 tx=0).stop 2026-02-17T06:01:06.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 -- 10.20.193.16:0/3373919434 shutdown_connections 2026-02-17T06:01:06.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fd1e0079190 0x7fd1e007b650 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:06.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fd21014f020 0x7fd210072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:06.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fd21014f400 0x7fd2101f3940 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:06.702 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 --2- 10.20.193.16:0/3373919434 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fd21006e120 0x7fd2101495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:06.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 -- 10.20.193.16:0/3373919434 >> 10.20.193.16:0/3373919434 conn(0x7fd2100cd770 msgr2=0x7fd2100cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:06.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 -- 10.20.193.16:0/3373919434 shutdown_connections 2026-02-17T06:01:06.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 -- 10.20.193.16:0/3373919434 wait complete. 2026-02-17T06:01:06.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:06.698+0000 7fd21740e640 1 librados: shutdown 2026-02-17T06:01:09.225 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:06:48.053 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:09.226 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:06:48.052 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:09.228 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:06:48.054 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:09.228 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:06:48.055 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:09.236 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:06:48.062 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:09.237 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:06:48.064 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:09.254 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:06:48.043 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:09.256 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:06:48.045 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:09.264 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:06:48.053 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:09.708 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.775+0000 7f30b6c98640 10 monclient: get_monmap_and_config 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.775+0000 7f30b6c98640 10 monclient: build_initial_monmap 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.775+0000 7f30b6c98640 10 monclient: monmap: 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:09.776724+0000 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:09.776724+0000 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 1 Processor -- start 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 1 -- start start 2026-02-17T06:01:09.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 10 monclient: init 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 10 monclient: _add_conns ranks=[5,1,0,4,3,2] 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f30b00700f0 0x7f30b00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 10 monclient(hunting): picked mon.noname-c con 0x7f30b00700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f30b00c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f30b01536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30b00c4820 con 0x7f30b01536f0 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30b00c4b00 con 0x7f30b00c7fe0 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f30b00c4de0 con 0x7f30b00700f0 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.776+0000 7f30b6c98640 10 monclient(hunting): authenticate will time out at 1322.345840s 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.777+0000 7f30b520e640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f30b01536f0 0x7f30b0153b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:43852/0 (socket says 10.20.193.16:43852) 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.777+0000 7f30b520e640 1 -- 10.20.193.16:0/3539623105 learned_addr learned my addr 10.20.193.16:0/3539623105 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:09.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.777+0000 7f30b4a0d640 1 --2- 10.20.193.16:0/3539623105 >> v2:10.20.193.148:3300/0 conn(0x7f30b00700f0 0x7f30b00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:09.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.777+0000 7f30b4a0d640 10 monclient(hunting): get_auth_request con 0x7f30b00700f0 auth_method 0 2026-02-17T06:01:09.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.777+0000 7f30b4a0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:09.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.777+0000 7f30b4a0d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:09.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.777+0000 7f30b4a0d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:09.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30b4a0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:09.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30b4a0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:09.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30b4a0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:09.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 1 -- 10.20.193.16:0/3539623105 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3338538797 0 0) 0x7f30b00c4820 con 0x7f30b01536f0 2026-02-17T06:01:09.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 10 monclient(hunting): _negotiate 2026-02-17T06:01:09.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:09.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:09.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 10 monclient(hunting): my global_id is 4956 2026-02-17T06:01:09.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 1 -- 10.20.193.16:0/3539623105 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f309c003620 con 0x7f30b01536f0 2026-02-17T06:01:09.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 1 -- 10.20.193.16:0/3539623105 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 189508155 0 0) 0x7f30b00c4b00 con 0x7f30b00c7fe0 2026-02-17T06:01:09.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 10 monclient(hunting): _negotiate 2026-02-17T06:01:09.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:09.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:09.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 10 monclient(hunting): my global_id is 5023 2026-02-17T06:01:09.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.778+0000 7f30a77fe640 1 -- 10.20.193.16:0/3539623105 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30b00c4820 con 0x7f30b00c7fe0 2026-02-17T06:01:09.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.779+0000 7f30b4a0d640 10 monclient(hunting): handle_auth_done global_id 5084 payload 274 2026-02-17T06:01:09.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.779+0000 7f30b4a0d640 1 -- 10.20.193.16:0/3539623105 >> v1:10.20.193.128:6789/0 conn(0x7f30b00c7fe0 legacy=0x7f30b00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:09.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.779+0000 7f30b4a0d640 1 -- 10.20.193.16:0/3539623105 >> v1:10.20.193.16:6789/0 conn(0x7f30b01536f0 legacy=0x7f30b0153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:09.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.779+0000 7f30b4a0d640 10 monclient: _finish_hunting 0 2026-02-17T06:01:09.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.779+0000 7f30b4a0d640 1 monclient: found mon.noname-c 2026-02-17T06:01:09.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.779+0000 7f30b4a0d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:09.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.779+0000 7f30b4a0d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:09.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.779+0000 7f30b4a0d640 1 -- 10.20.193.16:0/3539623105 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b00c50c0 con 0x7f30b00700f0 2026-02-17T06:01:09.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.781+0000 7f30b4a0d640 1 --2- 10.20.193.16:0/3539623105 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30b00700f0 0x7f30b00704d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f30a00098a0 tx=0x7f30a002edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81284d28f54d5f0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 1 -- 10.20.193.16:0/3539623105 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a003c070 con 0x7f30b00700f0 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 10 monclient: dump: 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 5 monclient: authenticate success, global_id 5084 2026-02-17T06:01:09.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 1 -- 10.20.193.16:0/3539623105 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a002fae0 con 0x7f30b00700f0 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 1 -- 10.20.193.16:0/3539623105 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a0037440 con 0x7f30b00700f0 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 4 set_mon_vals no callback set 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30a77fe640 10 monclient: dump: 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 10 monclient: shutdown 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 1 -- 10.20.193.16:0/3539623105 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30b00700f0 msgr2=0x7f30b00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 1 --2- 10.20.193.16:0/3539623105 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30b00700f0 0x7f30b00704d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f30a00098a0 tx=0x7f30a002edd0 comp rx=0 tx=0).stop 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 1 -- 10.20.193.16:0/3539623105 shutdown_connections 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 1 --2- 10.20.193.16:0/3539623105 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30b00700f0 0x7f30b00704d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 1 -- 10.20.193.16:0/3539623105 >> 10.20.193.16:0/3539623105 conn(0x7f30b00cd770 msgr2=0x7f30b00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 1 -- 10.20.193.16:0/3539623105 shutdown_connections 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.783+0000 7f30b6c98640 1 -- 10.20.193.16:0/3539623105 wait complete. 2026-02-17T06:01:09.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 10 monclient: build_initial_monmap 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 10 monclient: monmap: 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 1 librados: starting msgr at 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 1 librados: starting objecter 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 1 Processor -- start 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 1 -- start start 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 1 librados: setting wanted keys 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 1 librados: calling monclient init 2026-02-17T06:01:09.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 10 monclient: init 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30b00700f0 0x7f30b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 10 monclient(hunting): picked mon.noname-c con 0x7f30b00700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30b00c7fe0 0x7f30b006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 10 monclient(hunting): picked mon.noname-b con 0x7f30b00c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30b01536f0 0x7f30b0139690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 10 monclient(hunting): picked mon.noname-a con 0x7f30b01536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.784+0000 7f30b6c98640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b6c98640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30b0074c00 con 0x7f30b01536f0 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b6c98640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b6c98640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f30b00c5680 con 0x7f30b00c7fe0 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b6c98640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b6c98640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30b0074d80 con 0x7f30b00700f0 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b6c98640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b6c98640 10 monclient(hunting): authenticate will time out at 1322.354011s 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b520e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30b01536f0 0x7f30b0139690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30a7fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30b00c7fe0 0x7f30b006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b520e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30b01536f0 0x7f30b0139690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:53024/0 (socket says 10.20.193.16:53024) 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b4a0d640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30b00700f0 0x7f30b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b520e640 1 -- 10.20.193.16:0/3559811393 learned_addr learned my addr 10.20.193.16:0/3559811393 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b520e640 10 monclient(hunting): get_auth_request con 0x7f30b01536f0 auth_method 0 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b520e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b520e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b520e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30a7fff640 10 monclient(hunting): get_auth_request con 0x7f30b00c7fe0 auth_method 0 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b4a0d640 10 monclient(hunting): get_auth_request con 0x7f30b00700f0 auth_method 0 2026-02-17T06:01:09.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b4a0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b4a0d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b4a0d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b520e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b520e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30b520e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.785+0000 7f30a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.786+0000 7f30b4a0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.786+0000 7f30b4a0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.786+0000 7f30b4a0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.787+0000 7f30b520e640 10 monclient(hunting): handle_auth_done global_id 4959 payload 824 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.787+0000 7f30b520e640 1 -- 10.20.193.16:0/3559811393 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30b00700f0 msgr2=0x7f30b006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.787+0000 7f30b520e640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30b00700f0 0x7f30b006d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.787+0000 7f30b520e640 1 -- 10.20.193.16:0/3559811393 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30b00c7fe0 msgr2=0x7f30b006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.787+0000 7f30b520e640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30b00c7fe0 0x7f30b006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.787+0000 7f30b520e640 10 monclient: _finish_hunting 0 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.787+0000 7f30b520e640 1 monclient: found mon.noname-a 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.787+0000 7f30b520e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.787+0000 7f30b520e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:09.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.787+0000 7f30b520e640 1 -- 10.20.193.16:0/3559811393 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b0142270 con 0x7f30b01536f0 2026-02-17T06:01:09.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.788+0000 7f30b520e640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30b01536f0 0x7f30b0139690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00cad0 tx=0x7f30ac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:09.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.788+0000 7f30a57fa640 10 client.?.objecter ms_handle_connect 0x7f30b01536f0 2026-02-17T06:01:09.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.788+0000 7f30a57fa640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 1 -- 10.20.193.16:0/3559811393 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00ac50 con 0x7f30b01536f0 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 10 monclient: dump: 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 10 monclient: _finish_auth 0 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 10 monclient: _check_auth_tickets 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 1 -- 10.20.193.16:0/3559811393 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ac00ae10 con 0x7f30b01536f0 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 1 -- 10.20.193.16:0/3559811393 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac012640 con 0x7f30b01536f0 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 5 monclient: authenticate success, global_id 4959 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 10 monclient: _renew_subs 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a6ffd640 4 set_mon_vals no callback set 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 1 -- 10.20.193.16:0/3559811393 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b0142550 con 0x7f30b01536f0 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:09.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30a57fa640 10 monclient: dump: 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 10 client.4959.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 10 monclient: _renew_subs 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 1 -- 10.20.193.16:0/3559811393 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b0142b10 con 0x7f30b01536f0 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 1 librados: init done 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:09.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.790+0000 7f30b6c98640 1 -- 10.20.193.16:0/3559811393 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b00704d0 con 0x7f30b01536f0 2026-02-17T06:01:09.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.802+0000 7f30a57fa640 1 -- 10.20.193.16:0/3559811393 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f30ac0127e0 con 0x7f30b01536f0 2026-02-17T06:01:09.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.803+0000 7f30a57fa640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3080079170 0x7f308007b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:09.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.803+0000 7f30b4a0d640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3080079170 0x7f308007b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:09.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.803+0000 7f30b4a0d640 10 monclient: get_auth_request con 0x7f3080079170 auth_method 0 2026-02-17T06:01:09.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.803+0000 7f30b4a0d640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3080079170 0x7f308007b630 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f30a0039000 tx=0x7f30a0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:09.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.803+0000 7f30a57fa640 10 client.4959.objecter ms_handle_connect 0x7f3080079170 2026-02-17T06:01:09.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.805+0000 7f30a57fa640 1 -- 10.20.193.16:0/3559811393 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f30ac05ea70 con 0x7f30b01536f0 2026-02-17T06:01:09.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.805+0000 7f30a57fa640 10 client.4959.objecter ms_dispatch2 0x7f30b0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:09.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.805+0000 7f30a57fa640 3 client.4959.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:09.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.805+0000 7f30a57fa640 3 client.4959.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:09.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.805+0000 7f30a57fa640 20 client.4959.objecter dump_active .. 0 homeless 2026-02-17T06:01:09.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.910+0000 7f30a57fa640 1 -- 10.20.193.16:0/3559811393 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f30ac062a30 con 0x7f30b01536f0 2026-02-17T06:01:09.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.910+0000 7f30a57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:09.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:09.910+0000 7f30a57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:10.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.010+0000 7f30b6c98640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:10.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.010+0000 7f30b6c98640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:10.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.010+0000 7f30b6c98640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:10.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.010+0000 7f30b6c98640 1 -- 10.20.193.16:0/3559811393 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f30b01573f0 con 0x7f30b01536f0 2026-02-17T06:01:10.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.013+0000 7f30a57fa640 1 -- 10.20.193.16:0/3559811393 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f30ac066d30 con 0x7f30b01536f0 2026-02-17T06:01:10.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.013+0000 7f30a57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:10.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.013+0000 7f30a57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:10.014 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:10.014 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:10.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.014+0000 7f30b6c98640 10 librados: watch_flush enter 2026-02-17T06:01:10.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.014+0000 7f30b6c98640 10 librados: watch_flush exit 2026-02-17T06:01:10.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.014+0000 7f30b6c98640 20 client.4959.objecter shutdown clearing up homeless session... 2026-02-17T06:01:10.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.014+0000 7f30b6c98640 10 client.4959.objecter successfully canceled tick 2026-02-17T06:01:10.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 -- 10.20.193.16:0/3559811393 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3080079170 msgr2=0x7f308007b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:10.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3080079170 0x7f308007b630 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f30a0039000 tx=0x7f30a0033000 comp rx=0 tx=0).stop 2026-02-17T06:01:10.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 10 monclient: shutdown 2026-02-17T06:01:10.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:10.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 -- 10.20.193.16:0/3559811393 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30b01536f0 msgr2=0x7f30b0139690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:10.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30b01536f0 0x7f30b0139690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00cad0 tx=0x7f30ac00a060 comp rx=0 tx=0).stop 2026-02-17T06:01:10.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 -- 10.20.193.16:0/3559811393 shutdown_connections 2026-02-17T06:01:10.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3080079170 0x7f308007b630 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:10.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30b00c7fe0 0x7f30b006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:10.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30b01536f0 0x7f30b0139690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:10.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 --2- 10.20.193.16:0/3559811393 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30b00700f0 0x7f30b006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:10.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 -- 10.20.193.16:0/3559811393 >> 10.20.193.16:0/3559811393 conn(0x7f30b00cd770 msgr2=0x7f30b0153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:10.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 -- 10.20.193.16:0/3559811393 shutdown_connections 2026-02-17T06:01:10.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 -- 10.20.193.16:0/3559811393 wait complete. 2026-02-17T06:01:10.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:10.015+0000 7f30b6c98640 1 librados: shutdown 2026-02-17T06:01:13.024 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.093+0000 7f6d60996640 10 monclient: get_monmap_and_config 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.093+0000 7f6d60996640 10 monclient: build_initial_monmap 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.093+0000 7f6d60996640 10 monclient: monmap: 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:13.094849+0000 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:13.094849+0000 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:13.094 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:13.095 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:13.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 1 Processor -- start 2026-02-17T06:01:13.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 1 -- start start 2026-02-17T06:01:13.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 10 monclient: init 2026-02-17T06:01:13.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:13.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 10 monclient: _add_conns ranks=[5,0,2,4,3,1] 2026-02-17T06:01:13.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6d5814f020 0x7f6d5806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 10 monclient(hunting): picked mon.noname-c con 0x7f6d5814f020 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6d5806e120 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6d5815caf0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d580c5b10 con 0x7f6d5806e120 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5806e620 con 0x7f6d5815caf0 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5806e900 con 0x7f6d5814f020 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.094+0000 7f6d60996640 10 monclient(hunting): authenticate will time out at 1325.663888s 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.095+0000 7f6d5df0a640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f6d5806e120 0x7f6d5815c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:34774/0 (socket says 10.20.193.16:34774) 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.095+0000 7f6d5df0a640 1 -- 10.20.193.16:0/3514600968 learned_addr learned my addr 10.20.193.16:0/3514600968 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.095+0000 7f6d5e70b640 1 --2- 10.20.193.16:0/3514600968 >> v2:10.20.193.148:3300/0 conn(0x7f6d5814f020 0x7f6d5806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.095+0000 7f6d5e70b640 10 monclient(hunting): get_auth_request con 0x7f6d5814f020 auth_method 0 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.095+0000 7f6d5e70b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.095+0000 7f6d5e70b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.095+0000 7f6d5e70b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.096+0000 7f6d5e70b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.096+0000 7f6d5e70b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:13.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.096+0000 7f6d5e70b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.096+0000 7f6d5d709640 1 -- 10.20.193.16:0/3514600968 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1073030059 0 0) 0x7f6d580c5b10 con 0x7f6d5806e120 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.096+0000 7f6d5d709640 10 monclient(hunting): _negotiate 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.096+0000 7f6d5d709640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.096+0000 7f6d5d709640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.096+0000 7f6d5d709640 10 monclient(hunting): my global_id is 4962 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.096+0000 7f6d5d709640 1 -- 10.20.193.16:0/3514600968 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d48003620 con 0x7f6d5806e120 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.097+0000 7f6d5e70b640 10 monclient(hunting): handle_auth_done global_id 5090 payload 274 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.097+0000 7f6d5e70b640 1 -- 10.20.193.16:0/3514600968 >> v1:10.20.193.148:6789/0 conn(0x7f6d5815caf0 legacy=0x7f6d580c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.097+0000 7f6d5e70b640 1 -- 10.20.193.16:0/3514600968 >> v1:10.20.193.16:6789/0 conn(0x7f6d5806e120 legacy=0x7f6d5815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.097+0000 7f6d5e70b640 10 monclient: _finish_hunting 0 2026-02-17T06:01:13.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.097+0000 7f6d5e70b640 1 monclient: found mon.noname-c 2026-02-17T06:01:13.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.097+0000 7f6d5e70b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:13.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.097+0000 7f6d5e70b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:13.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.097+0000 7f6d5e70b640 1 -- 10.20.193.16:0/3514600968 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5806ebe0 con 0x7f6d5814f020 2026-02-17T06:01:13.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.099+0000 7f6d5e70b640 1 --2- 10.20.193.16:0/3514600968 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5814f020 0x7f6d5806dbe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d500098a0 tx=0x7f6d5002eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62a7a65fce9aa38b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:13.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 1 -- 10.20.193.16:0/3514600968 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5003c070 con 0x7f6d5814f020 2026-02-17T06:01:13.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:13.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:13.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 10 monclient: dump: 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 10 monclient: _finish_auth 0 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 10 monclient: _check_auth_tickets 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 1 -- 10.20.193.16:0/3514600968 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5002fb70 con 0x7f6d5814f020 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 5 monclient: authenticate success, global_id 5090 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 1 -- 10.20.193.16:0/3514600968 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50037440 con 0x7f6d5814f020 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d5d709640 10 monclient: dump: 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:13.103 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 4 set_mon_vals no callback set 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 10 monclient: shutdown 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 1 -- 10.20.193.16:0/3514600968 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5814f020 msgr2=0x7f6d5806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 1 --2- 10.20.193.16:0/3514600968 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5814f020 0x7f6d5806dbe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d500098a0 tx=0x7f6d5002eeb0 comp rx=0 tx=0).stop 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 1 -- 10.20.193.16:0/3514600968 shutdown_connections 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 1 --2- 10.20.193.16:0/3514600968 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5814f020 0x7f6d5806dbe0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 1 -- 10.20.193.16:0/3514600968 >> 10.20.193.16:0/3514600968 conn(0x7f6d580cd770 msgr2=0x7f6d580cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 1 -- 10.20.193.16:0/3514600968 shutdown_connections 2026-02-17T06:01:13.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.102+0000 7f6d60996640 1 -- 10.20.193.16:0/3514600968 wait complete. 2026-02-17T06:01:13.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient: build_initial_monmap 2026-02-17T06:01:13.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient: monmap: 2026-02-17T06:01:13.105 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:13.105 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:13.105 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:13.105 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:13.105 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 librados: starting msgr at 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 librados: starting objecter 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 Processor -- start 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 -- start start 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 librados: setting wanted keys 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 librados: calling monclient init 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient: init 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5806e120 0x7f6d581f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient(hunting): picked mon.noname-a con 0x7f6d5806e120 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5814f020 0x7f6d58142f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient(hunting): picked mon.noname-c con 0x7f6d5814f020 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5815caf0 0x7f6d581ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient(hunting): picked mon.noname-b con 0x7f6d5815caf0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d580c6720 con 0x7f6d5806e120 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5806f1a0 con 0x7f6d5815caf0 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6d580c68a0 con 0x7f6d5814f020 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.103+0000 7f6d60996640 10 monclient(hunting): authenticate will time out at 1325.672775s 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5e70b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5806e120 0x7f6d581f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5df0a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5814f020 0x7f6d58142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5ef0c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5815caf0 0x7f6d581ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5e70b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5806e120 0x7f6d581f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:58550/0 (socket says 10.20.193.16:58550) 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5e70b640 1 -- 10.20.193.16:0/167303493 learned_addr learned my addr 10.20.193.16:0/167303493 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5e70b640 10 monclient(hunting): get_auth_request con 0x7f6d5806e120 auth_method 0 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5e70b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5e70b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5e70b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5df0a640 10 monclient(hunting): get_auth_request con 0x7f6d5814f020 auth_method 0 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5df0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:13.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5df0a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5df0a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5ef0c640 10 monclient(hunting): get_auth_request con 0x7f6d5815caf0 auth_method 0 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5ef0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5ef0c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5ef0c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5e70b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5e70b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5e70b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5df0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5df0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5df0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5ef0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5ef0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.104+0000 7f6d5ef0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.105+0000 7f6d5e70b640 10 monclient(hunting): handle_auth_done global_id 4965 payload 824 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.105+0000 7f6d5e70b640 1 -- 10.20.193.16:0/167303493 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5814f020 msgr2=0x7f6d58142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.105+0000 7f6d5e70b640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5814f020 0x7f6d58142f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.105+0000 7f6d5e70b640 1 -- 10.20.193.16:0/167303493 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5815caf0 msgr2=0x7f6d581ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.105+0000 7f6d5e70b640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5815caf0 0x7f6d581ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.105+0000 7f6d5e70b640 10 monclient: _finish_hunting 0 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.105+0000 7f6d5e70b640 1 monclient: found mon.noname-a 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.105+0000 7f6d5e70b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.105+0000 7f6d5e70b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:13.107 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.105+0000 7f6d5e70b640 1 -- 10.20.193.16:0/167303493 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d581f35d0 con 0x7f6d5806e120 2026-02-17T06:01:13.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.107+0000 7f6d5e70b640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5806e120 0x7f6d581f54d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6d500098a0 tx=0x7f6d500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:13.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.107+0000 7f6d42ffd640 10 client.?.objecter ms_handle_connect 0x7f6d5806e120 2026-02-17T06:01:13.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.107+0000 7f6d42ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 1 -- 10.20.193.16:0/167303493 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50046020 con 0x7f6d5806e120 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 10 monclient: dump: 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 1 -- 10.20.193.16:0/167303493 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d50042620 con 0x7f6d5806e120 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 1 -- 10.20.193.16:0/167303493 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5003c040 con 0x7f6d5806e120 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d42ffd640 10 monclient: dump: 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:13.109 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d5cf08640 4 set_mon_vals no callback set 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d60996640 5 monclient: authenticate success, global_id 4965 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d60996640 10 monclient: _renew_subs 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d60996640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d60996640 1 -- 10.20.193.16:0/167303493 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d581f38b0 con 0x7f6d5806e120 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d60996640 10 client.4965.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d60996640 10 monclient: _renew_subs 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d60996640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d60996640 1 -- 10.20.193.16:0/167303493 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d581f3e70 con 0x7f6d5806e120 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.108+0000 7f6d60996640 1 librados: init done 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.109+0000 7f6d60996640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.109+0000 7f6d60996640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.109+0000 7f6d60996640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:13.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.109+0000 7f6d60996640 1 -- 10.20.193.16:0/167303493 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5814f400 con 0x7f6d5806e120 2026-02-17T06:01:13.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.214+0000 7f6d42ffd640 1 -- 10.20.193.16:0/167303493 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f6d5003f030 con 0x7f6d5806e120 2026-02-17T06:01:13.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.215+0000 7f6d42ffd640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079190 0x7f6d2807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:13.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.215+0000 7f6d5df0a640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079190 0x7f6d2807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:13.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.215+0000 7f6d5df0a640 10 monclient: get_auth_request con 0x7f6d28079190 auth_method 0 2026-02-17T06:01:13.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.215+0000 7f6d42ffd640 10 client.4965.objecter ms_handle_connect 0x7f6d28079190 2026-02-17T06:01:13.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.215+0000 7f6d5df0a640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079190 0x7f6d2807b650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6d44007b60 tx=0x7f6d44007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:13.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.222+0000 7f6d42ffd640 1 -- 10.20.193.16:0/167303493 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f6d5002fa30 con 0x7f6d5806e120 2026-02-17T06:01:13.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.222+0000 7f6d42ffd640 10 client.4965.objecter ms_dispatch2 0x7f6d58000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:13.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.222+0000 7f6d42ffd640 3 client.4965.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:13.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.222+0000 7f6d42ffd640 3 client.4965.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:13.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.223+0000 7f6d42ffd640 20 client.4965.objecter dump_active .. 0 homeless 2026-02-17T06:01:13.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.223+0000 7f6d42ffd640 1 -- 10.20.193.16:0/167303493 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6d50050030 con 0x7f6d5806e120 2026-02-17T06:01:13.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.223+0000 7f6d42ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:13.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.223+0000 7f6d42ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:13.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.322+0000 7f6d60996640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:13.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.323+0000 7f6d60996640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:13.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.323+0000 7f6d60996640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:13.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.323+0000 7f6d60996640 1 -- 10.20.193.16:0/167303493 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6d5814d910 con 0x7f6d5806e120 2026-02-17T06:01:13.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.325+0000 7f6d42ffd640 1 -- 10.20.193.16:0/167303493 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f6d500873c0 con 0x7f6d5806e120 2026-02-17T06:01:13.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.325+0000 7f6d42ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:13.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.325+0000 7f6d42ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:13.326 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:13.326 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:13.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 10 librados: watch_flush enter 2026-02-17T06:01:13.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 10 librados: watch_flush exit 2026-02-17T06:01:13.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 20 client.4965.objecter shutdown clearing up homeless session... 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 10 client.4965.objecter successfully canceled tick 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 -- 10.20.193.16:0/167303493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079190 msgr2=0x7f6d2807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079190 0x7f6d2807b650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6d44007b60 tx=0x7f6d44007ab0 comp rx=0 tx=0).stop 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 10 monclient: shutdown 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 -- 10.20.193.16:0/167303493 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5806e120 msgr2=0x7f6d581f54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5806e120 0x7f6d581f54d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6d500098a0 tx=0x7f6d500047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 -- 10.20.193.16:0/167303493 shutdown_connections 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6d28079190 0x7f6d2807b650 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6d5815caf0 0x7f6d581ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6d5814f020 0x7f6d58142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 --2- 10.20.193.16:0/167303493 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6d5806e120 0x7f6d581f54d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 -- 10.20.193.16:0/167303493 >> 10.20.193.16:0/167303493 conn(0x7f6d580cd770 msgr2=0x7f6d5814de30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:13.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 -- 10.20.193.16:0/167303493 shutdown_connections 2026-02-17T06:01:13.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 -- 10.20.193.16:0/167303493 wait complete. 2026-02-17T06:01:13.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:13.327+0000 7f6d60996640 1 librados: shutdown 2026-02-17T06:01:16.337 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:16.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.408+0000 7f1c9052c640 10 monclient: get_monmap_and_config 2026-02-17T06:01:16.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.408+0000 7f1c9052c640 10 monclient: build_initial_monmap 2026-02-17T06:01:16.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.408+0000 7f1c9052c640 10 monclient: monmap: 2026-02-17T06:01:16.409 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:16.409 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:16.409 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:16.409098+0000 2026-02-17T06:01:16.409 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:16.409098+0000 2026-02-17T06:01:16.409 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.408+0000 7f1c9052c640 1 Processor -- start 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.408+0000 7f1c9052c640 1 -- start start 2026-02-17T06:01:16.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.408+0000 7f1c9052c640 10 monclient: init 2026-02-17T06:01:16.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:16.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 10 monclient: _add_conns ranks=[2,4,3,1,5,0] 2026-02-17T06:01:16.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1c880700f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:16.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1c880c86f0 0x7f1c880704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:16.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 10 monclient(hunting): picked mon.noname-b con 0x7f1c880c86f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1c881536f0 0x7f1c88153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 10 monclient(hunting): picked mon.noname-a con 0x7f1c881536f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c880c4820 con 0x7f1c880700f0 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1c880c4b00 con 0x7f1c881536f0 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f1c880c4de0 con 0x7f1c880c86f0 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c9052c640 10 monclient(hunting): authenticate will time out at 1328.978066s 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8eaa2640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1c881536f0 0x7f1c88153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8daa0640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1c880c86f0 0x7f1c880704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:16.420 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8eaa2640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1c881536f0 0x7f1c88153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:58558/0 (socket says 10.20.193.16:58558) 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8eaa2640 1 -- 10.20.193.16:0/3932946921 learned_addr learned my addr 10.20.193.16:0/3932946921 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8eaa2640 10 monclient(hunting): get_auth_request con 0x7f1c881536f0 auth_method 0 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8eaa2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8eaa2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8eaa2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8daa0640 10 monclient(hunting): get_auth_request con 0x7f1c880c86f0 auth_method 0 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8daa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8daa0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8daa0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8eaa2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:16.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8eaa2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.409+0000 7f1c8eaa2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8daa0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8daa0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8daa0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8eaa2640 10 monclient(hunting): handle_auth_done global_id 4968 payload 274 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8eaa2640 1 -- 10.20.193.16:0/3932946921 >> v2:10.20.193.128:3300/0 conn(0x7f1c880c86f0 msgr2=0x7f1c880704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8eaa2640 1 --2- 10.20.193.16:0/3932946921 >> v2:10.20.193.128:3300/0 conn(0x7f1c880c86f0 0x7f1c880704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8eaa2640 1 -- 10.20.193.16:0/3932946921 >> v1:10.20.193.148:6789/0 conn(0x7f1c880700f0 legacy=0x7f1c880c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8eaa2640 10 monclient: _finish_hunting 0 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8eaa2640 1 monclient: found mon.noname-a 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8eaa2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8eaa2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:16.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.410+0000 7f1c8eaa2640 1 -- 10.20.193.16:0/3932946921 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c880c50c0 con 0x7f1c881536f0 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.411+0000 7f1c8eaa2640 1 --2- 10.20.193.16:0/3932946921 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 0x7f1c88153b20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c880c5b80 tx=0x7f1c8402f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dc680528f30094 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 1 -- 10.20.193.16:0/3932946921 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8402fcf0 con 0x7f1c881536f0 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 10 monclient: dump: 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:16.423 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 10 monclient: _finish_auth 0 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 10 monclient: _check_auth_tickets 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 1 -- 10.20.193.16:0/3932946921 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c840373f0 con 0x7f1c881536f0 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 5 monclient: authenticate success, global_id 4968 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 1 -- 10.20.193.16:0/3932946921 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c840377c0 con 0x7f1c881536f0 2026-02-17T06:01:16.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 4 set_mon_vals no callback set 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c8d29f640 10 monclient: dump: 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:16.425 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 10 monclient: shutdown 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 1 -- 10.20.193.16:0/3932946921 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 msgr2=0x7f1c88153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 1 --2- 10.20.193.16:0/3932946921 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 0x7f1c88153b20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c880c5b80 tx=0x7f1c8402f230 comp rx=0 tx=0).stop 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 1 -- 10.20.193.16:0/3932946921 shutdown_connections 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 1 --2- 10.20.193.16:0/3932946921 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 0x7f1c88153b20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 1 --2- 10.20.193.16:0/3932946921 >> v2:10.20.193.128:3300/0 conn(0x7f1c880c86f0 0x7f1c880704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 1 -- 10.20.193.16:0/3932946921 >> 10.20.193.16:0/3932946921 conn(0x7f1c880cd770 msgr2=0x7f1c880cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:16.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 1 -- 10.20.193.16:0/3932946921 shutdown_connections 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.413+0000 7f1c9052c640 1 -- 10.20.193.16:0/3932946921 wait complete. 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.414+0000 7f1c9052c640 10 monclient: build_initial_monmap 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.414+0000 7f1c9052c640 10 monclient: monmap: 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:16.427 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.414+0000 7f1c9052c640 1 librados: starting msgr at 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.414+0000 7f1c9052c640 1 librados: starting objecter 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.414+0000 7f1c9052c640 1 Processor -- start 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.414+0000 7f1c9052c640 1 -- start start 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.414+0000 7f1c9052c640 1 librados: setting wanted keys 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.414+0000 7f1c9052c640 1 librados: calling monclient init 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.414+0000 7f1c9052c640 10 monclient: init 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1c880700f0 0x7f1c88145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 10 monclient(hunting): picked mon.noname-b con 0x7f1c880700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:16.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1c880c86f0 0x7f1c881f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 10 monclient(hunting): picked mon.noname-c con 0x7f1c880c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 0x7f1c88140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 10 monclient(hunting): picked mon.noname-a con 0x7f1c881536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c88154060 con 0x7f1c881536f0 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1c880c5680 con 0x7f1c880700f0 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1c88074c00 con 0x7f1c880c86f0 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c9052c640 10 monclient(hunting): authenticate will time out at 1328.984256s 2026-02-17T06:01:16.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8eaa2640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 0x7f1c88140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8e2a1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1c880700f0 0x7f1c88145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8eaa2640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 0x7f1c88140d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:58564/0 (socket says 10.20.193.16:58564) 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8eaa2640 1 -- 10.20.193.16:0/2328063534 learned_addr learned my addr 10.20.193.16:0/2328063534 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8eaa2640 10 monclient(hunting): get_auth_request con 0x7f1c881536f0 auth_method 0 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8eaa2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8eaa2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8eaa2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8daa0640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1c880c86f0 0x7f1c881f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8e2a1640 10 monclient(hunting): get_auth_request con 0x7f1c880700f0 auth_method 0 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8e2a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8e2a1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:16.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8e2a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8eaa2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8eaa2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8eaa2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8daa0640 10 monclient(hunting): get_auth_request con 0x7f1c880c86f0 auth_method 0 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8daa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8daa0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.415+0000 7f1c8daa0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.416+0000 7f1c8e2a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.416+0000 7f1c8e2a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.416+0000 7f1c8e2a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.416+0000 7f1c8daa0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:16.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.416+0000 7f1c8daa0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.416+0000 7f1c8daa0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.417+0000 7f1c8eaa2640 10 monclient(hunting): handle_auth_done global_id 4971 payload 824 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.417+0000 7f1c8eaa2640 1 -- 10.20.193.16:0/2328063534 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1c880c86f0 msgr2=0x7f1c881f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.417+0000 7f1c8eaa2640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1c880c86f0 0x7f1c881f73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.417+0000 7f1c8eaa2640 1 -- 10.20.193.16:0/2328063534 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1c880700f0 msgr2=0x7f1c88145890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.417+0000 7f1c8eaa2640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1c880700f0 0x7f1c88145890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.417+0000 7f1c8eaa2640 10 monclient: _finish_hunting 0 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.417+0000 7f1c8eaa2640 1 monclient: found mon.noname-a 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.417+0000 7f1c8eaa2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.417+0000 7f1c8eaa2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.417+0000 7f1c8eaa2640 1 -- 10.20.193.16:0/2328063534 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c88141810 con 0x7f1c881536f0 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.418+0000 7f1c8eaa2640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 0x7f1c88140d20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009f50 tx=0x7f1c84004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:16.432 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.418+0000 7f1c6effd640 10 client.?.objecter ms_handle_connect 0x7f1c881536f0 2026-02-17T06:01:16.433 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.418+0000 7f1c6effd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 1 -- 10.20.193.16:0/2328063534 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84046020 con 0x7f1c881536f0 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 10 monclient: dump: 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:16.436 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 10 monclient: _finish_auth 0 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 1 -- 10.20.193.16:0/2328063534 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c84041450 con 0x7f1c881536f0 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c9052c640 5 monclient: authenticate success, global_id 4971 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c9052c640 10 monclient: _renew_subs 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c9052c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c9052c640 1 -- 10.20.193.16:0/2328063534 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c88141af0 con 0x7f1c881536f0 2026-02-17T06:01:16.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 1 -- 10.20.193.16:0/2328063534 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84009510 con 0x7f1c881536f0 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c8ca9e640 4 set_mon_vals no callback set 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c6effd640 10 monclient: dump: 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:16.438 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c9052c640 10 client.4971.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c9052c640 10 monclient: _renew_subs 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c9052c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c9052c640 1 -- 10.20.193.16:0/2328063534 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c881420b0 con 0x7f1c881536f0 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.419+0000 7f1c9052c640 1 librados: init done 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.420+0000 7f1c9052c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.420+0000 7f1c9052c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.420+0000 7f1c9052c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:16.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.420+0000 7f1c9052c640 1 -- 10.20.193.16:0/2328063534 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c880698c0 con 0x7f1c881536f0 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.431+0000 7f1c6effd640 1 -- 10.20.193.16:0/2328063534 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1c84041840 con 0x7f1c881536f0 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.432+0000 7f1c6effd640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1c580791c0 0x7f1c5807b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.432+0000 7f1c8e2a1640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1c580791c0 0x7f1c5807b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.432+0000 7f1c8e2a1640 10 monclient: get_auth_request con 0x7f1c580791c0 auth_method 0 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.432+0000 7f1c8e2a1640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1c580791c0 0x7f1c5807b680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1c780051f0 tx=0x7f1c78007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.432+0000 7f1c6effd640 10 client.4971.objecter ms_handle_connect 0x7f1c580791c0 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.434+0000 7f1c6effd640 1 -- 10.20.193.16:0/2328063534 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1c840839c0 con 0x7f1c881536f0 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.434+0000 7f1c6effd640 10 client.4971.objecter ms_dispatch2 0x7f1c88000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.434+0000 7f1c6effd640 3 client.4971.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.434+0000 7f1c6effd640 3 client.4971.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:16.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.434+0000 7f1c6effd640 20 client.4971.objecter dump_active .. 0 homeless 2026-02-17T06:01:16.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.536+0000 7f1c6effd640 1 -- 10.20.193.16:0/2328063534 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1c880698c0 con 0x7f1c881536f0 2026-02-17T06:01:16.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.536+0000 7f1c6effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:16.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.536+0000 7f1c6effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:16.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.637+0000 7f1c9052c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:16.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.637+0000 7f1c9052c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:16.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.637+0000 7f1c9052c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:16.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.637+0000 7f1c9052c640 1 -- 10.20.193.16:0/2328063534 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1c880704d0 con 0x7f1c881536f0 2026-02-17T06:01:16.641 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.641+0000 7f1c6effd640 1 -- 10.20.193.16:0/2328063534 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1c84050030 con 0x7f1c881536f0 2026-02-17T06:01:16.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.641+0000 7f1c6effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:16.642 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.641+0000 7f1c6effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:16.642 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:16.642 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:16.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 10 librados: watch_flush enter 2026-02-17T06:01:16.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 10 librados: watch_flush exit 2026-02-17T06:01:16.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 20 client.4971.objecter shutdown clearing up homeless session... 2026-02-17T06:01:16.644 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 10 client.4971.objecter successfully canceled tick 2026-02-17T06:01:16.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 -- 10.20.193.16:0/2328063534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1c580791c0 msgr2=0x7f1c5807b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:16.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1c580791c0 0x7f1c5807b680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1c780051f0 tx=0x7f1c78007f30 comp rx=0 tx=0).stop 2026-02-17T06:01:16.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 10 monclient: shutdown 2026-02-17T06:01:16.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:16.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 -- 10.20.193.16:0/2328063534 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 msgr2=0x7f1c88140d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:16.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 0x7f1c88140d20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009f50 tx=0x7f1c84004b40 comp rx=0 tx=0).stop 2026-02-17T06:01:16.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 -- 10.20.193.16:0/2328063534 shutdown_connections 2026-02-17T06:01:16.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1c580791c0 0x7f1c5807b680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:16.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1c881536f0 0x7f1c88140d20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:16.645 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1c880c86f0 0x7f1c881f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:16.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 --2- 10.20.193.16:0/2328063534 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1c880700f0 0x7f1c88145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:16.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 -- 10.20.193.16:0/2328063534 >> 10.20.193.16:0/2328063534 conn(0x7f1c880cd770 msgr2=0x7f1c88151180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:16.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 -- 10.20.193.16:0/2328063534 shutdown_connections 2026-02-17T06:01:16.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 -- 10.20.193.16:0/2328063534 wait complete. 2026-02-17T06:01:16.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:16.642+0000 7f1c9052c640 1 librados: shutdown 2026-02-17T06:01:19.654 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:19.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.720+0000 7fac545fb640 10 monclient: get_monmap_and_config 2026-02-17T06:01:19.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.720+0000 7fac545fb640 10 monclient: build_initial_monmap 2026-02-17T06:01:19.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.720+0000 7fac545fb640 10 monclient: monmap: 2026-02-17T06:01:19.721 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:19.721880+0000 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:19.721880+0000 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:19.722 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 1 Processor -- start 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 1 -- start start 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 10 monclient: init 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 10 monclient: _add_conns ranks=[4,2,5,3,0,1] 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fac4c0700f0 0x7fac4c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 10 monclient(hunting): picked mon.noname-b con 0x7fac4c0700f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fac4c0c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fac4c1536f0 0x7fac4c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 10 monclient(hunting): picked mon.noname-c con 0x7fac4c1536f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac4c0c4820 con 0x7fac4c0c7fe0 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fac4c0c4b00 con 0x7fac4c0700f0 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fac4c0c4de0 con 0x7fac4c1536f0 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.721+0000 7fac545fb640 10 monclient(hunting): authenticate will time out at 1332.290930s 2026-02-17T06:01:19.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52370640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fac4c0700f0 0x7fac4c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52b71640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fac4c1536f0 0x7fac4c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52370640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fac4c0700f0 0x7fac4c0704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:57880/0 (socket says 10.20.193.16:57880) 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52370640 1 -- 10.20.193.16:0/2822012797 learned_addr learned my addr 10.20.193.16:0/2822012797 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52370640 10 monclient(hunting): get_auth_request con 0x7fac4c0700f0 auth_method 0 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52370640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52370640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52370640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52b71640 10 monclient(hunting): get_auth_request con 0x7fac4c1536f0 auth_method 0 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52b71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52b71640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52b71640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52370640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52370640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.722+0000 7fac52370640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52b71640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52b71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52b71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52370640 10 monclient(hunting): handle_auth_done global_id 5038 payload 274 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52370640 1 -- 10.20.193.16:0/2822012797 >> v2:10.20.193.148:3300/0 conn(0x7fac4c1536f0 msgr2=0x7fac4c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52370640 1 --2- 10.20.193.16:0/2822012797 >> v2:10.20.193.148:3300/0 conn(0x7fac4c1536f0 0x7fac4c153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52370640 1 -- 10.20.193.16:0/2822012797 >> v1:10.20.193.148:6789/0 conn(0x7fac4c0c7fe0 legacy=0x7fac4c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52370640 10 monclient: _finish_hunting 0 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52370640 1 monclient: found mon.noname-b 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52370640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52370640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.723+0000 7fac52370640 1 -- 10.20.193.16:0/2822012797 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c0c50c0 con 0x7fac4c0700f0 2026-02-17T06:01:19.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.724+0000 7fac52370640 1 --2- 10.20.193.16:0/2822012797 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 0x7fac4c0704d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0c9540 tx=0x7fac3c02ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a7dc9a0069ecdf server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 1 -- 10.20.193.16:0/2822012797 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c03c070 con 0x7fac4c0700f0 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 10 monclient: dump: 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:19.727 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 10 monclient: _finish_auth 0 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 10 monclient: _check_auth_tickets 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 1 -- 10.20.193.16:0/2822012797 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3c02fa90 con 0x7fac4c0700f0 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac545fb640 5 monclient: authenticate success, global_id 5038 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 1 -- 10.20.193.16:0/2822012797 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c02fe60 con 0x7fac4c0700f0 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:19.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac5136e640 10 monclient: dump: 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:19.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac545fb640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:19.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac545fb640 4 set_mon_vals no callback set 2026-02-17T06:01:19.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac545fb640 10 monclient: shutdown 2026-02-17T06:01:19.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac545fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:19.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.726+0000 7fac545fb640 1 -- 10.20.193.16:0/2822012797 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 msgr2=0x7fac4c0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:19.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 --2- 10.20.193.16:0/2822012797 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 0x7fac4c0704d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0c9540 tx=0x7fac3c02ee60 comp rx=0 tx=0).stop 2026-02-17T06:01:19.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 -- 10.20.193.16:0/2822012797 shutdown_connections 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 --2- 10.20.193.16:0/2822012797 >> v2:10.20.193.148:3300/0 conn(0x7fac4c1536f0 0x7fac4c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 --2- 10.20.193.16:0/2822012797 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 0x7fac4c0704d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 -- 10.20.193.16:0/2822012797 >> 10.20.193.16:0/2822012797 conn(0x7fac4c0cd770 msgr2=0x7fac4c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 -- 10.20.193.16:0/2822012797 shutdown_connections 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 -- 10.20.193.16:0/2822012797 wait complete. 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 10 monclient: build_initial_monmap 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 10 monclient: monmap: 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:19.736 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 librados: starting msgr at 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 librados: starting objecter 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 Processor -- start 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 -- start start 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 librados: setting wanted keys 2026-02-17T06:01:19.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 1 librados: calling monclient init 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.727+0000 7fac545fb640 10 monclient: init 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 0x7fac4c139070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 10 monclient(hunting): picked mon.noname-b con 0x7fac4c0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c0c7fe0 0x7fac4c072d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 10 monclient(hunting): picked mon.noname-c con 0x7fac4c0c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c1536f0 0x7fac4c073260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 10 monclient(hunting): picked mon.noname-a con 0x7fac4c1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c154060 con 0x7fac4c1536f0 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:19.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c0c5680 con 0x7fac4c0700f0 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c074c00 con 0x7fac4c0c7fe0 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac545fb640 10 monclient(hunting): authenticate will time out at 1332.297356s 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52370640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 0x7fac4c139070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52370640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 0x7fac4c139070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:57892/0 (socket says 10.20.193.16:57892) 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52370640 1 -- 10.20.193.16:0/3063288024 learned_addr learned my addr 10.20.193.16:0/3063288024 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52370640 10 monclient(hunting): get_auth_request con 0x7fac4c0700f0 auth_method 0 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52370640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52370640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:19.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52370640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac51b6f640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c0c7fe0 0x7fac4c072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52b71640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c1536f0 0x7fac4c073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52370640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52370640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52370640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52b71640 10 monclient(hunting): get_auth_request con 0x7fac4c1536f0 auth_method 0 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52b71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52b71640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.728+0000 7fac52b71640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.729+0000 7fac51b6f640 10 monclient(hunting): get_auth_request con 0x7fac4c0c7fe0 auth_method 0 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.729+0000 7fac51b6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.729+0000 7fac51b6f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:19.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.729+0000 7fac51b6f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.729+0000 7fac52b71640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.729+0000 7fac52b71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.729+0000 7fac52b71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.729+0000 7fac51b6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.729+0000 7fac51b6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.729+0000 7fac51b6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.730+0000 7fac52370640 10 monclient(hunting): handle_auth_done global_id 5041 payload 824 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.730+0000 7fac52370640 1 -- 10.20.193.16:0/3063288024 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c0c7fe0 msgr2=0x7fac4c072d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.730+0000 7fac52370640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c0c7fe0 0x7fac4c072d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.730+0000 7fac52370640 1 -- 10.20.193.16:0/3063288024 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c1536f0 msgr2=0x7fac4c073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:19.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.730+0000 7fac52370640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c1536f0 0x7fac4c073260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.730+0000 7fac52370640 10 monclient: _finish_hunting 0 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.730+0000 7fac52370640 1 monclient: found mon.noname-b 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.730+0000 7fac52370640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.730+0000 7fac52370640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.730+0000 7fac52370640 1 -- 10.20.193.16:0/3063288024 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c139cc0 con 0x7fac4c0700f0 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.731+0000 7fac52370640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 0x7fac4c139070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009a70 tx=0x7fac3c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.731+0000 7fac32ffd640 10 client.?.objecter ms_handle_connect 0x7fac4c0700f0 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.731+0000 7fac32ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 1 -- 10.20.193.16:0/3063288024 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c03c040 con 0x7fac4c0700f0 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 10 monclient: dump: 2026-02-17T06:01:19.742 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:19.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac545fb640 5 monclient: authenticate success, global_id 5041 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 1 -- 10.20.193.16:0/3063288024 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3c042410 con 0x7fac4c0700f0 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 1 -- 10.20.193.16:0/3063288024 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c042800 con 0x7fac4c0700f0 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac32ffd640 10 monclient: dump: 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:19.744 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac50b6d640 4 set_mon_vals no callback set 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac545fb640 10 monclient: _renew_subs 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac545fb640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.732+0000 7fac545fb640 1 -- 10.20.193.16:0/3063288024 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4c0cb180 con 0x7fac4c0700f0 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.733+0000 7fac545fb640 10 client.5041.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.733+0000 7fac545fb640 10 monclient: _renew_subs 2026-02-17T06:01:19.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.733+0000 7fac545fb640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:19.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.733+0000 7fac545fb640 1 -- 10.20.193.16:0/3063288024 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4c0cb720 con 0x7fac4c0700f0 2026-02-17T06:01:19.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.733+0000 7fac545fb640 1 librados: init done 2026-02-17T06:01:19.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.733+0000 7fac545fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:19.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.733+0000 7fac545fb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:19.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.733+0000 7fac545fb640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:19.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.733+0000 7fac545fb640 1 -- 10.20.193.16:0/3063288024 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4c0704d0 con 0x7fac4c0700f0 2026-02-17T06:01:19.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.744+0000 7fac32ffd640 1 -- 10.20.193.16:0/3063288024 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fac3c0429a0 con 0x7fac4c0700f0 2026-02-17T06:01:19.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.745+0000 7fac32ffd640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac1c0791c0 0x7fac1c07b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:19.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.745+0000 7fac51b6f640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac1c0791c0 0x7fac1c07b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:19.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.745+0000 7fac51b6f640 10 monclient: get_auth_request con 0x7fac1c0791c0 auth_method 0 2026-02-17T06:01:19.748 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.745+0000 7fac51b6f640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac1c0791c0 0x7fac1c07b680 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac4c074800 tx=0x7fac40009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:19.748 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.745+0000 7fac32ffd640 10 client.5041.objecter ms_handle_connect 0x7fac1c0791c0 2026-02-17T06:01:19.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.843+0000 7fac32ffd640 1 -- 10.20.193.16:0/3063288024 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fac3c0839c0 con 0x7fac4c0700f0 2026-02-17T06:01:19.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.843+0000 7fac32ffd640 10 client.5041.objecter ms_dispatch2 0x7fac4c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:19.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.843+0000 7fac32ffd640 3 client.5041.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:19.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.843+0000 7fac32ffd640 3 client.5041.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:19.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.843+0000 7fac32ffd640 20 client.5041.objecter dump_active .. 0 homeless 2026-02-17T06:01:19.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.849+0000 7fac32ffd640 1 -- 10.20.193.16:0/3063288024 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fac4c139cc0 con 0x7fac4c0700f0 2026-02-17T06:01:19.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.849+0000 7fac32ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:19.850 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.849+0000 7fac32ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:19.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.950+0000 7fac545fb640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:19.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.950+0000 7fac545fb640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:19.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.950+0000 7fac545fb640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:19.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.950+0000 7fac545fb640 1 -- 10.20.193.16:0/3063288024 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fac4c13db20 con 0x7fac4c0700f0 2026-02-17T06:01:19.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.953+0000 7fac32ffd640 1 -- 10.20.193.16:0/3063288024 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fac3c087980 con 0x7fac4c0700f0 2026-02-17T06:01:19.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.953+0000 7fac32ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:19.953 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.953+0000 7fac32ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:19.953 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:19.953 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 10 librados: watch_flush enter 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 10 librados: watch_flush exit 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 20 client.5041.objecter shutdown clearing up homeless session... 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 10 client.5041.objecter successfully canceled tick 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 -- 10.20.193.16:0/3063288024 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac1c0791c0 msgr2=0x7fac1c07b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac1c0791c0 0x7fac1c07b680 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac4c074800 tx=0x7fac40009110 comp rx=0 tx=0).stop 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 10 monclient: shutdown 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 -- 10.20.193.16:0/3063288024 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 msgr2=0x7fac4c139070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 0x7fac4c139070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009a70 tx=0x7fac3c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:19.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 -- 10.20.193.16:0/3063288024 shutdown_connections 2026-02-17T06:01:19.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fac1c0791c0 0x7fac1c07b680 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:19.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fac4c1536f0 0x7fac4c073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:19.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fac4c0c7fe0 0x7fac4c072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:19.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 --2- 10.20.193.16:0/3063288024 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fac4c0700f0 0x7fac4c139070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:19.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 -- 10.20.193.16:0/3063288024 >> 10.20.193.16:0/3063288024 conn(0x7fac4c0cd770 msgr2=0x7fac4c14dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:19.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 -- 10.20.193.16:0/3063288024 shutdown_connections 2026-02-17T06:01:19.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 -- 10.20.193.16:0/3063288024 wait complete. 2026-02-17T06:01:19.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:19.954+0000 7fac545fb640 1 librados: shutdown 2026-02-17T06:01:22.964 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:23.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.032+0000 7f286536a640 10 monclient: get_monmap_and_config 2026-02-17T06:01:23.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.032+0000 7f286536a640 10 monclient: build_initial_monmap 2026-02-17T06:01:23.032 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.032+0000 7f286536a640 10 monclient: monmap: 2026-02-17T06:01:23.032 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:23.032 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:23.032 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:23.033217+0000 2026-02-17T06:01:23.032 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:23.033217+0000 2026-02-17T06:01:23.032 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:23.033 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:23.033 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:23.033 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:23.033 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:23.033 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:23.033 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:23.033 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:23.033 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.032+0000 7f286536a640 1 Processor -- start 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.032+0000 7f286536a640 1 -- start start 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.032+0000 7f286536a640 10 monclient: init 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 10 monclient: _add_conns ranks=[3,1,5,4,0,2] 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f28600700f0 0x7f28600704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 10 monclient(hunting): picked mon.noname-a con 0x7f28600700f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f28600c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f28601536f0 0x7f2860153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 10 monclient(hunting): picked mon.noname-c con 0x7f28601536f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28600c4820 con 0x7f28600c7fe0 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f28600c4b00 con 0x7f28600700f0 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f28600c4de0 con 0x7f28601536f0 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f286536a640 10 monclient(hunting): authenticate will time out at 1335.602162s 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285effd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f28600700f0 0x7f28600704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285e7fc640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f28600c7fe0 0x7f28600c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:59158/0 (socket says 10.20.193.16:59158) 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285e7fc640 1 -- 10.20.193.16:0/2184187896 learned_addr learned my addr 10.20.193.16:0/2184187896 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:23.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285f7fe640 1 --2- 10.20.193.16:0/2184187896 >> v2:10.20.193.148:3300/0 conn(0x7f28601536f0 0x7f2860153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285effd640 10 monclient(hunting): get_auth_request con 0x7f28600700f0 auth_method 0 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285f7fe640 10 monclient(hunting): get_auth_request con 0x7f28601536f0 auth_method 0 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.033+0000 7f285f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.034+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.034+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.034+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:23.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.034+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.034+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.034+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.034+0000 7f285effd640 10 monclient(hunting): handle_auth_done global_id 4977 payload 274 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.034+0000 7f285effd640 1 -- 10.20.193.16:0/2184187896 >> v2:10.20.193.148:3300/0 conn(0x7f28601536f0 msgr2=0x7f2860153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.034+0000 7f285effd640 1 --2- 10.20.193.16:0/2184187896 >> v2:10.20.193.148:3300/0 conn(0x7f28601536f0 0x7f2860153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.034+0000 7f285effd640 1 -- 10.20.193.16:0/2184187896 >> v1:10.20.193.128:6789/0 conn(0x7f28600c7fe0 legacy=0x7f28600c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.035+0000 7f285effd640 10 monclient: _finish_hunting 0 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.035+0000 7f285effd640 1 monclient: found mon.noname-a 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.035+0000 7f285effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.035+0000 7f285effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.035+0000 7f285effd640 1 -- 10.20.193.16:0/2184187896 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28600c50c0 con 0x7f28600700f0 2026-02-17T06:01:23.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.035+0000 7f285effd640 1 --2- 10.20.193.16:0/2184187896 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f28600700f0 0x7f28600704d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f28600c9540 tx=0x7f285402ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9b1c42584dec6c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:23.038 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 1 -- 10.20.193.16:0/2184187896 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285403c070 con 0x7f28600700f0 2026-02-17T06:01:23.038 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:23.038 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:23.038 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 10 monclient: dump: 2026-02-17T06:01:23.038 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:23.038 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 1 -- 10.20.193.16:0/2184187896 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285402fa60 con 0x7f28600700f0 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f286536a640 5 monclient: authenticate success, global_id 4977 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f286536a640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 1 -- 10.20.193.16:0/2184187896 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285402fe30 con 0x7f28600700f0 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f286536a640 4 set_mon_vals no callback set 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.037+0000 7f285dffb640 10 monclient: dump: 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:23.039 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 10 monclient: shutdown 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 -- 10.20.193.16:0/2184187896 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f28600700f0 msgr2=0x7f28600704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 --2- 10.20.193.16:0/2184187896 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f28600700f0 0x7f28600704d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f28600c9540 tx=0x7f285402ed50 comp rx=0 tx=0).stop 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 -- 10.20.193.16:0/2184187896 shutdown_connections 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 --2- 10.20.193.16:0/2184187896 >> v2:10.20.193.148:3300/0 conn(0x7f28601536f0 0x7f2860153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 --2- 10.20.193.16:0/2184187896 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f28600700f0 0x7f28600704d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 -- 10.20.193.16:0/2184187896 >> 10.20.193.16:0/2184187896 conn(0x7f28600cd770 msgr2=0x7f28600cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 -- 10.20.193.16:0/2184187896 shutdown_connections 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 -- 10.20.193.16:0/2184187896 wait complete. 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 10 monclient: build_initial_monmap 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 10 monclient: monmap: 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:23.040 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 librados: starting msgr at 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 librados: starting objecter 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 Processor -- start 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 -- start start 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 librados: setting wanted keys 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 1 librados: calling monclient init 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.038+0000 7f286536a640 10 monclient: init 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f28600700f0 0x7f2860151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 10 monclient(hunting): picked mon.noname-c con 0x7f28600700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f28600c7fe0 0x7f2860144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 10 monclient(hunting): picked mon.noname-b con 0x7f28600c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f28601536f0 0x7f28601f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 10 monclient(hunting): picked mon.noname-a con 0x7f28601536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2860154060 con 0x7f28601536f0 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f28600c5680 con 0x7f28600c7fe0 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2860074c00 con 0x7f28600700f0 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:23.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f286536a640 10 monclient(hunting): authenticate will time out at 1335.608229s 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285e7fc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f28600c7fe0 0x7f2860144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285effd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f28600700f0 0x7f2860151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285f7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f28601536f0 0x7f28601f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285e7fc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f28600c7fe0 0x7f2860144af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:34624/0 (socket says 10.20.193.16:34624) 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285e7fc640 1 -- 10.20.193.16:0/2819854801 learned_addr learned my addr 10.20.193.16:0/2819854801 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285e7fc640 10 monclient(hunting): get_auth_request con 0x7f28600c7fe0 auth_method 0 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285f7fe640 10 monclient(hunting): get_auth_request con 0x7f28601536f0 auth_method 0 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285effd640 10 monclient(hunting): get_auth_request con 0x7f28600700f0 auth_method 0 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.039+0000 7f285effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.040+0000 7f285e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.040+0000 7f285e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.040+0000 7f285e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.040+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.040+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.040+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.040+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.040+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.040+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.041+0000 7f285e7fc640 10 monclient(hunting): handle_auth_done global_id 5047 payload 824 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.041+0000 7f285e7fc640 1 -- 10.20.193.16:0/2819854801 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f28600700f0 msgr2=0x7f2860151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.041+0000 7f285e7fc640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f28600700f0 0x7f2860151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.041+0000 7f285e7fc640 1 -- 10.20.193.16:0/2819854801 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f28601536f0 msgr2=0x7f28601f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:23.043 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.041+0000 7f285e7fc640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f28601536f0 0x7f28601f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:23.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.041+0000 7f285e7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:01:23.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.041+0000 7f285e7fc640 1 monclient: found mon.noname-b 2026-02-17T06:01:23.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.041+0000 7f285e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:23.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.041+0000 7f285e7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:23.044 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.041+0000 7f285e7fc640 1 -- 10.20.193.16:0/2819854801 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28601f82b0 con 0x7f28600c7fe0 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.042+0000 7f285e7fc640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f28600c7fe0 0x7f2860144af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f284800c500 tx=0x7f284800c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.042+0000 7f283f7fe640 10 client.?.objecter ms_handle_connect 0x7f28600c7fe0 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.042+0000 7f283f7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 1 -- 10.20.193.16:0/2819854801 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848017070 con 0x7f28600c7fe0 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 10 monclient: dump: 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 5 monclient: authenticate success, global_id 5047 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 10 monclient: _renew_subs 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 1 -- 10.20.193.16:0/2819854801 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601f4db0 con 0x7f28600c7fe0 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 10 client.5047.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:23.045 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 10 monclient: _renew_subs 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 1 -- 10.20.193.16:0/2819854801 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601f3ff0 con 0x7f28600c7fe0 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 1 librados: init done 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f286536a640 1 -- 10.20.193.16:0/2819854801 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286014e510 con 0x7f28600c7fe0 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 1 -- 10.20.193.16:0/2819854801 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2848005de0 con 0x7f28600c7fe0 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 1 -- 10.20.193.16:0/2819854801 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848021060 con 0x7f28600c7fe0 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f283f7fe640 10 monclient: dump: 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:23.046 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.044+0000 7f285d7fa640 4 set_mon_vals no callback set 2026-02-17T06:01:23.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.151+0000 7f283f7fe640 1 -- 10.20.193.16:0/2819854801 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f2848007070 con 0x7f28600c7fe0 2026-02-17T06:01:23.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.151+0000 7f283f7fe640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f282c079140 0x7f282c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:23.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.151+0000 7f285effd640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f282c079140 0x7f282c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:23.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.151+0000 7f285effd640 10 monclient: get_auth_request con 0x7f282c079140 auth_method 0 2026-02-17T06:01:23.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.152+0000 7f283f7fe640 10 client.5047.objecter ms_handle_connect 0x7f282c079140 2026-02-17T06:01:23.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.152+0000 7f285effd640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f282c079140 0x7f282c07b600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2854039000 tx=0x7f2854033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:23.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.153+0000 7f283f7fe640 1 -- 10.20.193.16:0/2819854801 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f2848097070 con 0x7f28600c7fe0 2026-02-17T06:01:23.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.153+0000 7f283f7fe640 10 client.5047.objecter ms_dispatch2 0x7f2860000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:23.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.153+0000 7f283f7fe640 3 client.5047.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:23.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.153+0000 7f283f7fe640 3 client.5047.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:23.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.153+0000 7f283f7fe640 20 client.5047.objecter dump_active .. 0 homeless 2026-02-17T06:01:23.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.159+0000 7f283f7fe640 1 -- 10.20.193.16:0/2819854801 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f28480616a0 con 0x7f28600c7fe0 2026-02-17T06:01:23.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.159+0000 7f283f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:23.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.159+0000 7f283f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:23.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.258+0000 7f286536a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:23.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.258+0000 7f286536a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:23.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.258+0000 7f286536a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:23.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.258+0000 7f286536a640 1 -- 10.20.193.16:0/2819854801 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f28601573f0 con 0x7f28600c7fe0 2026-02-17T06:01:23.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.261+0000 7f283f7fe640 1 -- 10.20.193.16:0/2819854801 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f28480659a0 con 0x7f28600c7fe0 2026-02-17T06:01:23.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.261+0000 7f283f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:23.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.261+0000 7f283f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:23.262 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:23.262 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:23.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 10 librados: watch_flush enter 2026-02-17T06:01:23.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 10 librados: watch_flush exit 2026-02-17T06:01:23.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 20 client.5047.objecter shutdown clearing up homeless session... 2026-02-17T06:01:23.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 10 client.5047.objecter successfully canceled tick 2026-02-17T06:01:23.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 -- 10.20.193.16:0/2819854801 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f282c079140 msgr2=0x7f282c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:23.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f282c079140 0x7f282c07b600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2854039000 tx=0x7f2854033000 comp rx=0 tx=0).stop 2026-02-17T06:01:23.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 10 monclient: shutdown 2026-02-17T06:01:23.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:23.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 -- 10.20.193.16:0/2819854801 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f28600c7fe0 msgr2=0x7f2860144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:23.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f28600c7fe0 0x7f2860144af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f284800c500 tx=0x7f284800c9d0 comp rx=0 tx=0).stop 2026-02-17T06:01:23.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 -- 10.20.193.16:0/2819854801 shutdown_connections 2026-02-17T06:01:23.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f282c079140 0x7f282c07b600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:23.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f28600c7fe0 0x7f2860144af0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:23.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f28601536f0 0x7f28601f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:23.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 --2- 10.20.193.16:0/2819854801 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f28600700f0 0x7f2860151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:23.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 -- 10.20.193.16:0/2819854801 >> 10.20.193.16:0/2819854801 conn(0x7f28600cd770 msgr2=0x7f286014ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:23.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 -- 10.20.193.16:0/2819854801 shutdown_connections 2026-02-17T06:01:23.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 -- 10.20.193.16:0/2819854801 wait complete. 2026-02-17T06:01:23.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:23.262+0000 7f286536a640 1 librados: shutdown 2026-02-17T06:01:26.272 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:26.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.339+0000 7f9dbed5b640 10 monclient: get_monmap_and_config 2026-02-17T06:01:26.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.339+0000 7f9dbed5b640 10 monclient: build_initial_monmap 2026-02-17T06:01:26.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.339+0000 7f9dbed5b640 10 monclient: monmap: 2026-02-17T06:01:26.340 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:26.340 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:26.340 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:26.340714+0000 2026-02-17T06:01:26.340 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:26.340714+0000 2026-02-17T06:01:26.340 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 1 Processor -- start 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 1 -- start start 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 10 monclient: init 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:26.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 10 monclient: _add_conns ranks=[4,3,0,1,5,2] 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f9db80700f0 0x7f9db80704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 10 monclient(hunting): picked mon.noname-b con 0x7f9db80700f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f9db80c7fe0 0x7f9db80c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 10 monclient(hunting): picked mon.noname-a con 0x7f9db80c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9db81536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db80c8970 con 0x7f9db81536f0 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9db80c4820 con 0x7f9db80c7fe0 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f9db80c4b00 con 0x7f9db80700f0 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.340+0000 7f9dbed5b640 10 monclient(hunting): authenticate will time out at 1338.909669s 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9daffff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f9db80c7fe0 0x7f9db80c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9dbcad0640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f9db80700f0 0x7f9db80704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9daffff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f9db80c7fe0 0x7f9db80c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51484/0 (socket says 10.20.193.16:51484) 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9daffff640 1 -- 10.20.193.16:0/3323263564 learned_addr learned my addr 10.20.193.16:0/3323263564 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9dbcad0640 10 monclient(hunting): get_auth_request con 0x7f9db80700f0 auth_method 0 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9dbcad0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9dbcad0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9dbcad0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9daffff640 10 monclient(hunting): get_auth_request con 0x7f9db80c7fe0 auth_method 0 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9daffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9daffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9daffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9daffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9daffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9daffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9dbcad0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9dbcad0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:26.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.341+0000 7f9dbcad0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:26.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.342+0000 7f9daffff640 10 monclient(hunting): handle_auth_done global_id 4983 payload 274 2026-02-17T06:01:26.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.342+0000 7f9daffff640 1 -- 10.20.193.16:0/3323263564 >> v2:10.20.193.128:3300/0 conn(0x7f9db80700f0 msgr2=0x7f9db80704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:26.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.342+0000 7f9daffff640 1 --2- 10.20.193.16:0/3323263564 >> v2:10.20.193.128:3300/0 conn(0x7f9db80700f0 0x7f9db80704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.342+0000 7f9daffff640 1 -- 10.20.193.16:0/3323263564 >> v1:10.20.193.16:6789/0 conn(0x7f9db81536f0 legacy=0x7f9db8153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:26.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.342+0000 7f9daffff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:26.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.342+0000 7f9daffff640 1 monclient: found mon.noname-a 2026-02-17T06:01:26.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.342+0000 7f9daffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:26.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.342+0000 7f9daffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:26.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.342+0000 7f9daffff640 1 -- 10.20.193.16:0/3323263564 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db80c4de0 con 0x7f9db80c7fe0 2026-02-17T06:01:26.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.343+0000 7f9daffff640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db80c8430 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9db8070b90 tx=0x7f9da002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fe826ffa52c293 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daffff640 1 -- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 msgr2=0x7f9db80c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daffff640 1 -- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 msgr2=0x7f9db80c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 1 -- 10.20.193.16:0/3323263564 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da003c070 con 0x7f9db80c7fe0 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daffff640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db80c8430 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9db8070b90 tx=0x7f9da002ee60 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daffff640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db80c8430 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9db8070b90 tx=0x7f9da002ee60 comp rx=0 tx=0).stop 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient: dump: 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:26.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient: ms_handle_reset current mon [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 1 -- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 msgr2=0x7f9db80c8430 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db80c8430 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9da40038a0 0x7f9da4005cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient(hunting): picked mon.b con 0x7f9da40038a0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9da4006510 0x7f9da40089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient(hunting): picked mon.c con 0x7f9da4006510 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9da4009280 0x7f9da400b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient(hunting): picked mon.a con 0x7f9da4009280 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 1 -- 10.20.193.16:0/3323263564 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9da400c080 con 0x7f9da4009280 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 1 -- 10.20.193.16:0/3323263564 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f9da400c230 con 0x7f9da40038a0 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 1 -- 10.20.193.16:0/3323263564 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9da400c430 con 0x7f9da4006510 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.345+0000 7f9daf7fe640 10 monclient(hunting): ms_handle_reset hunted mon [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:26.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9dbcad0640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9da40038a0 0x7f9da4005cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9daffff640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9da4006510 0x7f9da40089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9dbcad0640 10 monclient(hunting): get_auth_request con 0x7f9da40038a0 auth_method 0 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9dbcad0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9dbcad0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9dbcad0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9daffff640 10 monclient(hunting): get_auth_request con 0x7f9da4006510 auth_method 0 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9daffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9daffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9daffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9dbcad0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9dbcad0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.346+0000 7f9dbcad0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9daffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9daffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9daffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbd2d1640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9da4009280 0x7f9da400b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 10 monclient(hunting): handle_auth_done global_id 4983 payload 293 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 1 -- 10.20.193.16:0/3323263564 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9da4006510 msgr2=0x7f9da40089a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9da4006510 0x7f9da40089a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 1 -- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9da4009280 msgr2=0x7f9da400b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9da4009280 0x7f9da400b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 10 monclient: _finish_hunting 0 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 1 monclient: found mon.b 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbd2d1640 1 -- 10.20.193.16:0/3323263564 reap_dead start 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 1 -- 10.20.193.16:0/3323263564 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=2+}) -- 0x7f9da400c6f0 con 0x7f9da40038a0 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 10 monclient: _finish_auth 0 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 10 monclient: _check_auth_tickets 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbcad0640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:26.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbed5b640 5 monclient: authenticate success, global_id 4983 2026-02-17T06:01:26.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.347+0000 7f9dbed5b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:26.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.348+0000 7f9dbcad0640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9da40038a0 0x7f9da4005cd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009bc0 tx=0x7f9da0009340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d1983cd57d49e2e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:26.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9daf7fe640 1 -- 10.20.193.16:0/3323263564 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8017070 con 0x7f9da40038a0 2026-02-17T06:01:26.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9daf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:26.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9daf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:26.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9daf7fe640 10 monclient: dump: 2026-02-17T06:01:26.350 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:26.350 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:26.350 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:26.350 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:26.350 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:26.350 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9daf7fe640 1 -- 10.20.193.16:0/3323263564 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da8009de0 con 0x7f9da40038a0 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9daf7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 4 set_mon_vals no callback set 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 10 monclient: shutdown 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 1 -- 10.20.193.16:0/3323263564 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9da40038a0 msgr2=0x7f9da4005cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9da40038a0 0x7f9da4005cd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009bc0 tx=0x7f9da0009340 comp rx=0 tx=0).stop 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 1 -- 10.20.193.16:0/3323263564 shutdown_connections 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9da40038a0 0x7f9da4005cd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 1 --2- 10.20.193.16:0/3323263564 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db80c8430 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 1 -- 10.20.193.16:0/3323263564 >> 10.20.193.16:0/3323263564 conn(0x7f9db80cd770 msgr2=0x7f9db80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 1 -- 10.20.193.16:0/3323263564 shutdown_connections 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.350+0000 7f9dbed5b640 1 -- 10.20.193.16:0/3323263564 wait complete. 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.351+0000 7f9dbed5b640 10 monclient: build_initial_monmap 2026-02-17T06:01:26.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.351+0000 7f9dbed5b640 10 monclient: monmap: 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.351+0000 7f9dbed5b640 1 librados: starting msgr at 2026-02-17T06:01:26.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.351+0000 7f9dbed5b640 1 librados: starting objecter 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.351+0000 7f9dbed5b640 1 Processor -- start 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.351+0000 7f9dbed5b640 1 -- start start 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.351+0000 7f9dbed5b640 1 librados: setting wanted keys 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.351+0000 7f9dbed5b640 1 librados: calling monclient init 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.351+0000 7f9dbed5b640 10 monclient: init 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9da40038a0 0x7f9db8144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 10 monclient(hunting): picked mon.noname-c con 0x7f9da40038a0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db81f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 10 monclient(hunting): picked mon.noname-a con 0x7f9db80c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db813c930 0x7f9db8146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 10 monclient(hunting): picked mon.noname-b con 0x7f9db813c930 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9db80c5680 con 0x7f9db80c7fe0 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f9db80c53a0 con 0x7f9db813c930 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9db8074c00 con 0x7f9da40038a0 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbed5b640 10 monclient(hunting): authenticate will time out at 1338.921058s 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9daffff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db81f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbd2d1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db813c930 0x7f9db8146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbd2d1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db813c930 0x7f9db8146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:34656/0 (socket says 10.20.193.16:34656) 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbd2d1640 1 -- 10.20.193.16:0/1462768355 learned_addr learned my addr 10.20.193.16:0/1462768355 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbd2d1640 10 monclient(hunting): get_auth_request con 0x7f9db813c930 auth_method 0 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbd2d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbd2d1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbd2d1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9daffff640 10 monclient(hunting): get_auth_request con 0x7f9db80c7fe0 auth_method 0 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9daffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:26.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9daffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9daffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbcad0640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9da40038a0 0x7f9db8144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbd2d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbd2d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbd2d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9daffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9daffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9daffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbcad0640 10 monclient(hunting): get_auth_request con 0x7f9da40038a0 auth_method 0 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbcad0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbcad0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.352+0000 7f9dbcad0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.353+0000 7f9dbcad0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.353+0000 7f9dbcad0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.353+0000 7f9dbcad0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.354+0000 7f9dbd2d1640 10 monclient(hunting): handle_auth_done global_id 5053 payload 824 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.354+0000 7f9dbd2d1640 1 -- 10.20.193.16:0/1462768355 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9da40038a0 msgr2=0x7f9db8144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.354+0000 7f9dbd2d1640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9da40038a0 0x7f9db8144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.354+0000 7f9dbd2d1640 1 -- 10.20.193.16:0/1462768355 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 msgr2=0x7f9db81f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.354+0000 7f9dbd2d1640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db81f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.354+0000 7f9dbd2d1640 10 monclient: _finish_hunting 0 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.354+0000 7f9dbd2d1640 1 monclient: found mon.noname-b 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.354+0000 7f9dbd2d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:26.354 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.354+0000 7f9dbd2d1640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:26.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.354+0000 7f9dbd2d1640 1 -- 10.20.193.16:0/1462768355 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db8144fb0 con 0x7f9db813c930 2026-02-17T06:01:26.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.355+0000 7f9dbd2d1640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db813c930 0x7f9db8146080 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9db8070a50 tx=0x7f9db400df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:26.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.355+0000 7f9dad7fa640 10 client.?.objecter ms_handle_connect 0x7f9db813c930 2026-02-17T06:01:26.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.355+0000 7f9dad7fa640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.356+0000 7f9dad7fa640 1 -- 10.20.193.16:0/1462768355 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db400bb80 con 0x7f9db813c930 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.356+0000 7f9dad7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.356+0000 7f9dad7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.356+0000 7f9dad7fa640 10 monclient: dump: 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.356+0000 7f9dad7fa640 10 monclient: _finish_auth 0 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.356+0000 7f9dad7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.356+0000 7f9dad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 5 monclient: authenticate success, global_id 5053 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dad7fa640 1 -- 10.20.193.16:0/1462768355 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db4017070 con 0x7f9db813c930 2026-02-17T06:01:26.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 10 monclient: _renew_subs 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 1 -- 10.20.193.16:0/1462768355 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db8145290 con 0x7f9db813c930 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dad7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dad7fa640 1 -- 10.20.193.16:0/1462768355 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4012410 con 0x7f9db813c930 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dad7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dad7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9daeffd640 4 set_mon_vals no callback set 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dad7fa640 10 monclient: dump: 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 10 client.5053.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 10 monclient: _renew_subs 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 1 -- 10.20.193.16:0/1462768355 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db8145850 con 0x7f9db813c930 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 1 librados: init done 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:26.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.357+0000 7f9dbed5b640 1 -- 10.20.193.16:0/1462768355 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db80c8430 con 0x7f9db813c930 2026-02-17T06:01:26.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.369+0000 7f9dad7fa640 1 -- 10.20.193.16:0/1462768355 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f9db40125b0 con 0x7f9db813c930 2026-02-17T06:01:26.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.369+0000 7f9dad7fa640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079190 0x7f9d8807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:26.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.369+0000 7f9dbcad0640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079190 0x7f9d8807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:26.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.369+0000 7f9dbcad0640 10 monclient: get_auth_request con 0x7f9d88079190 auth_method 0 2026-02-17T06:01:26.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.370+0000 7f9dbcad0640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079190 0x7f9d8807b650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9da8002410 tx=0x7f9da8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:26.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.370+0000 7f9dad7fa640 10 client.5053.objecter ms_handle_connect 0x7f9d88079190 2026-02-17T06:01:26.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.371+0000 7f9dad7fa640 1 -- 10.20.193.16:0/1462768355 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f9db405eb40 con 0x7f9db813c930 2026-02-17T06:01:26.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.371+0000 7f9dad7fa640 10 client.5053.objecter ms_dispatch2 0x7f9db8000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:26.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.371+0000 7f9dad7fa640 3 client.5053.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:26.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.371+0000 7f9dad7fa640 3 client.5053.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:26.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.371+0000 7f9dad7fa640 20 client.5053.objecter dump_active .. 0 homeless 2026-02-17T06:01:26.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.472+0000 7f9dad7fa640 1 -- 10.20.193.16:0/1462768355 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9db4062b00 con 0x7f9db813c930 2026-02-17T06:01:26.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.472+0000 7f9dad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:26.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.472+0000 7f9dad7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:26.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.572+0000 7f9dbed5b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:26.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.572+0000 7f9dbed5b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:26.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.572+0000 7f9dbed5b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:26.573 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.572+0000 7f9dbed5b640 1 -- 10.20.193.16:0/1462768355 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9db81573f0 con 0x7f9db813c930 2026-02-17T06:01:26.574 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.574+0000 7f9dad7fa640 1 -- 10.20.193.16:0/1462768355 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f9db4066e00 con 0x7f9db813c930 2026-02-17T06:01:26.575 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.574+0000 7f9dad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:26.575 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.574+0000 7f9dad7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:26.575 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:26.575 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:26.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 10 librados: watch_flush enter 2026-02-17T06:01:26.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 10 librados: watch_flush exit 2026-02-17T06:01:26.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 20 client.5053.objecter shutdown clearing up homeless session... 2026-02-17T06:01:26.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 10 client.5053.objecter successfully canceled tick 2026-02-17T06:01:26.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 -- 10.20.193.16:0/1462768355 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079190 msgr2=0x7f9d8807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:26.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079190 0x7f9d8807b650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9da8002410 tx=0x7f9da8008030 comp rx=0 tx=0).stop 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 10 monclient: shutdown 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 -- 10.20.193.16:0/1462768355 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db813c930 msgr2=0x7f9db8146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db813c930 0x7f9db8146080 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9db8070a50 tx=0x7f9db400df80 comp rx=0 tx=0).stop 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 -- 10.20.193.16:0/1462768355 shutdown_connections 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9d88079190 0x7f9d8807b650 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9db813c930 0x7f9db8146080 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9db80c7fe0 0x7f9db81f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 --2- 10.20.193.16:0/1462768355 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9da40038a0 0x7f9db8144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 -- 10.20.193.16:0/1462768355 >> 10.20.193.16:0/1462768355 conn(0x7f9db80cd770 msgr2=0x7f9db814fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 -- 10.20.193.16:0/1462768355 shutdown_connections 2026-02-17T06:01:26.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 -- 10.20.193.16:0/1462768355 wait complete. 2026-02-17T06:01:26.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:26.575+0000 7f9dbed5b640 1 librados: shutdown 2026-02-17T06:01:29.229 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:07:08.056 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:29.256 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:07:08.082 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:29.256 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:07:08.046 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:29.257 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:07:08.085 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:29.258 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:07:08.048 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:29.259 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:07:08.085 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:29.269 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:07:08.058 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:29.269 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:07:08.097 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:29.269 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:07:08.096 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:29.586 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:29.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.652+0000 7f1424946640 10 monclient: get_monmap_and_config 2026-02-17T06:01:29.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.652+0000 7f1424946640 10 monclient: build_initial_monmap 2026-02-17T06:01:29.653 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.652+0000 7f1424946640 10 monclient: monmap: 2026-02-17T06:01:29.654 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:29.654 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:29.654 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:29.653490+0000 2026-02-17T06:01:29.654 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:29.653490+0000 2026-02-17T06:01:29.654 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:29.654 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:29.654 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:29.654 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:29.654 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:29.654 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:29.655 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:29.655 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:29.655 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:29.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.652+0000 7f1424946640 1 Processor -- start 2026-02-17T06:01:29.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.652+0000 7f1424946640 1 -- start start 2026-02-17T06:01:29.655 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 10 monclient: init 2026-02-17T06:01:29.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:29.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 10 monclient: _add_conns ranks=[3,5,2,4,1,0] 2026-02-17T06:01:29.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f141c0700f0 0x7f141c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 10 monclient(hunting): picked mon.noname-a con 0x7f141c0700f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f141c0c7fe0 0x7f141c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 10 monclient(hunting): picked mon.noname-c con 0x7f141c0c7fe0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f141c1536f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f141c0c8970 con 0x7f141c1536f0 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f141c0c4820 con 0x7f141c0700f0 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f141c0c4b00 con 0x7f141c0c7fe0 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f1424946640 10 monclient(hunting): authenticate will time out at 1342.222536s 2026-02-17T06:01:29.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.653+0000 7f14226bb640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f141c0700f0 0x7f141c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f1421eba640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f141c0c7fe0 0x7f141c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f14226bb640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f141c0700f0 0x7f141c0704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51518/0 (socket says 10.20.193.16:51518) 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f14226bb640 1 -- 10.20.193.16:0/3960935825 learned_addr learned my addr 10.20.193.16:0/3960935825 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f14226bb640 10 monclient(hunting): get_auth_request con 0x7f141c0700f0 auth_method 0 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f14226bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f14226bb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f14226bb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f1421eba640 10 monclient(hunting): get_auth_request con 0x7f141c0c7fe0 auth_method 0 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f1421eba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f1421eba640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f1421eba640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:29.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f14226bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f14226bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f14226bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f1421eba640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f1421eba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.654+0000 7f1421eba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.655+0000 7f1421eba640 10 monclient(hunting): handle_auth_done global_id 5117 payload 274 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.655+0000 7f1421eba640 1 -- 10.20.193.16:0/3960935825 >> v2:10.20.193.16:3300/0 conn(0x7f141c0700f0 msgr2=0x7f141c0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.655+0000 7f1421eba640 1 --2- 10.20.193.16:0/3960935825 >> v2:10.20.193.16:3300/0 conn(0x7f141c0700f0 0x7f141c0704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.655+0000 7f1421eba640 1 -- 10.20.193.16:0/3960935825 >> v1:10.20.193.148:6789/0 conn(0x7f141c1536f0 legacy=0x7f141c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.655+0000 7f1421eba640 10 monclient: _finish_hunting 0 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.655+0000 7f1421eba640 1 monclient: found mon.noname-c 2026-02-17T06:01:29.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.655+0000 7f1421eba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:29.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.655+0000 7f1421eba640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:29.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.655+0000 7f1421eba640 1 -- 10.20.193.16:0/3960935825 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c0c4de0 con 0x7f141c0c7fe0 2026-02-17T06:01:29.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.657+0000 7f1421eba640 1 --2- 10.20.193.16:0/3960935825 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f141c0c7fe0 0x7f141c0c8430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f141c0c5b80 tx=0x7f140802ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8955e66ca7fcfe5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:29.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 1 -- 10.20.193.16:0/3960935825 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140803c070 con 0x7f141c0c7fe0 2026-02-17T06:01:29.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:29.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 10 monclient: dump: 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:29.671 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 10 monclient: _finish_auth 0 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 10 monclient: _check_auth_tickets 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 1 -- 10.20.193.16:0/3960935825 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f140802f9c0 con 0x7f141c0c7fe0 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 1 -- 10.20.193.16:0/3960935825 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140802fd90 con 0x7f141c0c7fe0 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f1424946640 5 monclient: authenticate success, global_id 5117 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f1424946640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f1424946640 4 set_mon_vals no callback set 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:29.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.658+0000 7f14216b9640 10 monclient: dump: 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 10 monclient: shutdown 2026-02-17T06:01:29.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 1 -- 10.20.193.16:0/3960935825 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f141c0c7fe0 msgr2=0x7f141c0c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 1 --2- 10.20.193.16:0/3960935825 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f141c0c7fe0 0x7f141c0c8430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f141c0c5b80 tx=0x7f140802ed50 comp rx=0 tx=0).stop 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 1 -- 10.20.193.16:0/3960935825 shutdown_connections 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 1 --2- 10.20.193.16:0/3960935825 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f141c0c7fe0 0x7f141c0c8430 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 1 --2- 10.20.193.16:0/3960935825 >> v2:10.20.193.16:3300/0 conn(0x7f141c0700f0 0x7f141c0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 1 -- 10.20.193.16:0/3960935825 >> 10.20.193.16:0/3960935825 conn(0x7f141c0cd770 msgr2=0x7f141c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 1 -- 10.20.193.16:0/3960935825 shutdown_connections 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 1 -- 10.20.193.16:0/3960935825 wait complete. 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 10 monclient: build_initial_monmap 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 10 monclient: monmap: 2026-02-17T06:01:29.674 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 1 librados: starting msgr at 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.659+0000 7f1424946640 1 librados: starting objecter 2026-02-17T06:01:29.675 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 1 Processor -- start 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 1 -- start start 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 1 librados: setting wanted keys 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 1 librados: calling monclient init 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient: init 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f141c0700f0 0x7f141c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient(hunting): picked mon.noname-c con 0x7f141c0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f141c0c7fe0 0x7f141c1f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient(hunting): picked mon.noname-b con 0x7f141c0c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f141c1536f0 0x7f141c145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient(hunting): picked mon.noname-a con 0x7f141c1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:29.676 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f141c0c5680 con 0x7f141c1536f0 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f141c0c53a0 con 0x7f141c0c7fe0 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f141c074c00 con 0x7f141c0700f0 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1424946640 10 monclient(hunting): authenticate will time out at 1342.229462s 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1422ebc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f141c1536f0 0x7f141c145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f14226bb640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f141c0700f0 0x7f141c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1422ebc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f141c1536f0 0x7f141c145910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:51520/0 (socket says 10.20.193.16:51520) 2026-02-17T06:01:29.677 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1422ebc640 1 -- 10.20.193.16:0/377336243 learned_addr learned my addr 10.20.193.16:0/377336243 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1422ebc640 10 monclient(hunting): get_auth_request con 0x7f141c1536f0 auth_method 0 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1422ebc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1422ebc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1422ebc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f14226bb640 10 monclient(hunting): get_auth_request con 0x7f141c0700f0 auth_method 0 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f14226bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f14226bb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f14226bb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.660+0000 7f1421eba640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f141c0c7fe0 0x7f141c1f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f1422ebc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f1422ebc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:29.678 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f1422ebc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f14226bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f14226bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f14226bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f1421eba640 10 monclient(hunting): get_auth_request con 0x7f141c0c7fe0 auth_method 0 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f1421eba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f1421eba640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f1421eba640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f1421eba640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f1421eba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.661+0000 7f1421eba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.662+0000 7f14226bb640 10 monclient(hunting): handle_auth_done global_id 5120 payload 824 2026-02-17T06:01:29.679 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.662+0000 7f14226bb640 1 -- 10.20.193.16:0/377336243 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f141c0c7fe0 msgr2=0x7f141c1f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.662+0000 7f14226bb640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f141c0c7fe0 0x7f141c1f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.662+0000 7f14226bb640 1 -- 10.20.193.16:0/377336243 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f141c1536f0 msgr2=0x7f141c145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.662+0000 7f14226bb640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f141c1536f0 0x7f141c145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.662+0000 7f14226bb640 10 monclient: _finish_hunting 0 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.662+0000 7f14226bb640 1 monclient: found mon.noname-c 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.662+0000 7f14226bb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.662+0000 7f14226bb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.662+0000 7f14226bb640 1 -- 10.20.193.16:0/377336243 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c1f7e20 con 0x7f141c0700f0 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.663+0000 7f14226bb640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f141c0700f0 0x7f141c06d270 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1414006dc0 tx=0x7f141400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.663+0000 7f1406ffd640 10 client.?.objecter ms_handle_connect 0x7f141c0700f0 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.663+0000 7f1406ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1406ffd640 1 -- 10.20.193.16:0/377336243 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414017070 con 0x7f141c0700f0 2026-02-17T06:01:29.680 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1406ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1406ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1406ffd640 10 monclient: dump: 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:29.681 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1406ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1406ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1406ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 5 monclient: authenticate success, global_id 5120 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 10 monclient: _renew_subs 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 1 -- 10.20.193.16:0/377336243 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141c1f3b50 con 0x7f141c0700f0 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 10 client.5120.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 10 monclient: _renew_subs 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 1 -- 10.20.193.16:0/377336243 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141c1f7b40 con 0x7f141c0700f0 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 1 librados: init done 2026-02-17T06:01:29.682 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.664+0000 7f1424946640 1 -- 10.20.193.16:0/377336243 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141c0704d0 con 0x7f141c0700f0 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.665+0000 7f1406ffd640 1 -- 10.20.193.16:0/377336243 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1414005a90 con 0x7f141c0700f0 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.665+0000 7f1406ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.665+0000 7f1406ffd640 1 -- 10.20.193.16:0/377336243 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414021060 con 0x7f141c0700f0 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.665+0000 7f1406ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.665+0000 7f1406ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.665+0000 7f1420eb8640 4 set_mon_vals no callback set 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.665+0000 7f1406ffd640 10 monclient: dump: 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:29.683 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:29.684 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:29.684 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:29.684 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:29.684 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:29.684 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:29.684 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:29.684 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:29.684 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:29.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.773+0000 7f1406ffd640 1 -- 10.20.193.16:0/377336243 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f141400c910 con 0x7f141c0700f0 2026-02-17T06:01:29.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.773+0000 7f1406ffd640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f13ec079140 0x7f13ec07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:29.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.774+0000 7f1421eba640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f13ec079140 0x7f13ec07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:29.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.774+0000 7f1421eba640 10 monclient: get_auth_request con 0x7f13ec079140 auth_method 0 2026-02-17T06:01:29.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.774+0000 7f1421eba640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f13ec079140 0x7f13ec07b600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1408039000 tx=0x7f1408033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:29.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.774+0000 7f1406ffd640 10 client.5120.objecter ms_handle_connect 0x7f13ec079140 2026-02-17T06:01:29.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.775+0000 7f1406ffd640 1 -- 10.20.193.16:0/377336243 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1414097fe0 con 0x7f141c0700f0 2026-02-17T06:01:29.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.775+0000 7f1406ffd640 10 client.5120.objecter ms_dispatch2 0x7f141c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:29.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.775+0000 7f1406ffd640 3 client.5120.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:29.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.775+0000 7f1406ffd640 3 client.5120.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:29.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.775+0000 7f1406ffd640 20 client.5120.objecter dump_active .. 0 homeless 2026-02-17T06:01:29.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.781+0000 7f1406ffd640 1 -- 10.20.193.16:0/377336243 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1414062610 con 0x7f141c0700f0 2026-02-17T06:01:29.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.781+0000 7f1406ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:29.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.781+0000 7f1406ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:29.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.883+0000 7f1424946640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:29.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.883+0000 7f1424946640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:29.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.883+0000 7f1424946640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:29.884 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.883+0000 7f1424946640 1 -- 10.20.193.16:0/377336243 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f141c1573f0 con 0x7f141c0700f0 2026-02-17T06:01:29.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.886+0000 7f1406ffd640 1 -- 10.20.193.16:0/377336243 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1414066910 con 0x7f141c0700f0 2026-02-17T06:01:29.886 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.886+0000 7f1406ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:29.887 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.886+0000 7f1406ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:29.887 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:29.887 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:29.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 10 librados: watch_flush enter 2026-02-17T06:01:29.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 10 librados: watch_flush exit 2026-02-17T06:01:29.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 20 client.5120.objecter shutdown clearing up homeless session... 2026-02-17T06:01:29.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 10 client.5120.objecter successfully canceled tick 2026-02-17T06:01:29.889 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 -- 10.20.193.16:0/377336243 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f13ec079140 msgr2=0x7f13ec07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f13ec079140 0x7f13ec07b600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1408039000 tx=0x7f1408033000 comp rx=0 tx=0).stop 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 10 monclient: shutdown 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 -- 10.20.193.16:0/377336243 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f141c0700f0 msgr2=0x7f141c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f141c0700f0 0x7f141c06d270 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1414006dc0 tx=0x7f141400ed70 comp rx=0 tx=0).stop 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 -- 10.20.193.16:0/377336243 shutdown_connections 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f13ec079140 0x7f13ec07b600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f141c0c7fe0 0x7f141c1f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f141c1536f0 0x7f141c145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 --2- 10.20.193.16:0/377336243 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f141c0700f0 0x7f141c06d270 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:29.890 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 -- 10.20.193.16:0/377336243 >> 10.20.193.16:0/377336243 conn(0x7f141c0cd770 msgr2=0x7f141c14ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:29.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 -- 10.20.193.16:0/377336243 shutdown_connections 2026-02-17T06:01:29.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 -- 10.20.193.16:0/377336243 wait complete. 2026-02-17T06:01:29.891 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:29.887+0000 7f1424946640 1 librados: shutdown 2026-02-17T06:01:32.897 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.967+0000 7f1d2123b640 10 monclient: get_monmap_and_config 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.967+0000 7f1d2123b640 10 monclient: build_initial_monmap 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.967+0000 7f1d2123b640 10 monclient: monmap: 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:32.968753+0000 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:32.968753+0000 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:32.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 1 Processor -- start 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 1 -- start start 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 10 monclient: init 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 10 monclient: _add_conns ranks=[4,5,0,3,1,2] 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1d1c14f020 0x7f1d1c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 10 monclient(hunting): picked mon.noname-b con 0x7f1d1c14f020 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f1d1c06e120 0x7f1d1c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 10 monclient(hunting): picked mon.noname-c con 0x7f1d1c06e120 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1d1c14f400 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d1c0c5b10 con 0x7f1d1c14f400 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f1d1c06e620 con 0x7f1d1c14f020 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f1d1c06e900 con 0x7f1d1c06e120 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.968+0000 7f1d2123b640 10 monclient(hunting): authenticate will time out at 1345.537831s 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1b577640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f1d1c14f400 0x7f1d1c0c6010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:49990/0 (socket says 10.20.193.16:49990) 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1b577640 1 -- 10.20.193.16:0/2083791695 learned_addr learned my addr 10.20.193.16:0/2083791695 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1ad76640 1 --2- 10.20.193.16:0/2083791695 >> v2:10.20.193.128:3300/0 conn(0x7f1d1c14f020 0x7f1d1c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:32.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1a575640 1 --2- 10.20.193.16:0/2083791695 >> v2:10.20.193.148:3300/0 conn(0x7f1d1c06e120 0x7f1d1c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1ad76640 10 monclient(hunting): get_auth_request con 0x7f1d1c14f020 auth_method 0 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1a575640 10 monclient(hunting): get_auth_request con 0x7f1d1c06e120 auth_method 0 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.969+0000 7f1d1ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d19d74640 1 -- 10.20.193.16:0/2083791695 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1980085981 0 0) 0x7f1d1c0c5b10 con 0x7f1d1c14f400 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d19d74640 10 monclient(hunting): _negotiate 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d19d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d19d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d19d74640 10 monclient(hunting): my global_id is 4995 2026-02-17T06:01:32.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d19d74640 1 -- 10.20.193.16:0/2083791695 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d08003620 con 0x7f1d1c14f400 2026-02-17T06:01:32.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1ad76640 10 monclient(hunting): handle_auth_done global_id 5059 payload 274 2026-02-17T06:01:32.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1ad76640 1 -- 10.20.193.16:0/2083791695 >> v2:10.20.193.148:3300/0 conn(0x7f1d1c06e120 msgr2=0x7f1d1c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:32.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1ad76640 1 --2- 10.20.193.16:0/2083791695 >> v2:10.20.193.148:3300/0 conn(0x7f1d1c06e120 0x7f1d1c15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:32.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1ad76640 1 -- 10.20.193.16:0/2083791695 >> v1:10.20.193.16:6789/0 conn(0x7f1d1c14f400 legacy=0x7f1d1c0c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:32.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1ad76640 10 monclient: _finish_hunting 0 2026-02-17T06:01:32.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1ad76640 1 monclient: found mon.noname-b 2026-02-17T06:01:32.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:32.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1ad76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:32.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.970+0000 7f1d1ad76640 1 -- 10.20.193.16:0/2083791695 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c06ebe0 con 0x7f1d1c14f020 2026-02-17T06:01:32.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.971+0000 7f1d1ad76640 1 --2- 10.20.193.16:0/2083791695 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 0x7f1d1c06dbe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c156400 tx=0x7f1d1002eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=488c651f505568f9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d19d74640 1 -- 10.20.193.16:0/2083791695 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1003c070 con 0x7f1d1c14f020 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d19d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d19d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d19d74640 10 monclient: dump: 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d19d74640 10 monclient: _finish_auth 0 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d19d74640 10 monclient: _check_auth_tickets 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d19d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d19d74640 1 -- 10.20.193.16:0/2083791695 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d1002fb80 con 0x7f1d1c14f020 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d19d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d19d74640 1 -- 10.20.193.16:0/2083791695 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10037440 con 0x7f1d1c14f020 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.973+0000 7f1d2123b640 5 monclient: authenticate success, global_id 5059 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 4 set_mon_vals no callback set 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d19d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d19d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d19d74640 10 monclient: dump: 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:32.974 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 10 monclient: shutdown 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 1 -- 10.20.193.16:0/2083791695 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 msgr2=0x7f1d1c06dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 1 --2- 10.20.193.16:0/2083791695 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 0x7f1d1c06dbe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c156400 tx=0x7f1d1002eeb0 comp rx=0 tx=0).stop 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 1 -- 10.20.193.16:0/2083791695 shutdown_connections 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 1 --2- 10.20.193.16:0/2083791695 >> v2:10.20.193.148:3300/0 conn(0x7f1d1c06e120 0x7f1d1c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 1 --2- 10.20.193.16:0/2083791695 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 0x7f1d1c06dbe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 1 -- 10.20.193.16:0/2083791695 >> 10.20.193.16:0/2083791695 conn(0x7f1d1c0cd770 msgr2=0x7f1d1c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 1 -- 10.20.193.16:0/2083791695 shutdown_connections 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 1 -- 10.20.193.16:0/2083791695 wait complete. 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 10 monclient: build_initial_monmap 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.974+0000 7f1d2123b640 10 monclient: monmap: 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:32.975 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 librados: starting msgr at 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 librados: starting objecter 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 Processor -- start 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 -- start start 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 librados: setting wanted keys 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 librados: calling monclient init 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient: init 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:32.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1d1c06e120 0x7f1d1c15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient(hunting): picked mon.noname-a con 0x7f1d1c06e120 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 0x7f1d1c0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient(hunting): picked mon.noname-b con 0x7f1d1c14f020 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d1c14f400 0x7f1d1c143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient(hunting): picked mon.noname-c con 0x7f1d1c14f400 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c0c6720 con 0x7f1d1c06e120 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c06f1a0 con 0x7f1d1c14f020 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c0c68a0 con 0x7f1d1c14f400 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d2123b640 10 monclient(hunting): authenticate will time out at 1345.544560s 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d1a575640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 0x7f1d1c0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d1ad76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1d1c06e120 0x7f1d1c15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d1a575640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 0x7f1d1c0cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:34426/0 (socket says 10.20.193.16:34426) 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d1a575640 1 -- 10.20.193.16:0/1714920651 learned_addr learned my addr 10.20.193.16:0/1714920651 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d1a575640 10 monclient(hunting): get_auth_request con 0x7f1d1c14f020 auth_method 0 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d1a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d1a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.975+0000 7f1d1a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1ad76640 10 monclient(hunting): get_auth_request con 0x7f1d1c06e120 auth_method 0 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1b577640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d1c14f400 0x7f1d1c143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1b577640 10 monclient(hunting): get_auth_request con 0x7f1d1c14f400 auth_method 0 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1b577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1b577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1b577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:32.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.976+0000 7f1d1b577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.977+0000 7f1d1a575640 10 monclient(hunting): handle_auth_done global_id 5062 payload 824 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.977+0000 7f1d1a575640 1 -- 10.20.193.16:0/1714920651 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d1c14f400 msgr2=0x7f1d1c143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.977+0000 7f1d1a575640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d1c14f400 0x7f1d1c143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.977+0000 7f1d1a575640 1 -- 10.20.193.16:0/1714920651 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1d1c06e120 msgr2=0x7f1d1c15c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.977+0000 7f1d1a575640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1d1c06e120 0x7f1d1c15c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.977+0000 7f1d1a575640 10 monclient: _finish_hunting 0 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.977+0000 7f1d1a575640 1 monclient: found mon.noname-b 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.977+0000 7f1d1a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.977+0000 7f1d1a575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.977+0000 7f1d1a575640 1 -- 10.20.193.16:0/1714920651 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c144500 con 0x7f1d1c14f020 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.978+0000 7f1d1a575640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 0x7f1d1c0cc550 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400e890 tx=0x7f1d0400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.978+0000 7f1cfb7fe640 10 client.?.objecter ms_handle_connect 0x7f1d1c14f020 2026-02-17T06:01:32.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.978+0000 7f1cfb7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 1 -- 10.20.193.16:0/1714920651 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04017070 con 0x7f1d1c14f020 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 10 monclient: dump: 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:32.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 1 -- 10.20.193.16:0/1714920651 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d0400c940 con 0x7f1d1c14f020 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 1 -- 10.20.193.16:0/1714920651 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400cd10 con 0x7f1d1c14f020 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 5 monclient: authenticate success, global_id 5062 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d19573640 4 set_mon_vals no callback set 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 10 monclient: _renew_subs 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 1 -- 10.20.193.16:0/1714920651 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c1447e0 con 0x7f1d1c14f020 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1cfb7fe640 10 monclient: dump: 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 10 client.5062.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 10 monclient: _renew_subs 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 1 -- 10.20.193.16:0/1714920651 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c159e10 con 0x7f1d1c14f020 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 1 librados: init done 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:32.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:32.980+0000 7f1d2123b640 1 -- 10.20.193.16:0/1714920651 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1c06dbe0 con 0x7f1d1c14f020 2026-02-17T06:01:33.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.085+0000 7f1cfb7fe640 1 -- 10.20.193.16:0/1714920651 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1d04013470 con 0x7f1d1c14f020 2026-02-17T06:01:33.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.085+0000 7f1cfb7fe640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ce8079140 0x7f1ce807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:33.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.085+0000 7f1d1ad76640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ce8079140 0x7f1ce807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:33.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.085+0000 7f1d1ad76640 10 monclient: get_auth_request con 0x7f1ce8079140 auth_method 0 2026-02-17T06:01:33.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.085+0000 7f1cfb7fe640 10 client.5062.objecter ms_handle_connect 0x7f1ce8079140 2026-02-17T06:01:33.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.085+0000 7f1d1ad76640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ce8079140 0x7f1ce807b600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d10039000 tx=0x7f1d10033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:33.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.086+0000 7f1cfb7fe640 1 -- 10.20.193.16:0/1714920651 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1d0405e610 con 0x7f1d1c14f020 2026-02-17T06:01:33.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.086+0000 7f1cfb7fe640 10 client.5062.objecter ms_dispatch2 0x7f1d1c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:33.087 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.086+0000 7f1cfb7fe640 3 client.5062.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:33.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.086+0000 7f1cfb7fe640 3 client.5062.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:33.088 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.086+0000 7f1cfb7fe640 20 client.5062.objecter dump_active .. 0 homeless 2026-02-17T06:01:33.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.092+0000 7f1cfb7fe640 1 -- 10.20.193.16:0/1714920651 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1d040625d0 con 0x7f1d1c14f020 2026-02-17T06:01:33.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.092+0000 7f1cfb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:33.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.092+0000 7f1cfb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:33.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.191+0000 7f1d2123b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:33.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.191+0000 7f1d2123b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:33.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.191+0000 7f1d2123b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:33.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.191+0000 7f1d2123b640 1 -- 10.20.193.16:0/1714920651 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1d1c1562c0 con 0x7f1d1c14f020 2026-02-17T06:01:33.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.194+0000 7f1cfb7fe640 1 -- 10.20.193.16:0/1714920651 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1d040668d0 con 0x7f1d1c14f020 2026-02-17T06:01:33.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.194+0000 7f1cfb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:33.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.194+0000 7f1cfb7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:33.195 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:33.195 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:33.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 10 librados: watch_flush enter 2026-02-17T06:01:33.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 10 librados: watch_flush exit 2026-02-17T06:01:33.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 20 client.5062.objecter shutdown clearing up homeless session... 2026-02-17T06:01:33.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 10 client.5062.objecter successfully canceled tick 2026-02-17T06:01:33.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 -- 10.20.193.16:0/1714920651 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ce8079140 msgr2=0x7f1ce807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:33.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ce8079140 0x7f1ce807b600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d10039000 tx=0x7f1d10033000 comp rx=0 tx=0).stop 2026-02-17T06:01:33.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 10 monclient: shutdown 2026-02-17T06:01:33.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:33.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 -- 10.20.193.16:0/1714920651 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 msgr2=0x7f1d1c0cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:33.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 0x7f1d1c0cc550 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400e890 tx=0x7f1d0400ed60 comp rx=0 tx=0).stop 2026-02-17T06:01:33.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 -- 10.20.193.16:0/1714920651 shutdown_connections 2026-02-17T06:01:33.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1ce8079140 0x7f1ce807b600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:33.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d1c14f020 0x7f1d1c0cc550 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:33.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d1c14f400 0x7f1d1c143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:33.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 --2- 10.20.193.16:0/1714920651 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1d1c06e120 0x7f1d1c15c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:33.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 -- 10.20.193.16:0/1714920651 >> 10.20.193.16:0/1714920651 conn(0x7f1d1c0cd770 msgr2=0x7f1d1c14df40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:33.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 -- 10.20.193.16:0/1714920651 shutdown_connections 2026-02-17T06:01:33.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 -- 10.20.193.16:0/1714920651 wait complete. 2026-02-17T06:01:33.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:33.195+0000 7f1d2123b640 1 librados: shutdown 2026-02-17T06:01:36.206 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:36.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.274+0000 7f804336a640 10 monclient: get_monmap_and_config 2026-02-17T06:01:36.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.274+0000 7f804336a640 10 monclient: build_initial_monmap 2026-02-17T06:01:36.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.274+0000 7f804336a640 10 monclient: monmap: 2026-02-17T06:01:36.275 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:36.275 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:36.275 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:36.275170+0000 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:36.275170+0000 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.274+0000 7f804336a640 1 Processor -- start 2026-02-17T06:01:36.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.274+0000 7f804336a640 1 -- start start 2026-02-17T06:01:36.277 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.274+0000 7f804336a640 10 monclient: init 2026-02-17T06:01:36.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:36.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 10 monclient: _add_conns ranks=[5,1,2,4,3,0] 2026-02-17T06:01:36.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f803c0700f0 0x7f803c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 10 monclient(hunting): picked mon.noname-c con 0x7f803c0700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:36.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f803c0c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f803c1536f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c0c4820 con 0x7f803c0c7fe0 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c0c4b00 con 0x7f803c1536f0 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f803c0c4de0 con 0x7f803c0700f0 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f804336a640 10 monclient(hunting): authenticate will time out at 1348.844235s 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f80408de640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f803c0c7fe0 0x7f803c0c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:50834/0 (socket says 10.20.193.16:50834) 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f80408de640 1 -- 10.20.193.16:0/3755897583 learned_addr learned my addr 10.20.193.16:0/3755897583 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f80410df640 1 --2- 10.20.193.16:0/3755897583 >> v2:10.20.193.148:3300/0 conn(0x7f803c0700f0 0x7f803c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.275+0000 7f80410df640 10 monclient(hunting): get_auth_request con 0x7f803c0700f0 auth_method 0 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.276+0000 7f80410df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.276+0000 7f80410df640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.276+0000 7f80410df640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.276+0000 7f80410df640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.276+0000 7f80410df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.276+0000 7f80410df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f802bfff640 1 -- 10.20.193.16:0/3755897583 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4025057815 0 0) 0x7f803c0c4820 con 0x7f803c0c7fe0 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f802bfff640 10 monclient(hunting): _negotiate 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f802bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f802bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f802bfff640 10 monclient(hunting): my global_id is 5065 2026-02-17T06:01:36.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f802bfff640 1 -- 10.20.193.16:0/3755897583 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8024003620 con 0x7f803c0c7fe0 2026-02-17T06:01:36.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f80410df640 10 monclient(hunting): handle_auth_done global_id 5129 payload 274 2026-02-17T06:01:36.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f80410df640 1 -- 10.20.193.16:0/3755897583 >> v1:10.20.193.148:6789/0 conn(0x7f803c1536f0 legacy=0x7f803c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:36.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f80410df640 1 -- 10.20.193.16:0/3755897583 >> v1:10.20.193.128:6789/0 conn(0x7f803c0c7fe0 legacy=0x7f803c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f80410df640 10 monclient: _finish_hunting 0 2026-02-17T06:01:36.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f80410df640 1 monclient: found mon.noname-c 2026-02-17T06:01:36.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f80410df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:36.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f80410df640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:36.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.277+0000 7f80410df640 1 -- 10.20.193.16:0/3755897583 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c0c50c0 con 0x7f803c0700f0 2026-02-17T06:01:36.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.278+0000 7f80410df640 1 --2- 10.20.193.16:0/3755897583 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c0700f0 0x7f803c0704d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f802c0098a0 tx=0x7f802c02ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a58d3ee12204070b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:36.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 1 -- 10.20.193.16:0/3755897583 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c03c070 con 0x7f803c0700f0 2026-02-17T06:01:36.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 10 monclient: dump: 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:36.290 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 10 monclient: _finish_auth 0 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 1 -- 10.20.193.16:0/3755897583 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f802c02fa40 con 0x7f803c0700f0 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f804336a640 5 monclient: authenticate success, global_id 5129 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f804336a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 1 -- 10.20.193.16:0/3755897583 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c02fe10 con 0x7f803c0700f0 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f802bfff640 10 monclient: dump: 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:36.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f804336a640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.280+0000 7f804336a640 4 set_mon_vals no callback set 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 10 monclient: shutdown 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 1 -- 10.20.193.16:0/3755897583 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c0700f0 msgr2=0x7f803c0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 1 --2- 10.20.193.16:0/3755897583 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c0700f0 0x7f803c0704d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f802c0098a0 tx=0x7f802c02ed80 comp rx=0 tx=0).stop 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 1 -- 10.20.193.16:0/3755897583 shutdown_connections 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 1 --2- 10.20.193.16:0/3755897583 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c0700f0 0x7f803c0704d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 1 -- 10.20.193.16:0/3755897583 >> 10.20.193.16:0/3755897583 conn(0x7f803c0cd770 msgr2=0x7f803c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 1 -- 10.20.193.16:0/3755897583 shutdown_connections 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 1 -- 10.20.193.16:0/3755897583 wait complete. 2026-02-17T06:01:36.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 10 monclient: build_initial_monmap 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.281+0000 7f804336a640 10 monclient: monmap: 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:36.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 librados: starting msgr at 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 librados: starting objecter 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 Processor -- start 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 -- start start 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 librados: setting wanted keys 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 librados: calling monclient init 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient: init 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f803c0700f0 0x7f803c1f1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient(hunting): picked mon.noname-a con 0x7f803c0700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c0c7fe0 0x7f803c1f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient(hunting): picked mon.noname-b con 0x7f803c0c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c1536f0 0x7f803c14dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient(hunting): picked mon.noname-c con 0x7f803c1536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f803c074c00 con 0x7f803c0700f0 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0c5680 con 0x7f803c0c7fe0 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f803c074d80 con 0x7f803c1536f0 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f804336a640 10 monclient(hunting): authenticate will time out at 1348.851600s 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f80418e0640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c1536f0 0x7f803c14dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f80410df640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f803c0700f0 0x7f803c1f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.296 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.282+0000 7f80408de640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c0c7fe0 0x7f803c1f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80418e0640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c1536f0 0x7f803c14dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:38004/0 (socket says 10.20.193.16:38004) 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80418e0640 1 -- 10.20.193.16:0/621522531 learned_addr learned my addr 10.20.193.16:0/621522531 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80418e0640 10 monclient(hunting): get_auth_request con 0x7f803c1536f0 auth_method 0 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80418e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80418e0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80418e0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80410df640 10 monclient(hunting): get_auth_request con 0x7f803c0700f0 auth_method 0 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80410df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80410df640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80410df640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80408de640 10 monclient(hunting): get_auth_request con 0x7f803c0c7fe0 auth_method 0 2026-02-17T06:01:36.297 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80408de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80408de640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80408de640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80408de640 1 -- 10.20.193.16:0/621522531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c0c7fe0 msgr2=0x7f803c1f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80408de640 1 -- 10.20.193.16:0/621522531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c0c7fe0 msgr2=0x7f803c1f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80408de640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c0c7fe0 0x7f803c1f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80408de640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c0c7fe0 0x7f803c1f4580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80418e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80418e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80418e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80410df640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80410df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:36.298 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.283+0000 7f80410df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.284+0000 7f80418e0640 10 monclient(hunting): handle_auth_done global_id 5132 payload 824 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.284+0000 7f80418e0640 1 -- 10.20.193.16:0/621522531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c0c7fe0 msgr2=0x7f803c1f4580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.284+0000 7f80418e0640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c0c7fe0 0x7f803c1f4580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.284+0000 7f80418e0640 1 -- 10.20.193.16:0/621522531 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f803c0700f0 msgr2=0x7f803c1f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.284+0000 7f80418e0640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f803c0700f0 0x7f803c1f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.284+0000 7f80418e0640 10 monclient: _finish_hunting 0 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.284+0000 7f80418e0640 1 monclient: found mon.noname-c 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.284+0000 7f80418e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.284+0000 7f80418e0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.284+0000 7f80418e0640 1 -- 10.20.193.16:0/621522531 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c1f4ac0 con 0x7f803c1536f0 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.285+0000 7f80418e0640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c1536f0 0x7f803c14dbb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f803800b990 tx=0x7f803800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:36.299 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.285+0000 7f8029ffb640 10 client.?.objecter ms_handle_connect 0x7f803c1536f0 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.285+0000 7f8029ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 1 -- 10.20.193.16:0/621522531 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803800c790 con 0x7f803c1536f0 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 10 monclient: dump: 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:36.300 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 1 -- 10.20.193.16:0/621522531 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8038010070 con 0x7f803c1536f0 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 5 monclient: authenticate success, global_id 5132 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 1 -- 10.20.193.16:0/621522531 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803800cbb0 con 0x7f803c1536f0 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f802b7fe640 4 set_mon_vals no callback set 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f8029ffb640 10 monclient: dump: 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:36.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 10 monclient: _renew_subs 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 1 -- 10.20.193.16:0/621522531 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c1f4d90 con 0x7f803c1536f0 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 10 client.5132.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 10 monclient: _renew_subs 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 1 -- 10.20.193.16:0/621522531 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c1f5350 con 0x7f803c1536f0 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 1 librados: init done 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:36.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.287+0000 7f804336a640 1 -- 10.20.193.16:0/621522531 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f803c0704d0 con 0x7f803c1536f0 2026-02-17T06:01:36.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.299+0000 7f8029ffb640 1 -- 10.20.193.16:0/621522531 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f8038004070 con 0x7f803c1536f0 2026-02-17T06:01:36.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.299+0000 7f8029ffb640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 0x7f800c07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.300+0000 7f80410df640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 0x7f800c07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.300+0000 7f80410df640 10 monclient: get_auth_request con 0x7f800c079190 auth_method 0 2026-02-17T06:01:36.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.300+0000 7f8029ffb640 10 client.5132.objecter ms_handle_connect 0x7f800c079190 2026-02-17T06:01:36.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.300+0000 7f80410df640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 0x7f800c07b650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f802c039000 tx=0x7f802c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:36.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.300+0000 7f8029ffb640 1 -- 10.20.193.16:0/621522531 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f803800cd50 con 0x7f803c1536f0 2026-02-17T06:01:36.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.300+0000 7f8029ffb640 10 client.5132.objecter ms_dispatch2 0x7f803c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:36.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.300+0000 7f8029ffb640 3 client.5132.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:36.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.300+0000 7f8029ffb640 3 client.5132.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:36.306 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.300+0000 7f8029ffb640 20 client.5132.objecter dump_active .. 0 homeless 2026-02-17T06:01:36.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.399+0000 7f8029ffb640 1 -- 10.20.193.16:0/621522531 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f80380624e0 con 0x7f803c1536f0 2026-02-17T06:01:36.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.399+0000 7f8029ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:36.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.399+0000 7f8029ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:36.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.501+0000 7f804336a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:36.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.501+0000 7f804336a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:36.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.501+0000 7f804336a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:36.502 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.501+0000 7f804336a640 1 -- 10.20.193.16:0/621522531 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f803c1573f0 con 0x7f803c1536f0 2026-02-17T06:01:36.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.504+0000 7f8029ffb640 1 -- 10.20.193.16:0/621522531 <== mon.2 v2:10.20.193.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8038061de0 con 0x7f803c1536f0 2026-02-17T06:01:36.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.504+0000 7f8029ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:36.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.504+0000 7f8029ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:36.505 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:36.505 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:36.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 10 librados: watch_flush enter 2026-02-17T06:01:36.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 10 librados: watch_flush exit 2026-02-17T06:01:36.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 20 client.5132.objecter shutdown clearing up homeless session... 2026-02-17T06:01:36.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 10 client.5132.objecter successfully canceled tick 2026-02-17T06:01:36.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 -- 10.20.193.16:0/621522531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 msgr2=0x7f800c07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 0x7f800c07b650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f802c039000 tx=0x7f802c033000 comp rx=0 tx=0).stop 2026-02-17T06:01:36.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 10 monclient: shutdown 2026-02-17T06:01:36.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:36.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 -- 10.20.193.16:0/621522531 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c1536f0 msgr2=0x7f803c14dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c1536f0 0x7f803c14dbb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f803800b990 tx=0x7f803800be60 comp rx=0 tx=0).stop 2026-02-17T06:01:36.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 -- 10.20.193.16:0/621522531 shutdown_connections 2026-02-17T06:01:36.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f800c079190 0x7f800c07b650 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f803c0c7fe0 0x7f803c1f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f803c1536f0 0x7f803c14dbb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 --2- 10.20.193.16:0/621522531 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f803c0700f0 0x7f803c1f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 -- 10.20.193.16:0/621522531 >> 10.20.193.16:0/621522531 conn(0x7f803c0cd770 msgr2=0x7f803c0c6040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:36.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 -- 10.20.193.16:0/621522531 shutdown_connections 2026-02-17T06:01:36.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 -- 10.20.193.16:0/621522531 wait complete. 2026-02-17T06:01:36.508 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:36.505+0000 7f804336a640 1 librados: shutdown 2026-02-17T06:01:39.516 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.586+0000 7f4538555640 10 monclient: get_monmap_and_config 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.586+0000 7f4538555640 10 monclient: build_initial_monmap 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.586+0000 7f4538555640 10 monclient: monmap: 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:39.587309+0000 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:39.587309+0000 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:39.586 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.586+0000 7f4538555640 1 Processor -- start 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.586+0000 7f4538555640 1 -- start start 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.586+0000 7f4538555640 10 monclient: init 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 10 monclient: _add_conns ranks=[5,0,4,3,2,1] 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f453014f020 0x7f453006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 10 monclient(hunting): picked mon.noname-c con 0x7f453014f020 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f453006e120 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f453015caf0 0x7f45300c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 10 monclient(hunting): picked mon.noname-b con 0x7f453015caf0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45300c5b10 con 0x7f453006e120 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f453006e620 con 0x7f453015caf0 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f453006e900 con 0x7f453014f020 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:39.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4538555640 10 monclient(hunting): authenticate will time out at 1352.156389s 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4536acb640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f453015caf0 0x7f45300c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4535ac9640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f453006e120 0x7f453015c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:50006/0 (socket says 10.20.193.16:50006) 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f4535ac9640 1 -- 10.20.193.16:0/4061416754 learned_addr learned my addr 10.20.193.16:0/4061416754 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.587+0000 7f45362ca640 1 --2- 10.20.193.16:0/4061416754 >> v2:10.20.193.148:3300/0 conn(0x7f453014f020 0x7f453006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f4536acb640 10 monclient(hunting): get_auth_request con 0x7f453015caf0 auth_method 0 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f4536acb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f4536acb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f4536acb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f45362ca640 10 monclient(hunting): get_auth_request con 0x7f453014f020 auth_method 0 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f45362ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f45362ca640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f45362ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f4536acb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f4536acb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f4536acb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f45362ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f45362ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:39.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.588+0000 7f45362ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f45352c8640 1 -- 10.20.193.16:0/4061416754 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3487107195 0 0) 0x7f45300c5b10 con 0x7f453006e120 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f45352c8640 10 monclient(hunting): _negotiate 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f45352c8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f45352c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f45352c8640 10 monclient(hunting): my global_id is 5004 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f45352c8640 1 -- 10.20.193.16:0/4061416754 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4518003620 con 0x7f453006e120 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f4536acb640 10 monclient(hunting): handle_auth_done global_id 5071 payload 274 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f4536acb640 1 -- 10.20.193.16:0/4061416754 >> v2:10.20.193.148:3300/0 conn(0x7f453014f020 msgr2=0x7f453006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f4536acb640 1 --2- 10.20.193.16:0/4061416754 >> v2:10.20.193.148:3300/0 conn(0x7f453014f020 0x7f453006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f4536acb640 1 -- 10.20.193.16:0/4061416754 >> v1:10.20.193.16:6789/0 conn(0x7f453006e120 legacy=0x7f453015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f4536acb640 10 monclient: _finish_hunting 0 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f4536acb640 1 monclient: found mon.noname-b 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f4536acb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f4536acb640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:39.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.589+0000 7f4536acb640 1 -- 10.20.193.16:0/4061416754 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453006ebe0 con 0x7f453015caf0 2026-02-17T06:01:39.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.590+0000 7f4536acb640 1 --2- 10.20.193.16:0/4061416754 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453015caf0 0x7f45300c6010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f45300ce3c0 tx=0x7f452c02f250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb53920c69e8710c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 1 -- 10.20.193.16:0/4061416754 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c03c070 con 0x7f453015caf0 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 10 monclient: dump: 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:39.592 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 10 monclient: _finish_auth 0 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 10 monclient: _check_auth_tickets 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 1 -- 10.20.193.16:0/4061416754 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452c037440 con 0x7f453015caf0 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 1 -- 10.20.193.16:0/4061416754 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c037810 con 0x7f453015caf0 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 5 monclient: authenticate success, global_id 5071 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 4 set_mon_vals no callback set 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f45352c8640 10 monclient: dump: 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 10 monclient: shutdown 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 1 -- 10.20.193.16:0/4061416754 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453015caf0 msgr2=0x7f45300c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 1 --2- 10.20.193.16:0/4061416754 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453015caf0 0x7f45300c6010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f45300ce3c0 tx=0x7f452c02f250 comp rx=0 tx=0).stop 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 1 -- 10.20.193.16:0/4061416754 shutdown_connections 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 1 --2- 10.20.193.16:0/4061416754 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453015caf0 0x7f45300c6010 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 1 --2- 10.20.193.16:0/4061416754 >> v2:10.20.193.148:3300/0 conn(0x7f453014f020 0x7f453006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 1 -- 10.20.193.16:0/4061416754 >> 10.20.193.16:0/4061416754 conn(0x7f45300cd770 msgr2=0x7f45300cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 1 -- 10.20.193.16:0/4061416754 shutdown_connections 2026-02-17T06:01:39.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.592+0000 7f4538555640 1 -- 10.20.193.16:0/4061416754 wait complete. 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient: build_initial_monmap 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient: monmap: 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 librados: starting msgr at 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 librados: starting objecter 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 Processor -- start 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 -- start start 2026-02-17T06:01:39.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 librados: setting wanted keys 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 librados: calling monclient init 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient: init 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453006e120 0x7f4530146750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient(hunting): picked mon.noname-b con 0x7f453006e120 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f453014f020 0x7f4530070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient(hunting): picked mon.noname-c con 0x7f453014f020 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f453015caf0 0x7f45301f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient(hunting): picked mon.noname-a con 0x7f453015caf0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f453014f400 con 0x7f453015caf0 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f453006f1a0 con 0x7f453006e120 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f453014f580 con 0x7f453014f020 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.593+0000 7f4538555640 10 monclient(hunting): authenticate will time out at 1352.162809s 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4536acb640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f453015caf0 0x7f45301f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f45362ca640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453006e120 0x7f4530146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f45362ca640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453006e120 0x7f4530146750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:34460/0 (socket says 10.20.193.16:34460) 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f45362ca640 1 -- 10.20.193.16:0/3027475130 learned_addr learned my addr 10.20.193.16:0/3027475130 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f45362ca640 10 monclient(hunting): get_auth_request con 0x7f453006e120 auth_method 0 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f45362ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f45362ca640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f45362ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4535ac9640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f453014f020 0x7f4530070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4536acb640 10 monclient(hunting): get_auth_request con 0x7f453015caf0 auth_method 0 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4536acb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4536acb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4536acb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4535ac9640 10 monclient(hunting): get_auth_request con 0x7f453014f020 auth_method 0 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4535ac9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4535ac9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:39.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4535ac9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f45362ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f45362ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f45362ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4536acb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4536acb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4536acb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4535ac9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4535ac9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.594+0000 7f4535ac9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.595+0000 7f45362ca640 10 monclient(hunting): handle_auth_done global_id 5074 payload 824 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.595+0000 7f45362ca640 1 -- 10.20.193.16:0/3027475130 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f453014f020 msgr2=0x7f4530070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.595+0000 7f45362ca640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f453014f020 0x7f4530070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.595+0000 7f45362ca640 1 -- 10.20.193.16:0/3027475130 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f453015caf0 msgr2=0x7f45301f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.595+0000 7f45362ca640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f453015caf0 0x7f45301f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.595+0000 7f45362ca640 10 monclient: _finish_hunting 0 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.595+0000 7f45362ca640 1 monclient: found mon.noname-b 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.595+0000 7f45362ca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.595+0000 7f45362ca640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.595+0000 7f45362ca640 1 -- 10.20.193.16:0/3027475130 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4530148e80 con 0x7f453006e120 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.596+0000 7f45362ca640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453006e120 0x7f4530146750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f452000ebe0 tx=0x7f452000c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:39.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.596+0000 7f4516ffd640 10 client.?.objecter ms_handle_connect 0x7f453006e120 2026-02-17T06:01:39.597 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.596+0000 7f4516ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:39.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 1 -- 10.20.193.16:0/3027475130 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520017070 con 0x7f453006e120 2026-02-17T06:01:39.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:39.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:39.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 10 monclient: dump: 2026-02-17T06:01:39.598 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:39.598 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:39.598 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:39.598 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 1 -- 10.20.193.16:0/3027475130 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452000cc40 con 0x7f453006e120 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 1 -- 10.20.193.16:0/3027475130 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520013470 con 0x7f453006e120 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4534ac7640 4 set_mon_vals no callback set 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4516ffd640 10 monclient: dump: 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4538555640 5 monclient: authenticate success, global_id 5074 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4538555640 10 monclient: _renew_subs 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4538555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4538555640 1 -- 10.20.193.16:0/3027475130 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45301f2650 con 0x7f453006e120 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4538555640 10 client.5074.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4538555640 10 monclient: _renew_subs 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4538555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4538555640 1 -- 10.20.193.16:0/3027475130 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45301f2bd0 con 0x7f453006e120 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.598+0000 7f4538555640 1 librados: init done 2026-02-17T06:01:39.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.599+0000 7f4538555640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:39.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.599+0000 7f4538555640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:39.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.599+0000 7f4538555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:39.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.599+0000 7f4538555640 1 -- 10.20.193.16:0/3027475130 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453006dbe0 con 0x7f453006e120 2026-02-17T06:01:39.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.610+0000 7f4516ffd640 1 -- 10.20.193.16:0/3027475130 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f4520024060 con 0x7f453006e120 2026-02-17T06:01:39.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.611+0000 7f4516ffd640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4500079170 0x7f450007b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:39.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.611+0000 7f4535ac9640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4500079170 0x7f450007b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:39.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.611+0000 7f4535ac9640 10 monclient: get_auth_request con 0x7f4500079170 auth_method 0 2026-02-17T06:01:39.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.611+0000 7f4516ffd640 10 client.5074.objecter ms_handle_connect 0x7f4500079170 2026-02-17T06:01:39.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.611+0000 7f4535ac9640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4500079170 0x7f450007b630 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f45301588e0 tx=0x7f4524007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:39.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.710+0000 7f4516ffd640 1 -- 10.20.193.16:0/3027475130 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f452005e480 con 0x7f453006e120 2026-02-17T06:01:39.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.710+0000 7f4516ffd640 10 client.5074.objecter ms_dispatch2 0x7f4530000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:39.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.710+0000 7f4516ffd640 3 client.5074.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:39.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.710+0000 7f4516ffd640 3 client.5074.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:39.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.710+0000 7f4516ffd640 20 client.5074.objecter dump_active .. 0 homeless 2026-02-17T06:01:39.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.716+0000 7f4516ffd640 1 -- 10.20.193.16:0/3027475130 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4520062440 con 0x7f453006e120 2026-02-17T06:01:39.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.716+0000 7f4516ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:39.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.716+0000 7f4516ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:39.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.815+0000 7f4538555640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:39.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.815+0000 7f4538555640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:39.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.815+0000 7f4538555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:39.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.815+0000 7f4538555640 1 -- 10.20.193.16:0/3027475130 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f453014b9f0 con 0x7f453006e120 2026-02-17T06:01:39.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.818+0000 7f4516ffd640 1 -- 10.20.193.16:0/3027475130 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f4520066740 con 0x7f453006e120 2026-02-17T06:01:39.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.818+0000 7f4516ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:39.818 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.818+0000 7f4516ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:39.819 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:39.819 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:39.820 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 10 librados: watch_flush enter 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 10 librados: watch_flush exit 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 20 client.5074.objecter shutdown clearing up homeless session... 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 10 client.5074.objecter successfully canceled tick 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 -- 10.20.193.16:0/3027475130 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4500079170 msgr2=0x7f450007b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4500079170 0x7f450007b630 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f45301588e0 tx=0x7f4524007ab0 comp rx=0 tx=0).stop 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 10 monclient: shutdown 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 -- 10.20.193.16:0/3027475130 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453006e120 msgr2=0x7f4530146750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453006e120 0x7f4530146750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f452000ebe0 tx=0x7f452000c520 comp rx=0 tx=0).stop 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 -- 10.20.193.16:0/3027475130 shutdown_connections 2026-02-17T06:01:39.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4500079170 0x7f450007b630 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:39.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f453015caf0 0x7f45301f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:39.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f453014f020 0x7f4530070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:39.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 --2- 10.20.193.16:0/3027475130 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f453006e120 0x7f4530146750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:39.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 -- 10.20.193.16:0/3027475130 >> 10.20.193.16:0/3027475130 conn(0x7f45300cd770 msgr2=0x7f453014d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:39.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 -- 10.20.193.16:0/3027475130 shutdown_connections 2026-02-17T06:01:39.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 -- 10.20.193.16:0/3027475130 wait complete. 2026-02-17T06:01:39.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:39.819+0000 7f4538555640 1 librados: shutdown 2026-02-17T06:01:42.829 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:42.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.895+0000 7f1aa8e87640 10 monclient: get_monmap_and_config 2026-02-17T06:01:42.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.895+0000 7f1aa8e87640 10 monclient: build_initial_monmap 2026-02-17T06:01:42.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.895+0000 7f1aa8e87640 10 monclient: monmap: 2026-02-17T06:01:42.896 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:42.896 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:42.896 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:42.896310+0000 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:42.896310+0000 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.895+0000 7f1aa8e87640 1 Processor -- start 2026-02-17T06:01:42.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.895+0000 7f1aa8e87640 1 -- start start 2026-02-17T06:01:42.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.895+0000 7f1aa8e87640 10 monclient: init 2026-02-17T06:01:42.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:42.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 10 monclient: _add_conns ranks=[0,4,3,2,5,1] 2026-02-17T06:01:42.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1aa40700f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:42.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1aa40c86f0 0x7f1aa40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:42.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 10 monclient(hunting): picked mon.noname-b con 0x7f1aa40c86f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:42.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1aa41536f0 0x7f1aa4153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:42.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 10 monclient(hunting): picked mon.noname-a con 0x7f1aa41536f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:42.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa40c4820 con 0x7f1aa40700f0 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1aa40c4b00 con 0x7f1aa41536f0 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f1aa40c4de0 con 0x7f1aa40c86f0 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa8e87640 10 monclient(hunting): authenticate will time out at 1355.465267s 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa2d76640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1aa41536f0 0x7f1aa4153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa1d74640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1aa40c86f0 0x7f1aa40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa1d74640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1aa40c86f0 0x7f1aa40704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:33784/0 (socket says 10.20.193.16:33784) 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa1d74640 1 -- 10.20.193.16:0/557095836 learned_addr learned my addr 10.20.193.16:0/557095836 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa1d74640 10 monclient(hunting): get_auth_request con 0x7f1aa40c86f0 auth_method 0 2026-02-17T06:01:42.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.896+0000 7f1aa1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.897+0000 7f1aa2d76640 10 monclient(hunting): get_auth_request con 0x7f1aa41536f0 auth_method 0 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.897+0000 7f1aa2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.897+0000 7f1aa2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.897+0000 7f1aa2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.897+0000 7f1aa1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.897+0000 7f1aa1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.897+0000 7f1aa1d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.897+0000 7f1aa2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.897+0000 7f1aa2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.897+0000 7f1aa2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:42.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.898+0000 7f1aa1d74640 10 monclient(hunting): handle_auth_done global_id 5077 payload 274 2026-02-17T06:01:42.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.898+0000 7f1aa1d74640 1 -- 10.20.193.16:0/557095836 >> v2:10.20.193.16:3300/0 conn(0x7f1aa41536f0 msgr2=0x7f1aa4153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:42.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.898+0000 7f1aa1d74640 1 --2- 10.20.193.16:0/557095836 >> v2:10.20.193.16:3300/0 conn(0x7f1aa41536f0 0x7f1aa4153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:42.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.898+0000 7f1aa1d74640 1 -- 10.20.193.16:0/557095836 >> v1:10.20.193.16:6789/0 conn(0x7f1aa40700f0 legacy=0x7f1aa40c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:42.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.898+0000 7f1aa1d74640 10 monclient: _finish_hunting 0 2026-02-17T06:01:42.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.898+0000 7f1aa1d74640 1 monclient: found mon.noname-b 2026-02-17T06:01:42.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.898+0000 7f1aa1d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:42.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.898+0000 7f1aa1d74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:42.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.898+0000 7f1aa1d74640 1 -- 10.20.193.16:0/557095836 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa40c50c0 con 0x7f1aa40c86f0 2026-02-17T06:01:42.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.899+0000 7f1aa1d74640 1 --2- 10.20.193.16:0/557095836 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 0x7f1aa40704d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40c5b80 tx=0x7f1a8c02ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c70c96e6fdb04c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:42.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 1 -- 10.20.193.16:0/557095836 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c03c070 con 0x7f1aa40c86f0 2026-02-17T06:01:42.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:42.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:42.910 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 10 monclient: dump: 2026-02-17T06:01:42.910 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:42.910 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 10 monclient: _finish_auth 0 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 10 monclient: _check_auth_tickets 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:42.911 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 1 -- 10.20.193.16:0/557095836 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c02fa90 con 0x7f1aa40c86f0 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 5 monclient: authenticate success, global_id 5077 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 1 -- 10.20.193.16:0/557095836 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c02fe60 con 0x7f1aa40c86f0 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 4 set_mon_vals no callback set 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa1573640 10 monclient: dump: 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:42.912 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 10 monclient: shutdown 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 1 -- 10.20.193.16:0/557095836 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 msgr2=0x7f1aa40704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 1 --2- 10.20.193.16:0/557095836 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 0x7f1aa40704d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40c5b80 tx=0x7f1a8c02ee60 comp rx=0 tx=0).stop 2026-02-17T06:01:42.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 1 -- 10.20.193.16:0/557095836 shutdown_connections 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 1 --2- 10.20.193.16:0/557095836 >> v2:10.20.193.16:3300/0 conn(0x7f1aa41536f0 0x7f1aa4153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 1 --2- 10.20.193.16:0/557095836 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 0x7f1aa40704d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 1 -- 10.20.193.16:0/557095836 >> 10.20.193.16:0/557095836 conn(0x7f1aa40cd770 msgr2=0x7f1aa40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 1 -- 10.20.193.16:0/557095836 shutdown_connections 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 1 -- 10.20.193.16:0/557095836 wait complete. 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 10 monclient: build_initial_monmap 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.901+0000 7f1aa8e87640 10 monclient: monmap: 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:42.914 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 librados: starting msgr at 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 librados: starting objecter 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 Processor -- start 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 -- start start 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 librados: setting wanted keys 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 librados: calling monclient init 2026-02-17T06:01:42.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient: init 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1aa40700f0 0x7f1aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient(hunting): picked mon.noname-a con 0x7f1aa40700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 0x7f1aa414f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient(hunting): picked mon.noname-b con 0x7f1aa40c86f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1aa41536f0 0x7f1aa406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient(hunting): picked mon.noname-c con 0x7f1aa41536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4154060 con 0x7f1aa40700f0 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa40c5680 con 0x7f1aa40c86f0 2026-02-17T06:01:42.916 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4074c00 con 0x7f1aa41536f0 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa8e87640 10 monclient(hunting): authenticate will time out at 1355.471587s 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa2575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1aa40700f0 0x7f1aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.902+0000 7f1aa1d74640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 0x7f1aa414f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa1d74640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 0x7f1aa414f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:33786/0 (socket says 10.20.193.16:33786) 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa1d74640 1 -- 10.20.193.16:0/3985099799 learned_addr learned my addr 10.20.193.16:0/3985099799 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa1d74640 10 monclient(hunting): get_auth_request con 0x7f1aa40c86f0 auth_method 0 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:42.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2575640 10 monclient(hunting): get_auth_request con 0x7f1aa40700f0 auth_method 0 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2d76640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1aa41536f0 0x7f1aa406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa1d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2d76640 10 monclient(hunting): get_auth_request con 0x7f1aa41536f0 auth_method 0 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:42.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.903+0000 7f1aa2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.904+0000 7f1aa1d74640 10 monclient(hunting): handle_auth_done global_id 5080 payload 824 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.904+0000 7f1aa1d74640 1 -- 10.20.193.16:0/3985099799 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1aa41536f0 msgr2=0x7f1aa406d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.904+0000 7f1aa1d74640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1aa41536f0 0x7f1aa406d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.904+0000 7f1aa1d74640 1 -- 10.20.193.16:0/3985099799 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1aa40700f0 msgr2=0x7f1aa406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.904+0000 7f1aa1d74640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1aa40700f0 0x7f1aa406edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.904+0000 7f1aa1d74640 10 monclient: _finish_hunting 0 2026-02-17T06:01:42.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.904+0000 7f1aa1d74640 1 monclient: found mon.noname-b 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.904+0000 7f1aa1d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.904+0000 7f1aa1d74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.904+0000 7f1aa1d74640 1 -- 10.20.193.16:0/3985099799 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa406f2f0 con 0x7f1aa40c86f0 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.905+0000 7f1aa1d74640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 0x7f1aa414f7e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009a70 tx=0x7f1a8c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.905+0000 7f1a8affd640 10 client.?.objecter ms_handle_connect 0x7f1aa40c86f0 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.905+0000 7f1a8affd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 1 -- 10.20.193.16:0/3985099799 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c03c040 con 0x7f1aa40c86f0 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 10 monclient: dump: 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:42.920 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 10 monclient: _finish_auth 0 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 1 -- 10.20.193.16:0/3985099799 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c042410 con 0x7f1aa40c86f0 2026-02-17T06:01:42.921 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 5 monclient: authenticate success, global_id 5080 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 1 -- 10.20.193.16:0/3985099799 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c042800 con 0x7f1aa40c86f0 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1a8affd640 10 monclient: dump: 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:42.922 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa0d72640 4 set_mon_vals no callback set 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 10 monclient: _renew_subs 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 1 -- 10.20.193.16:0/3985099799 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa406dd60 con 0x7f1aa40c86f0 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 10 client.5080.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 10 monclient: _renew_subs 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 1 -- 10.20.193.16:0/3985099799 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa406e320 con 0x7f1aa40c86f0 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 1 librados: init done 2026-02-17T06:01:42.923 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:42.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:42.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:42.924 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.907+0000 7f1aa8e87640 1 -- 10.20.193.16:0/3985099799 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa40704d0 con 0x7f1aa40c86f0 2026-02-17T06:01:42.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.919+0000 7f1a8affd640 1 -- 10.20.193.16:0/3985099799 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1a8c0429a0 con 0x7f1aa40c86f0 2026-02-17T06:01:42.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.919+0000 7f1a8affd640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1a70079170 0x7f1a7007b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:42.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.919+0000 7f1aa2575640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1a70079170 0x7f1a7007b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:42.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.919+0000 7f1aa2575640 10 monclient: get_auth_request con 0x7f1a70079170 auth_method 0 2026-02-17T06:01:42.925 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.919+0000 7f1a8affd640 10 client.5080.objecter ms_handle_connect 0x7f1a70079170 2026-02-17T06:01:42.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.919+0000 7f1aa2575640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1a70079170 0x7f1a7007b630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1a98004660 tx=0x7f1a98009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:42.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.920+0000 7f1a8affd640 1 -- 10.20.193.16:0/3985099799 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1a8c083950 con 0x7f1aa40c86f0 2026-02-17T06:01:42.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.920+0000 7f1a8affd640 10 client.5080.objecter ms_dispatch2 0x7f1aa4000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:42.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.920+0000 7f1a8affd640 3 client.5080.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:42.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.920+0000 7f1a8affd640 3 client.5080.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:42.926 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:42.921+0000 7f1a8affd640 20 client.5080.objecter dump_active .. 0 homeless 2026-02-17T06:01:43.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.024+0000 7f1a8affd640 1 -- 10.20.193.16:0/3985099799 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1a8c087910 con 0x7f1aa40c86f0 2026-02-17T06:01:43.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.024+0000 7f1a8affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:43.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.024+0000 7f1a8affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:43.126 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.126+0000 7f1aa8e87640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:43.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.126+0000 7f1aa8e87640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:43.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.126+0000 7f1aa8e87640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:43.127 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.126+0000 7f1aa8e87640 1 -- 10.20.193.16:0/3985099799 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1aa4151b00 con 0x7f1aa40c86f0 2026-02-17T06:01:43.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.128+0000 7f1a8affd640 1 -- 10.20.193.16:0/3985099799 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1a8c08bc10 con 0x7f1aa40c86f0 2026-02-17T06:01:43.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.128+0000 7f1a8affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:43.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.128+0000 7f1a8affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:43.129 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:43.129 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:43.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 10 librados: watch_flush enter 2026-02-17T06:01:43.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 10 librados: watch_flush exit 2026-02-17T06:01:43.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 20 client.5080.objecter shutdown clearing up homeless session... 2026-02-17T06:01:43.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 10 client.5080.objecter successfully canceled tick 2026-02-17T06:01:43.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 -- 10.20.193.16:0/3985099799 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1a70079170 msgr2=0x7f1a7007b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:43.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1a70079170 0x7f1a7007b630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1a98004660 tx=0x7f1a98009110 comp rx=0 tx=0).stop 2026-02-17T06:01:43.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 10 monclient: shutdown 2026-02-17T06:01:43.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:43.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 -- 10.20.193.16:0/3985099799 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 msgr2=0x7f1aa414f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:43.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 0x7f1aa414f7e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009a70 tx=0x7f1a8c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:43.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 -- 10.20.193.16:0/3985099799 shutdown_connections 2026-02-17T06:01:43.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1a70079170 0x7f1a7007b630 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:43.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1aa40c86f0 0x7f1aa414f7e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:43.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1aa41536f0 0x7f1aa406d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:43.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 --2- 10.20.193.16:0/3985099799 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1aa40700f0 0x7f1aa406edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:43.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 -- 10.20.193.16:0/3985099799 >> 10.20.193.16:0/3985099799 conn(0x7f1aa40cd770 msgr2=0x7f1aa4151180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:43.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 -- 10.20.193.16:0/3985099799 shutdown_connections 2026-02-17T06:01:43.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 -- 10.20.193.16:0/3985099799 wait complete. 2026-02-17T06:01:43.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:43.129+0000 7f1aa8e87640 1 librados: shutdown 2026-02-17T06:01:46.140 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.208+0000 7ff42db23640 10 monclient: get_monmap_and_config 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.208+0000 7ff42db23640 10 monclient: build_initial_monmap 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.208+0000 7ff42db23640 10 monclient: monmap: 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:46.209210+0000 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:46.209210+0000 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:46.208 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.208+0000 7ff42db23640 1 Processor -- start 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.208+0000 7ff42db23640 1 -- start start 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.208+0000 7ff42db23640 10 monclient: init 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 10 monclient: _add_conns ranks=[5,3,4,2,1,0] 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7ff4280700f0 0x7ff4280704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 10 monclient(hunting): picked mon.noname-c con 0x7ff4280700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7ff4280c7fe0 0x7ff4280c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 10 monclient(hunting): picked mon.noname-a con 0x7ff4280c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7ff4281536f0 0x7ff428153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 10 monclient(hunting): picked mon.noname-b con 0x7ff4281536f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff4280c8970 con 0x7ff4280c7fe0 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7ff4280c4820 con 0x7ff4281536f0 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7ff4280c4b00 con 0x7ff4280700f0 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:46.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff42db23640 10 monclient(hunting): authenticate will time out at 1358.778250s 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff426ffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7ff4280c7fe0 0x7ff4280c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff427fff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7ff4281536f0 0x7ff428153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff426ffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7ff4280c7fe0 0x7ff4280c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:53678/0 (socket says 10.20.193.16:53678) 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff426ffd640 1 -- 10.20.193.16:0/12219087 learned_addr learned my addr 10.20.193.16:0/12219087 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff426ffd640 10 monclient(hunting): get_auth_request con 0x7ff4280c7fe0 auth_method 0 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff4277fe640 1 --2- 10.20.193.16:0/12219087 >> v2:10.20.193.148:3300/0 conn(0x7ff4280700f0 0x7ff4280704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff426ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff426ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff426ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff427fff640 10 monclient(hunting): get_auth_request con 0x7ff4281536f0 auth_method 0 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff427fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff427fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff427fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff4277fe640 10 monclient(hunting): get_auth_request con 0x7ff4280700f0 auth_method 0 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff4277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff4277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.209+0000 7ff4277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.210+0000 7ff426ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.210+0000 7ff426ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:46.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.210+0000 7ff426ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.210+0000 7ff427fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.210+0000 7ff427fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.210+0000 7ff427fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.210+0000 7ff4277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.210+0000 7ff4277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.210+0000 7ff4277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 10 monclient(hunting): handle_auth_done global_id 5016 payload 274 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 1 -- 10.20.193.16:0/12219087 >> v2:10.20.193.148:3300/0 conn(0x7ff4280700f0 msgr2=0x7ff4280704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 1 --2- 10.20.193.16:0/12219087 >> v2:10.20.193.148:3300/0 conn(0x7ff4280700f0 0x7ff4280704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 1 -- 10.20.193.16:0/12219087 >> v2:10.20.193.128:3300/0 conn(0x7ff4281536f0 msgr2=0x7ff428153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 1 --2- 10.20.193.16:0/12219087 >> v2:10.20.193.128:3300/0 conn(0x7ff4281536f0 0x7ff428153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 1 monclient: found mon.noname-a 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:46.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 1 -- 10.20.193.16:0/12219087 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4280c4de0 con 0x7ff4280c7fe0 2026-02-17T06:01:46.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.211+0000 7ff426ffd640 1 --2- 10.20.193.16:0/12219087 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4280c7fe0 0x7ff4280c8430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff4280c9b90 tx=0x7ff41802edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add9e775c270c7fb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 1 -- 10.20.193.16:0/12219087 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41803c070 con 0x7ff4280c7fe0 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 10 monclient: dump: 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 1 -- 10.20.193.16:0/12219087 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41802fab0 con 0x7ff4280c7fe0 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 1 -- 10.20.193.16:0/12219087 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418037440 con 0x7ff4280c7fe0 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff42db23640 5 monclient: authenticate success, global_id 5016 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff42db23640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff42db23640 4 set_mon_vals no callback set 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.213+0000 7ff4267fc640 10 monclient: dump: 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:46.214 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 10 monclient: shutdown 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 -- 10.20.193.16:0/12219087 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4280c7fe0 msgr2=0x7ff4280c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 --2- 10.20.193.16:0/12219087 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4280c7fe0 0x7ff4280c8430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff4280c9b90 tx=0x7ff41802edf0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 -- 10.20.193.16:0/12219087 shutdown_connections 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 --2- 10.20.193.16:0/12219087 >> v2:10.20.193.128:3300/0 conn(0x7ff4281536f0 0x7ff428153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 --2- 10.20.193.16:0/12219087 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4280c7fe0 0x7ff4280c8430 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 --2- 10.20.193.16:0/12219087 >> v2:10.20.193.148:3300/0 conn(0x7ff4280700f0 0x7ff4280704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 -- 10.20.193.16:0/12219087 >> 10.20.193.16:0/12219087 conn(0x7ff4280cd770 msgr2=0x7ff4280cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 -- 10.20.193.16:0/12219087 shutdown_connections 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 -- 10.20.193.16:0/12219087 wait complete. 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 10 monclient: build_initial_monmap 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 10 monclient: monmap: 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 librados: starting msgr at 2026-02-17T06:01:46.215 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.214+0000 7ff42db23640 1 librados: starting objecter 2026-02-17T06:01:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 1 Processor -- start 2026-02-17T06:01:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 1 -- start start 2026-02-17T06:01:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 1 librados: setting wanted keys 2026-02-17T06:01:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 1 librados: calling monclient init 2026-02-17T06:01:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient: init 2026-02-17T06:01:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4280700f0 0x7ff428150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient(hunting): picked mon.noname-b con 0x7ff4280700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:46.216 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff4280c7fe0 0x7ff4281508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient(hunting): picked mon.noname-c con 0x7ff4280c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4281536f0 0x7ff428144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient(hunting): picked mon.noname-a con 0x7ff4281536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280c5680 con 0x7ff4281536f0 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280c53a0 con 0x7ff4280700f0 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff428154040 con 0x7ff4280c7fe0 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff42db23640 10 monclient(hunting): authenticate will time out at 1358.784473s 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff427fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4281536f0 0x7ff428144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff427fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4281536f0 0x7ff428144740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:53682/0 (socket says 10.20.193.16:53682) 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff427fff640 1 -- 10.20.193.16:0/1413803385 learned_addr learned my addr 10.20.193.16:0/1413803385 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff427fff640 10 monclient(hunting): get_auth_request con 0x7ff4281536f0 auth_method 0 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff4277fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4280700f0 0x7ff428150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff427fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff427fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff427fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.215+0000 7ff426ffd640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff4280c7fe0 0x7ff4281508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff4277fe640 10 monclient(hunting): get_auth_request con 0x7ff4280700f0 auth_method 0 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff4277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff4277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff4277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff427fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff427fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff427fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff426ffd640 10 monclient(hunting): get_auth_request con 0x7ff4280c7fe0 auth_method 0 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff426ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff426ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff426ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff4277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff4277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff4277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff426ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff426ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:46.217 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.216+0000 7ff426ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.217+0000 7ff427fff640 10 monclient(hunting): handle_auth_done global_id 5019 payload 824 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.217+0000 7ff427fff640 1 -- 10.20.193.16:0/1413803385 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff4280c7fe0 msgr2=0x7ff4281508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.217+0000 7ff427fff640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff4280c7fe0 0x7ff4281508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.217+0000 7ff427fff640 1 -- 10.20.193.16:0/1413803385 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4280700f0 msgr2=0x7ff428150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.217+0000 7ff427fff640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4280700f0 0x7ff428150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.217+0000 7ff427fff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.217+0000 7ff427fff640 1 monclient: found mon.noname-a 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.217+0000 7ff427fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.217+0000 7ff427fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.217+0000 7ff427fff640 1 -- 10.20.193.16:0/1413803385 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42806e1e0 con 0x7ff4281536f0 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.218+0000 7ff427fff640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4281536f0 0x7ff428144740 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00efc0 tx=0x7ff41c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.218+0000 7ff403fff640 10 client.?.objecter ms_handle_connect 0x7ff4281536f0 2026-02-17T06:01:46.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.218+0000 7ff403fff640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 1 -- 10.20.193.16:0/1413803385 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c018020 con 0x7ff4281536f0 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 10 monclient: dump: 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 10 monclient: _finish_auth 0 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 1 -- 10.20.193.16:0/1413803385 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41c014480 con 0x7ff4281536f0 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 5 monclient: authenticate success, global_id 5019 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 10 monclient: _renew_subs 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:46.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 1 -- 10.20.193.16:0/1413803385 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42806d380 con 0x7ff4281536f0 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 1 -- 10.20.193.16:0/1413803385 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c014870 con 0x7ff4281536f0 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff425ffb640 4 set_mon_vals no callback set 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff403fff640 10 monclient: dump: 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 10 client.5019.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 10 monclient: _renew_subs 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 1 -- 10.20.193.16:0/1413803385 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42806d940 con 0x7ff4281536f0 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 1 librados: init done 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:46.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.220+0000 7ff42db23640 1 -- 10.20.193.16:0/1413803385 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff428074c00 con 0x7ff4281536f0 2026-02-17T06:01:46.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.232+0000 7ff403fff640 1 -- 10.20.193.16:0/1413803385 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7ff41c014a10 con 0x7ff4281536f0 2026-02-17T06:01:46.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.233+0000 7ff403fff640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff3f80791c0 0x7ff3f807b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:46.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.233+0000 7ff4277fe640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff3f80791c0 0x7ff3f807b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:46.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.233+0000 7ff4277fe640 10 monclient: get_auth_request con 0x7ff3f80791c0 auth_method 0 2026-02-17T06:01:46.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.233+0000 7ff4277fe640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff3f80791c0 0x7ff3f807b680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ba0 tx=0x7ff414009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:46.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.233+0000 7ff403fff640 10 client.5019.objecter ms_handle_connect 0x7ff3f80791c0 2026-02-17T06:01:46.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.337+0000 7ff403fff640 1 -- 10.20.193.16:0/1413803385 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7ff41c05ef00 con 0x7ff4281536f0 2026-02-17T06:01:46.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.337+0000 7ff403fff640 10 client.5019.objecter ms_dispatch2 0x7ff428000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:46.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.337+0000 7ff403fff640 3 client.5019.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:46.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.337+0000 7ff403fff640 3 client.5019.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:46.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.337+0000 7ff403fff640 20 client.5019.objecter dump_active .. 0 homeless 2026-02-17T06:01:46.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.337+0000 7ff403fff640 1 -- 10.20.193.16:0/1413803385 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff41c022070 con 0x7ff4281536f0 2026-02-17T06:01:46.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.337+0000 7ff403fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:46.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.337+0000 7ff403fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:46.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.437+0000 7ff42db23640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:46.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.437+0000 7ff42db23640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:46.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.437+0000 7ff42db23640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:46.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.437+0000 7ff42db23640 1 -- 10.20.193.16:0/1413803385 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff42815bfb0 con 0x7ff4281536f0 2026-02-17T06:01:46.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.439+0000 7ff403fff640 1 -- 10.20.193.16:0/1413803385 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff41c010040 con 0x7ff4281536f0 2026-02-17T06:01:46.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.439+0000 7ff403fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:46.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.439+0000 7ff403fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:46.441 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:46.441 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:46.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 10 librados: watch_flush enter 2026-02-17T06:01:46.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 10 librados: watch_flush exit 2026-02-17T06:01:46.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 20 client.5019.objecter shutdown clearing up homeless session... 2026-02-17T06:01:46.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 10 client.5019.objecter successfully canceled tick 2026-02-17T06:01:46.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 -- 10.20.193.16:0/1413803385 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff3f80791c0 msgr2=0x7ff3f807b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:46.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff3f80791c0 0x7ff3f807b680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ba0 tx=0x7ff414009290 comp rx=0 tx=0).stop 2026-02-17T06:01:46.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 10 monclient: shutdown 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 -- 10.20.193.16:0/1413803385 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4281536f0 msgr2=0x7ff428144740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4281536f0 0x7ff428144740 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00efc0 tx=0x7ff41c00c490 comp rx=0 tx=0).stop 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 -- 10.20.193.16:0/1413803385 shutdown_connections 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff3f80791c0 0x7ff3f807b680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff4281536f0 0x7ff428144740 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff4280c7fe0 0x7ff4281508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 --2- 10.20.193.16:0/1413803385 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff4280700f0 0x7ff428150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 -- 10.20.193.16:0/1413803385 >> 10.20.193.16:0/1413803385 conn(0x7ff4280cd770 msgr2=0x7ff42814e640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 -- 10.20.193.16:0/1413803385 shutdown_connections 2026-02-17T06:01:46.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 -- 10.20.193.16:0/1413803385 wait complete. 2026-02-17T06:01:46.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:46.441+0000 7ff42db23640 1 librados: shutdown 2026-02-17T06:01:49.254 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:07:28.081 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:49.281 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:07:28.108 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:49.281 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:07:28.107 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:49.283 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:07:28.109 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:49.289 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:07:28.117 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:49.292 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:07:28.118 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:49.308 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:07:28.097 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:49.310 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:07:28.099 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:49.318 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:07:28.107 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:01:49.450 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:49.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.518+0000 7f0067891640 10 monclient: get_monmap_and_config 2026-02-17T06:01:49.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.518+0000 7f0067891640 10 monclient: build_initial_monmap 2026-02-17T06:01:49.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.518+0000 7f0067891640 10 monclient: monmap: 2026-02-17T06:01:49.519 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:49.519 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:49.519 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:49.519298+0000 2026-02-17T06:01:49.519 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:49.519298+0000 2026-02-17T06:01:49.519 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:49.519 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:49.520 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:49.520 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:49.520 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:49.520 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:49.520 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:49.520 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:49.520 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:49.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.518+0000 7f0067891640 1 Processor -- start 2026-02-17T06:01:49.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.518+0000 7f0067891640 1 -- start start 2026-02-17T06:01:49.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.518+0000 7f0067891640 10 monclient: init 2026-02-17T06:01:49.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:49.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 10 monclient: _add_conns ranks=[4,5,2,1,3,0] 2026-02-17T06:01:49.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f00600700f0 0x7f00600704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:49.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 10 monclient(hunting): picked mon.noname-b con 0x7f00600700f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:49.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f00600c7fe0 0x7f00600c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 10 monclient(hunting): picked mon.noname-c con 0x7f00600c7fe0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f00601536f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00600c8970 con 0x7f00601536f0 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f00600c4820 con 0x7f00600700f0 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f00600c4b00 con 0x7f00600c7fe0 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0067891640 10 monclient(hunting): authenticate will time out at 1362.088346s 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0065606640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f00600700f0 0x7f00600704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:49.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.519+0000 7f0064e05640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f00600c7fe0 0x7f00600c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0065606640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f00600700f0 0x7f00600704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:33814/0 (socket says 10.20.193.16:33814) 2026-02-17T06:01:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0065606640 1 -- 10.20.193.16:0/2511030663 learned_addr learned my addr 10.20.193.16:0/2511030663 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0065606640 10 monclient(hunting): get_auth_request con 0x7f00600700f0 auth_method 0 2026-02-17T06:01:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0065606640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0065606640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0065606640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0064e05640 10 monclient(hunting): get_auth_request con 0x7f00600c7fe0 auth_method 0 2026-02-17T06:01:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0064e05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0064e05640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:49.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0064e05640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0065606640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0065606640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0065606640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0064e05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0064e05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.520+0000 7f0064e05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.521+0000 7f0064e05640 10 monclient(hunting): handle_auth_done global_id 5150 payload 274 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.521+0000 7f0064e05640 1 -- 10.20.193.16:0/2511030663 >> v2:10.20.193.128:3300/0 conn(0x7f00600700f0 msgr2=0x7f00600704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.521+0000 7f0064e05640 1 --2- 10.20.193.16:0/2511030663 >> v2:10.20.193.128:3300/0 conn(0x7f00600700f0 0x7f00600704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.521+0000 7f0064e05640 1 -- 10.20.193.16:0/2511030663 >> v1:10.20.193.148:6789/0 conn(0x7f00601536f0 legacy=0x7f0060153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.521+0000 7f0064e05640 10 monclient: _finish_hunting 0 2026-02-17T06:01:49.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.521+0000 7f0064e05640 1 monclient: found mon.noname-c 2026-02-17T06:01:49.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.521+0000 7f0064e05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:49.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.521+0000 7f0064e05640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:49.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.521+0000 7f0064e05640 1 -- 10.20.193.16:0/2511030663 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00600c4de0 con 0x7f00600c7fe0 2026-02-17T06:01:49.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.523+0000 7f0064e05640 1 --2- 10.20.193.16:0/2511030663 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f00600c7fe0 0x7f00600c8430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f00600c5b80 tx=0x7f005402ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae1af5f01fd1b58 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:49.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 1 -- 10.20.193.16:0/2511030663 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005403c070 con 0x7f00600c7fe0 2026-02-17T06:01:49.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:49.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:49.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 10 monclient: dump: 2026-02-17T06:01:49.536 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:49.536 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:49.536 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:49.536 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:49.536 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:49.536 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:49.536 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 10 monclient: _finish_auth 0 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 1 -- 10.20.193.16:0/2511030663 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f005402f9c0 con 0x7f00600c7fe0 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 1 -- 10.20.193.16:0/2511030663 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005402fd90 con 0x7f00600c7fe0 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f0067891640 5 monclient: authenticate success, global_id 5150 2026-02-17T06:01:49.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f004ffff640 10 monclient: dump: 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:49.538 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f0067891640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.524+0000 7f0067891640 4 set_mon_vals no callback set 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 10 monclient: shutdown 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 1 -- 10.20.193.16:0/2511030663 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f00600c7fe0 msgr2=0x7f00600c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 1 --2- 10.20.193.16:0/2511030663 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f00600c7fe0 0x7f00600c8430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f00600c5b80 tx=0x7f005402ed50 comp rx=0 tx=0).stop 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 1 -- 10.20.193.16:0/2511030663 shutdown_connections 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 1 --2- 10.20.193.16:0/2511030663 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f00600c7fe0 0x7f00600c8430 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 1 --2- 10.20.193.16:0/2511030663 >> v2:10.20.193.128:3300/0 conn(0x7f00600700f0 0x7f00600704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 1 -- 10.20.193.16:0/2511030663 >> 10.20.193.16:0/2511030663 conn(0x7f00600cd770 msgr2=0x7f00600cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 1 -- 10.20.193.16:0/2511030663 shutdown_connections 2026-02-17T06:01:49.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 1 -- 10.20.193.16:0/2511030663 wait complete. 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 10 monclient: build_initial_monmap 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.525+0000 7f0067891640 10 monclient: monmap: 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:49.540 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 librados: starting msgr at 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 librados: starting objecter 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 Processor -- start 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 -- start start 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 librados: setting wanted keys 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 librados: calling monclient init 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient: init 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f00600700f0 0x7f006006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient(hunting): picked mon.noname-c con 0x7f00600700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:49.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f00600c7fe0 0x7f00601f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient(hunting): picked mon.noname-a con 0x7f00600c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f00601536f0 0x7f0060145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient(hunting): picked mon.noname-b con 0x7f00601536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00600c5680 con 0x7f00600c7fe0 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f00600c53a0 con 0x7f00601536f0 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0060074c00 con 0x7f00600700f0 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.526+0000 7f0067891640 10 monclient(hunting): authenticate will time out at 1362.095648s 2026-02-17T06:01:49.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065606640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f00600700f0 0x7f006006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065e07640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f00601536f0 0x7f0060145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0064e05640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f00600c7fe0 0x7f00601f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065e07640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f00601536f0 0x7f0060145910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:33822/0 (socket says 10.20.193.16:33822) 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065e07640 1 -- 10.20.193.16:0/3439559451 learned_addr learned my addr 10.20.193.16:0/3439559451 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065e07640 10 monclient(hunting): get_auth_request con 0x7f00601536f0 auth_method 0 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065e07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065e07640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065e07640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0064e05640 10 monclient(hunting): get_auth_request con 0x7f00600c7fe0 auth_method 0 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0064e05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0064e05640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:49.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0064e05640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065606640 10 monclient(hunting): get_auth_request con 0x7f00600700f0 auth_method 0 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065606640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065606640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065606640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065e07640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065e07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065e07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0064e05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0064e05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0064e05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065606640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065606640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:49.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.527+0000 7f0065606640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.528+0000 7f0065e07640 10 monclient(hunting): handle_auth_done global_id 5092 payload 824 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.529+0000 7f0065e07640 1 -- 10.20.193.16:0/3439559451 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f00600700f0 msgr2=0x7f006006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.529+0000 7f0065e07640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f00600700f0 0x7f006006d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.529+0000 7f0065e07640 1 -- 10.20.193.16:0/3439559451 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f00600c7fe0 msgr2=0x7f00601f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.529+0000 7f0065e07640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f00600c7fe0 0x7f00601f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.529+0000 7f0065e07640 10 monclient: _finish_hunting 0 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.529+0000 7f0065e07640 1 monclient: found mon.noname-b 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.529+0000 7f0065e07640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.529+0000 7f0065e07640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.529+0000 7f0065e07640 1 -- 10.20.193.16:0/3439559451 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00601f7e20 con 0x7f00601536f0 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.529+0000 7f0065606640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f00600700f0 0x7f006006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:49.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.530+0000 7f0065e07640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f00601536f0 0x7f0060145910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f005c00b990 tx=0x7f005c00be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.530+0000 7f004dffb640 10 client.?.objecter ms_handle_connect 0x7f00601536f0 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.530+0000 7f004dffb640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 1 -- 10.20.193.16:0/3439559451 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c00c790 con 0x7f00601536f0 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 10 monclient: dump: 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:49.546 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 1 -- 10.20.193.16:0/3439559451 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f005c010070 con 0x7f00601536f0 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f0067891640 5 monclient: authenticate success, global_id 5092 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f0067891640 10 monclient: _renew_subs 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f0067891640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:49.547 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f0067891640 1 -- 10.20.193.16:0/3439559451 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00601f3b50 con 0x7f00601536f0 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 1 -- 10.20.193.16:0/3439559451 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c00cbb0 con 0x7f00601536f0 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.531+0000 7f004dffb640 10 monclient: dump: 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:49.548 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.532+0000 7f004f7fe640 4 set_mon_vals no callback set 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.532+0000 7f0067891640 10 client.5092.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.532+0000 7f0067891640 10 monclient: _renew_subs 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.532+0000 7f0067891640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.532+0000 7f0067891640 1 -- 10.20.193.16:0/3439559451 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00601f7b40 con 0x7f00601536f0 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.532+0000 7f0067891640 1 librados: init done 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.532+0000 7f0067891640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.532+0000 7f0067891640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.532+0000 7f0067891640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:49.549 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.532+0000 7f0067891640 1 -- 10.20.193.16:0/3439559451 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00600704d0 con 0x7f00601536f0 2026-02-17T06:01:49.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.544+0000 7f004dffb640 1 -- 10.20.193.16:0/3439559451 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f005c004070 con 0x7f00601536f0 2026-02-17T06:01:49.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.544+0000 7f004dffb640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0030079140 0x7f003007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:49.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.544+0000 7f0065606640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0030079140 0x7f003007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:49.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.544+0000 7f0065606640 10 monclient: get_auth_request con 0x7f0030079140 auth_method 0 2026-02-17T06:01:49.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.544+0000 7f0065606640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0030079140 0x7f003007b600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0050006dc0 tx=0x7f0050009340 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:49.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.544+0000 7f004dffb640 10 client.5092.objecter ms_handle_connect 0x7f0030079140 2026-02-17T06:01:49.551 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.546+0000 7f004dffb640 1 -- 10.20.193.16:0/3439559451 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f005c00cd50 con 0x7f00601536f0 2026-02-17T06:01:49.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.546+0000 7f004dffb640 10 client.5092.objecter ms_dispatch2 0x7f0060000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:49.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.546+0000 7f004dffb640 3 client.5092.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:49.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.546+0000 7f004dffb640 3 client.5092.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:49.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.546+0000 7f004dffb640 20 client.5092.objecter dump_active .. 0 homeless 2026-02-17T06:01:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.649+0000 7f004dffb640 1 -- 10.20.193.16:0/3439559451 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f005c061de0 con 0x7f00601536f0 2026-02-17T06:01:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.649+0000 7f004dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:49.650 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.649+0000 7f004dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:49.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.749+0000 7f0067891640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:49.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.749+0000 7f0067891640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:49.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.749+0000 7f0067891640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:49.750 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.749+0000 7f0067891640 1 -- 10.20.193.16:0/3439559451 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f00601573f0 con 0x7f00601536f0 2026-02-17T06:01:49.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.752+0000 7f004dffb640 1 -- 10.20.193.16:0/3439559451 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f005c0660e0 con 0x7f00601536f0 2026-02-17T06:01:49.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.752+0000 7f004dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:49.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.752+0000 7f004dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:49.753 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:49.753 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:49.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 10 librados: watch_flush enter 2026-02-17T06:01:49.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 10 librados: watch_flush exit 2026-02-17T06:01:49.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 20 client.5092.objecter shutdown clearing up homeless session... 2026-02-17T06:01:49.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 10 client.5092.objecter successfully canceled tick 2026-02-17T06:01:49.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 -- 10.20.193.16:0/3439559451 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0030079140 msgr2=0x7f003007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:49.755 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0030079140 0x7f003007b600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0050006dc0 tx=0x7f0050009340 comp rx=0 tx=0).stop 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 10 monclient: shutdown 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 -- 10.20.193.16:0/3439559451 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f00601536f0 msgr2=0x7f0060145910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f00601536f0 0x7f0060145910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f005c00b990 tx=0x7f005c00be60 comp rx=0 tx=0).stop 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 -- 10.20.193.16:0/3439559451 shutdown_connections 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0030079140 0x7f003007b600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f00601536f0 0x7f0060145910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f00600c7fe0 0x7f00601f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 --2- 10.20.193.16:0/3439559451 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f00600700f0 0x7f006006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 -- 10.20.193.16:0/3439559451 >> 10.20.193.16:0/3439559451 conn(0x7f00600cd770 msgr2=0x7f006014ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:49.756 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 -- 10.20.193.16:0/3439559451 shutdown_connections 2026-02-17T06:01:49.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.753+0000 7f0067891640 1 -- 10.20.193.16:0/3439559451 wait complete. 2026-02-17T06:01:49.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:49.754+0000 7f0067891640 1 librados: shutdown 2026-02-17T06:01:52.763 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T06:01:52.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.831+0000 7f9fd0951640 10 monclient: get_monmap_and_config 2026-02-17T06:01:52.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.831+0000 7f9fd0951640 10 monclient: build_initial_monmap 2026-02-17T06:01:52.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.831+0000 7f9fd0951640 10 monclient: monmap: 2026-02-17T06:01:52.831 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:52.831 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:52.831 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:52.832222+0000 2026-02-17T06:01:52.831 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:52.832222+0000 2026-02-17T06:01:52.831 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:52.831 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:52.831 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.831+0000 7f9fd0951640 1 Processor -- start 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.831+0000 7f9fd0951640 1 -- start start 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.831+0000 7f9fd0951640 10 monclient: init 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 10 monclient: _add_conns ranks=[2,5,4,1,3,0] 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9fc80700f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f9fc80c86f0 0x7f9fc80704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 10 monclient(hunting): picked mon.noname-c con 0x7f9fc80c86f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f9fc81536f0 0x7f9fc8153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 10 monclient(hunting): picked mon.noname-b con 0x7f9fc81536f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc80c4820 con 0x7f9fc80700f0 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc80c4b00 con 0x7f9fc81536f0 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc80c4de0 con 0x7f9fc80c86f0 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:52.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fd0951640 10 monclient(hunting): authenticate will time out at 1365.401185s 2026-02-17T06:01:52.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fceec7640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f9fc81536f0 0x7f9fc8153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:52.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fce6c6640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f9fc80700f0 0x7f9fc80c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:51256/0 (socket says 10.20.193.16:51256) 2026-02-17T06:01:52.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fce6c6640 1 -- 10.20.193.16:0/2042461962 learned_addr learned my addr 10.20.193.16:0/2042461962 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:52.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fceec7640 10 monclient(hunting): get_auth_request con 0x7f9fc81536f0 auth_method 0 2026-02-17T06:01:52.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fceec7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:52.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fceec7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:52.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.832+0000 7f9fceec7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:52.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.833+0000 7f9fceec7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:52.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.833+0000 7f9fceec7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:52.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.833+0000 7f9fceec7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:52.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.834+0000 7f9fceec7640 10 monclient(hunting): handle_auth_done global_id 5095 payload 274 2026-02-17T06:01:52.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.834+0000 7f9fceec7640 1 -- 10.20.193.16:0/2042461962 >> v2:10.20.193.148:3300/0 conn(0x7f9fc80c86f0 msgr2=0x7f9fc80704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:52.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.834+0000 7f9fceec7640 1 --2- 10.20.193.16:0/2042461962 >> v2:10.20.193.148:3300/0 conn(0x7f9fc80c86f0 0x7f9fc80704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:52.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.834+0000 7f9fceec7640 1 -- 10.20.193.16:0/2042461962 >> v1:10.20.193.148:6789/0 conn(0x7f9fc80700f0 legacy=0x7f9fc80c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:52.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.834+0000 7f9fceec7640 10 monclient: _finish_hunting 0 2026-02-17T06:01:52.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.834+0000 7f9fceec7640 1 monclient: found mon.noname-b 2026-02-17T06:01:52.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.834+0000 7f9fceec7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:52.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.834+0000 7f9fceec7640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:52.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.834+0000 7f9fceec7640 1 -- 10.20.193.16:0/2042461962 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc80c50c0 con 0x7f9fc81536f0 2026-02-17T06:01:52.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.835+0000 7f9fceec7640 1 --2- 10.20.193.16:0/2042461962 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 0x7f9fc8153b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80c5b80 tx=0x7f9fbc02f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc6c94dda793c2b4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:52.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 1 -- 10.20.193.16:0/2042461962 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc03c070 con 0x7f9fc81536f0 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 10 monclient: dump: 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 10 monclient: _finish_auth 0 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 10 monclient: _check_auth_tickets 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 5 monclient: authenticate success, global_id 5095 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 1 -- 10.20.193.16:0/2042461962 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fbc02fe40 con 0x7f9fc81536f0 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 1 -- 10.20.193.16:0/2042461962 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc037660 con 0x7f9fc81536f0 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 4 set_mon_vals no callback set 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fcd6c4640 10 monclient: dump: 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:52.838 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:52.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 10 monclient: shutdown 2026-02-17T06:01:52.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:52.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 1 -- 10.20.193.16:0/2042461962 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 msgr2=0x7f9fc8153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:52.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 1 --2- 10.20.193.16:0/2042461962 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 0x7f9fc8153b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80c5b80 tx=0x7f9fbc02f130 comp rx=0 tx=0).stop 2026-02-17T06:01:52.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 1 -- 10.20.193.16:0/2042461962 shutdown_connections 2026-02-17T06:01:52.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 1 --2- 10.20.193.16:0/2042461962 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 0x7f9fc8153b20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:52.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 1 --2- 10.20.193.16:0/2042461962 >> v2:10.20.193.148:3300/0 conn(0x7f9fc80c86f0 0x7f9fc80704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:52.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 1 -- 10.20.193.16:0/2042461962 >> 10.20.193.16:0/2042461962 conn(0x7f9fc80cd770 msgr2=0x7f9fc80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:52.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 1 -- 10.20.193.16:0/2042461962 shutdown_connections 2026-02-17T06:01:52.839 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.837+0000 7f9fd0951640 1 -- 10.20.193.16:0/2042461962 wait complete. 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient: build_initial_monmap 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient: monmap: 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 librados: starting msgr at 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 librados: starting objecter 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 Processor -- start 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 -- start start 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 librados: setting wanted keys 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 librados: calling monclient init 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient: init 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9fc80700f0 0x7f9fc80cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient(hunting): picked mon.noname-a con 0x7f9fc80700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9fc80c86f0 0x7f9fc8151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient(hunting): picked mon.noname-c con 0x7f9fc80c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:52.840 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 0x7f9fc8151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient(hunting): picked mon.noname-b con 0x7f9fc81536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8154060 con 0x7f9fc80700f0 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc80c5680 con 0x7f9fc81536f0 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8074c00 con 0x7f9fc80c86f0 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.838+0000 7f9fd0951640 10 monclient(hunting): authenticate will time out at 1365.407814s 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fceec7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 0x7f9fc8151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fce6c6640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9fc80700f0 0x7f9fc80cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fcdec5640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9fc80c86f0 0x7f9fc8151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fceec7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 0x7f9fc8151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43182/0 (socket says 10.20.193.16:43182) 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fceec7640 1 -- 10.20.193.16:0/2037198501 learned_addr learned my addr 10.20.193.16:0/2037198501 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fceec7640 10 monclient(hunting): get_auth_request con 0x7f9fc81536f0 auth_method 0 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fceec7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fceec7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fceec7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fce6c6640 10 monclient(hunting): get_auth_request con 0x7f9fc80700f0 auth_method 0 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fce6c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fce6c6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fce6c6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fcdec5640 10 monclient(hunting): get_auth_request con 0x7f9fc80c86f0 auth_method 0 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fcdec5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fcdec5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fcdec5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fceec7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fceec7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fceec7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fce6c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fce6c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fce6c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fcdec5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fcdec5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:52.841 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.839+0000 7f9fcdec5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.840+0000 7f9fceec7640 10 monclient(hunting): handle_auth_done global_id 5098 payload 824 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.840+0000 7f9fceec7640 1 -- 10.20.193.16:0/2037198501 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9fc80c86f0 msgr2=0x7f9fc8151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.840+0000 7f9fceec7640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9fc80c86f0 0x7f9fc8151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.840+0000 7f9fceec7640 1 -- 10.20.193.16:0/2037198501 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9fc80700f0 msgr2=0x7f9fc80cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.840+0000 7f9fceec7640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9fc80700f0 0x7f9fc80cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.840+0000 7f9fceec7640 10 monclient: _finish_hunting 0 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.840+0000 7f9fceec7640 1 monclient: found mon.noname-b 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.840+0000 7f9fceec7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.840+0000 7f9fceec7640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.840+0000 7f9fceec7640 1 -- 10.20.193.16:0/2037198501 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc8073670 con 0x7f9fc81536f0 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.841+0000 7f9fceec7640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 0x7f9fc8151870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc03a040 tx=0x7f9fbc004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.841+0000 7f9fb2ffd640 10 client.?.objecter ms_handle_connect 0x7f9fc81536f0 2026-02-17T06:01:52.842 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.841+0000 7f9fb2ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:52.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 1 -- 10.20.193.16:0/2037198501 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc047020 con 0x7f9fc81536f0 2026-02-17T06:01:52.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:52.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:52.843 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 10 monclient: dump: 2026-02-17T06:01:52.843 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:52.843 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:52.843 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:52.843 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:52.843 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:52.843 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 1 -- 10.20.193.16:0/2037198501 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fbc042670 con 0x7f9fc81536f0 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 1 -- 10.20.193.16:0/2037198501 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc03c040 con 0x7f9fc81536f0 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fb2ffd640 10 monclient: dump: 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 5 monclient: authenticate success, global_id 5098 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fccec3640 4 set_mon_vals no callback set 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 10 monclient: _renew_subs 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 1 -- 10.20.193.16:0/2037198501 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc8073950 con 0x7f9fc81536f0 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 10 client.5098.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 10 monclient: _renew_subs 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 1 -- 10.20.193.16:0/2037198501 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc8073f10 con 0x7f9fc81536f0 2026-02-17T06:01:52.844 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 1 librados: init done 2026-02-17T06:01:52.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:52.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:52.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:52.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.843+0000 7f9fd0951640 1 -- 10.20.193.16:0/2037198501 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc80704d0 con 0x7f9fc81536f0 2026-02-17T06:01:52.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.855+0000 7f9fb2ffd640 1 -- 10.20.193.16:0/2037198501 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f9fbc042a40 con 0x7f9fc81536f0 2026-02-17T06:01:52.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.856+0000 7f9fb2ffd640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9f98079190 0x7f9f9807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:52.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.856+0000 7f9fce6c6640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9f98079190 0x7f9f9807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:52.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.856+0000 7f9fce6c6640 10 monclient: get_auth_request con 0x7f9f98079190 auth_method 0 2026-02-17T06:01:52.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.856+0000 7f9fce6c6640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9f98079190 0x7f9f9807b650 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0002770 tx=0x7f9fc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:52.856 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.856+0000 7f9fb2ffd640 10 client.5098.objecter ms_handle_connect 0x7f9f98079190 2026-02-17T06:01:52.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.857+0000 7f9fb2ffd640 1 -- 10.20.193.16:0/2037198501 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f9fbc042d60 con 0x7f9fc81536f0 2026-02-17T06:01:52.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.857+0000 7f9fb2ffd640 10 client.5098.objecter ms_dispatch2 0x7f9fc8000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:52.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.857+0000 7f9fb2ffd640 3 client.5098.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:52.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.857+0000 7f9fb2ffd640 3 client.5098.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:52.857 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.857+0000 7f9fb2ffd640 20 client.5098.objecter dump_active .. 0 homeless 2026-02-17T06:01:52.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.956+0000 7f9fb2ffd640 1 -- 10.20.193.16:0/2037198501 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9fbc051030 con 0x7f9fc81536f0 2026-02-17T06:01:52.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.956+0000 7f9fb2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:52.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:52.956+0000 7f9fb2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:53.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.055+0000 7f9fd0951640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T06:01:53.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.055+0000 7f9fd0951640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:53.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.055+0000 7f9fd0951640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.055+0000 7f9fd0951640 1 -- 10.20.193.16:0/2037198501 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9fc80c5800 con 0x7f9fc81536f0 2026-02-17T06:01:53.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.058+0000 7f9fb2ffd640 1 -- 10.20.193.16:0/2037198501 <== mon.1 v2:10.20.193.128:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f9fbc088580 con 0x7f9fc81536f0 2026-02-17T06:01:53.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.058+0000 7f9fb2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T06:01:53.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.058+0000 7f9fb2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:53.059 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:53.059 INFO:teuthology.orchestra.run.trial016.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"11 OSD(s) experiencing slow operations in BlueStore","count":11},"muted":false}},"mutes":[]} 2026-02-17T06:01:53.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 10 librados: watch_flush enter 2026-02-17T06:01:53.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 10 librados: watch_flush exit 2026-02-17T06:01:53.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 20 client.5098.objecter shutdown clearing up homeless session... 2026-02-17T06:01:53.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 10 client.5098.objecter successfully canceled tick 2026-02-17T06:01:53.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 -- 10.20.193.16:0/2037198501 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9f98079190 msgr2=0x7f9f9807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9f98079190 0x7f9f9807b650 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0002770 tx=0x7f9fc0009040 comp rx=0 tx=0).stop 2026-02-17T06:01:53.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 10 monclient: shutdown 2026-02-17T06:01:53.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:53.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 -- 10.20.193.16:0/2037198501 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 msgr2=0x7f9fc8151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 0x7f9fc8151870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc03a040 tx=0x7f9fbc004b40 comp rx=0 tx=0).stop 2026-02-17T06:01:53.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 -- 10.20.193.16:0/2037198501 shutdown_connections 2026-02-17T06:01:53.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f9f98079190 0x7f9f9807b650 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f9fc81536f0 0x7f9fc8151870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f9fc80c86f0 0x7f9fc8151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 --2- 10.20.193.16:0/2037198501 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f9fc80700f0 0x7f9fc80cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 -- 10.20.193.16:0/2037198501 >> 10.20.193.16:0/2037198501 conn(0x7f9fc80cd770 msgr2=0x7f9fc81e8290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:53.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 -- 10.20.193.16:0/2037198501 shutdown_connections 2026-02-17T06:01:53.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 -- 10.20.193.16:0/2037198501 wait complete. 2026-02-17T06:01:53.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.059+0000 7f9fd0951640 1 librados: shutdown 2026-02-17T06:01:53.068 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-17T06:01:53.068 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:01:53.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.135+0000 7f2b26dba640 10 monclient: get_monmap_and_config 2026-02-17T06:01:53.135 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.135+0000 7f2b26dba640 10 monclient: build_initial_monmap 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.135+0000 7f2b26dba640 10 monclient: monmap: 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:53.136187+0000 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:53.136187+0000 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:53.136 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:53.137 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:53.137 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.135+0000 7f2b26dba640 1 Processor -- start 2026-02-17T06:01:53.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.135+0000 7f2b26dba640 1 -- start start 2026-02-17T06:01:53.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.135+0000 7f2b26dba640 10 monclient: init 2026-02-17T06:01:53.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:53.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 10 monclient: _add_conns ranks=[1,5,4,3,2,0] 2026-02-17T06:01:53.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f2b200700f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:53.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f2b200c86f0 0x7f2b200704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 10 monclient(hunting): picked mon.noname-c con 0x7f2b200c86f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:53.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f2b201536f0 0x7f2b20153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.140 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 10 monclient(hunting): picked mon.noname-b con 0x7f2b201536f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b200c4820 con 0x7f2b200700f0 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f2b200c4b00 con 0x7f2b201536f0 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f2b200c4de0 con 0x7f2b200c86f0 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b26dba640 10 monclient(hunting): authenticate will time out at 1365.705137s 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b24b2f640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f2b200700f0 0x7f2b200c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:51260/0 (socket says 10.20.193.16:51260) 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b24b2f640 1 -- 10.20.193.16:0/2290029771 learned_addr learned my addr 10.20.193.16:0/2290029771 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b17fff640 1 --2- 10.20.193.16:0/2290029771 >> v2:10.20.193.148:3300/0 conn(0x7f2b200c86f0 0x7f2b200704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.141 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b17fff640 10 monclient(hunting): get_auth_request con 0x7f2b200c86f0 auth_method 0 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b17fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b17fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.136+0000 7f2b17fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.137+0000 7f2b17fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.137+0000 7f2b17fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.137+0000 7f2b17fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b25330640 1 --2- 10.20.193.16:0/2290029771 >> v2:10.20.193.128:3300/0 conn(0x7f2b201536f0 0x7f2b20153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b25330640 10 monclient(hunting): get_auth_request con 0x7f2b201536f0 auth_method 0 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b25330640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b25330640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b25330640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.142 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b17fff640 10 monclient(hunting): handle_auth_done global_id 5162 payload 274 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b17fff640 1 -- 10.20.193.16:0/2290029771 >> v2:10.20.193.128:3300/0 conn(0x7f2b201536f0 msgr2=0x7f2b20153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b17fff640 1 --2- 10.20.193.16:0/2290029771 >> v2:10.20.193.128:3300/0 conn(0x7f2b201536f0 0x7f2b20153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b17fff640 1 -- 10.20.193.16:0/2290029771 >> v1:10.20.193.128:6789/0 conn(0x7f2b200700f0 legacy=0x7f2b200c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b17fff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b17fff640 1 monclient: found mon.noname-c 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b177fe640 1 -- 10.20.193.16:0/2290029771 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1949981788 0 0) 0x7f2b200c4820 con 0x7f2b200700f0 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b17fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b17fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b17fff640 1 -- 10.20.193.16:0/2290029771 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b200c50c0 con 0x7f2b200c86f0 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.138+0000 7f2b177fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T06:01:53.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.139+0000 7f2b17fff640 1 --2- 10.20.193.16:0/2290029771 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200c86f0 0x7f2b200704d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2b200c5b80 tx=0x7f2b0802ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bd3e345d88dd89d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:53.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 1 -- 10.20.193.16:0/2290029771 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0803c070 con 0x7f2b200c86f0 2026-02-17T06:01:53.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:53.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 10 monclient: dump: 2026-02-17T06:01:53.155 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.155 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.155 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.155 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.155 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.156 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.156 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.156 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.156 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.156 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.156 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:53.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:53.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:53.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 1 -- 10.20.193.16:0/2290029771 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b0802fa60 con 0x7f2b200c86f0 2026-02-17T06:01:53.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:53.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 1 -- 10.20.193.16:0/2290029771 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0802fe30 con 0x7f2b200c86f0 2026-02-17T06:01:53.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:53.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b177fe640 10 monclient: dump: 2026-02-17T06:01:53.158 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.158 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.158 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.158 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.158 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.159 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.159 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.159 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.159 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.159 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.159 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 5 monclient: authenticate success, global_id 5162 2026-02-17T06:01:53.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:53.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 4 set_mon_vals no callback set 2026-02-17T06:01:53.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 10 monclient: shutdown 2026-02-17T06:01:53.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:53.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 1 -- 10.20.193.16:0/2290029771 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200c86f0 msgr2=0x7f2b200704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.160 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 1 --2- 10.20.193.16:0/2290029771 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200c86f0 0x7f2b200704d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2b200c5b80 tx=0x7f2b0802ed50 comp rx=0 tx=0).stop 2026-02-17T06:01:53.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 1 -- 10.20.193.16:0/2290029771 shutdown_connections 2026-02-17T06:01:53.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 1 --2- 10.20.193.16:0/2290029771 >> v2:10.20.193.128:3300/0 conn(0x7f2b201536f0 0x7f2b20153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 1 --2- 10.20.193.16:0/2290029771 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200c86f0 0x7f2b200704d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 1 -- 10.20.193.16:0/2290029771 >> 10.20.193.16:0/2290029771 conn(0x7f2b200cd770 msgr2=0x7f2b200cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:53.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 1 -- 10.20.193.16:0/2290029771 shutdown_connections 2026-02-17T06:01:53.161 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.141+0000 7f2b26dba640 1 -- 10.20.193.16:0/2290029771 wait complete. 2026-02-17T06:01:53.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient: build_initial_monmap 2026-02-17T06:01:53.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient: monmap: 2026-02-17T06:01:53.162 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:53.162 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:53.162 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:53.162 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:53.163 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:53.163 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.163 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:53.163 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:53.163 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:53.163 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 librados: starting msgr at 2026-02-17T06:01:53.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 librados: starting objecter 2026-02-17T06:01:53.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 Processor -- start 2026-02-17T06:01:53.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 -- start start 2026-02-17T06:01:53.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 librados: setting wanted keys 2026-02-17T06:01:53.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 librados: calling monclient init 2026-02-17T06:01:53.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient: init 2026-02-17T06:01:53.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:53.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:53.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200700f0 0x7f2b201f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient(hunting): picked mon.noname-c con 0x7f2b200700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:53.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2b200c86f0 0x7f2b201f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient(hunting): picked mon.noname-a con 0x7f2b200c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:53.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2b201536f0 0x7f2b200cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient(hunting): picked mon.noname-b con 0x7f2b201536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:53.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20154060 con 0x7f2b200c86f0 2026-02-17T06:01:53.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f2b200c5680 con 0x7f2b201536f0 2026-02-17T06:01:53.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20074c00 con 0x7f2b200700f0 2026-02-17T06:01:53.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:53.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.142+0000 7f2b26dba640 10 monclient(hunting): authenticate will time out at 1365.711864s 2026-02-17T06:01:53.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b24b2f640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200700f0 0x7f2b201f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.167 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b17fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2b200c86f0 0x7f2b201f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b25330640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2b201536f0 0x7f2b200cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b24b2f640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200700f0 0x7f2b201f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:53060/0 (socket says 10.20.193.16:53060) 2026-02-17T06:01:53.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b24b2f640 1 -- 10.20.193.16:0/2057121520 learned_addr learned my addr 10.20.193.16:0/2057121520 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:53.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b24b2f640 10 monclient(hunting): get_auth_request con 0x7f2b200700f0 auth_method 0 2026-02-17T06:01:53.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b24b2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b24b2f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b24b2f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b25330640 10 monclient(hunting): get_auth_request con 0x7f2b201536f0 auth_method 0 2026-02-17T06:01:53.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b25330640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b25330640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b25330640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b17fff640 10 monclient(hunting): get_auth_request con 0x7f2b200c86f0 auth_method 0 2026-02-17T06:01:53.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b17fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b17fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b17fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b24b2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b24b2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b24b2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b17fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b17fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b17fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b25330640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b25330640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.143+0000 7f2b25330640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.145+0000 7f2b24b2f640 10 monclient(hunting): handle_auth_done global_id 5165 payload 824 2026-02-17T06:01:53.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.145+0000 7f2b24b2f640 1 -- 10.20.193.16:0/2057121520 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2b201536f0 msgr2=0x7f2b200cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.145+0000 7f2b24b2f640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2b201536f0 0x7f2b200cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.145+0000 7f2b24b2f640 1 -- 10.20.193.16:0/2057121520 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2b200c86f0 msgr2=0x7f2b201f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.145+0000 7f2b24b2f640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2b200c86f0 0x7f2b201f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.145+0000 7f2b24b2f640 10 monclient: _finish_hunting 0 2026-02-17T06:01:53.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.145+0000 7f2b24b2f640 1 monclient: found mon.noname-c 2026-02-17T06:01:53.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.145+0000 7f2b24b2f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:53.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.145+0000 7f2b24b2f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:53.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.145+0000 7f2b24b2f640 1 -- 10.20.193.16:0/2057121520 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b201f2900 con 0x7f2b200700f0 2026-02-17T06:01:53.172 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.146+0000 7f2b24b2f640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200700f0 0x7f2b201f8360 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000c4d0 tx=0x7f2b1000c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:53.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.146+0000 7f2b157fa640 10 client.?.objecter ms_handle_connect 0x7f2b200700f0 2026-02-17T06:01:53.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.146+0000 7f2b157fa640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:53.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 1 -- 10.20.193.16:0/2057121520 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10018070 con 0x7f2b200700f0 2026-02-17T06:01:53.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:53.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 10 monclient: dump: 2026-02-17T06:01:53.173 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.173 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.174 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.174 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.174 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.174 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.174 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.174 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.174 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.175 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 10 monclient: _finish_auth 0 2026-02-17T06:01:53.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 10 monclient: _check_auth_tickets 2026-02-17T06:01:53.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:53.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 5 monclient: authenticate success, global_id 5165 2026-02-17T06:01:53.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 10 monclient: _renew_subs 2026-02-17T06:01:53.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:53.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 1 -- 10.20.193.16:0/2057121520 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b201f4e90 con 0x7f2b200700f0 2026-02-17T06:01:53.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 10 client.5165.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:53.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 10 monclient: _renew_subs 2026-02-17T06:01:53.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:53.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 1 -- 10.20.193.16:0/2057121520 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2006e540 con 0x7f2b200700f0 2026-02-17T06:01:53.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 1 librados: init done 2026-02-17T06:01:53.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:53.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:53.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:53.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b26dba640 1 -- 10.20.193.16:0/2057121520 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b200704d0 con 0x7f2b200700f0 2026-02-17T06:01:53.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 1 -- 10.20.193.16:0/2057121520 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b10004cf0 con 0x7f2b200700f0 2026-02-17T06:01:53.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:53.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 1 -- 10.20.193.16:0/2057121520 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10022060 con 0x7f2b200700f0 2026-02-17T06:01:53.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:53.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b157fa640 10 monclient: dump: 2026-02-17T06:01:53.177 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.177 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.178 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.178 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.178 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.178 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.178 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.178 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.178 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.178 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.179 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.147+0000 7f2b16ffd640 4 set_mon_vals no callback set 2026-02-17T06:01:53.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.254+0000 7f2b157fa640 1 -- 10.20.193.16:0/2057121520 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f2b10025080 con 0x7f2b200700f0 2026-02-17T06:01:53.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.254+0000 7f2b157fa640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2af0079140 0x7f2af007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.254+0000 7f2b17fff640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2af0079140 0x7f2af007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.254+0000 7f2b17fff640 10 monclient: get_auth_request con 0x7f2af0079140 auth_method 0 2026-02-17T06:01:53.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.255+0000 7f2b157fa640 10 client.5165.objecter ms_handle_connect 0x7f2af0079140 2026-02-17T06:01:53.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.255+0000 7f2b17fff640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2af0079140 0x7f2af007b600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b08039000 tx=0x7f2b08033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:53.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.255+0000 7f2b157fa640 1 -- 10.20.193.16:0/2057121520 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f2b1005d780 con 0x7f2b200700f0 2026-02-17T06:01:53.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.255+0000 7f2b157fa640 10 client.5165.objecter ms_dispatch2 0x7f2b20000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:53.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.255+0000 7f2b157fa640 3 client.5165.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:53.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.255+0000 7f2b157fa640 3 client.5165.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:53.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.255+0000 7f2b157fa640 20 client.5165.objecter dump_active .. 0 homeless 2026-02-17T06:01:53.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.262+0000 7f2b157fa640 1 -- 10.20.193.16:0/2057121520 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2b10061740 con 0x7f2b200700f0 2026-02-17T06:01:53.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.262+0000 7f2b157fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:53.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.262+0000 7f2b157fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:53.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.343+0000 7f2b26dba640 1 -- 10.20.193.16:0/2057121520 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b200cd0a0 con 0x7f2af0079140 2026-02-17T06:01:53.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.344+0000 7f2b157fa640 1 -- 10.20.193.16:0/2057121520 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44800 (secure 0 0 0) 0x7f2b200cd0a0 con 0x7f2af0079140 2026-02-17T06:01:53.345 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:53.347 INFO:teuthology.orchestra.run.trial016.stderr:dumped all 2026-02-17T06:01:53.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 10 librados: watch_flush enter 2026-02-17T06:01:53.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 10 librados: watch_flush exit 2026-02-17T06:01:53.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 20 client.5165.objecter shutdown clearing up homeless session... 2026-02-17T06:01:53.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 10 client.5165.objecter successfully canceled tick 2026-02-17T06:01:53.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 -- 10.20.193.16:0/2057121520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2af0079140 msgr2=0x7f2af007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2af0079140 0x7f2af007b600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b08039000 tx=0x7f2b08033000 comp rx=0 tx=0).stop 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 10 monclient: shutdown 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 -- 10.20.193.16:0/2057121520 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200700f0 msgr2=0x7f2b201f8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200700f0 0x7f2b201f8360 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000c4d0 tx=0x7f2b1000c9a0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 -- 10.20.193.16:0/2057121520 shutdown_connections 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2af0079140 0x7f2af007b600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2b201536f0 0x7f2b200cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2b200c86f0 0x7f2b201f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 --2- 10.20.193.16:0/2057121520 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2b200700f0 0x7f2b201f8360 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 -- 10.20.193.16:0/2057121520 >> 10.20.193.16:0/2057121520 conn(0x7f2b200cd770 msgr2=0x7f2b200c6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:53.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 -- 10.20.193.16:0/2057121520 shutdown_connections 2026-02-17T06:01:53.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 -- 10.20.193.16:0/2057121520 wait complete. 2026-02-17T06:01:53.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.345+0000 7f2b26dba640 1 librados: shutdown 2026-02-17T06:01:53.354 INFO:teuthology.orchestra.run.trial016.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2026-02-17T06:01:51.925515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437976,"kb_used_data":6552,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509511464,"statfs":{"total":1546188226560,"available":1545739739136,"internally_reserved":0,"allocated":6709248,"data_stored":3434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25505,"internal_metadata":441424991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002029"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:57.003241+0000","last_change":"2026-02-17T05:56:45.946554+0000","last_active":"2026-02-17T05:58:57.003241+0000","last_peered":"2026-02-17T05:58:57.003241+0000","last_clean":"2026-02-17T05:58:57.003241+0000","last_became_active":"2026-02-17T05:56:43.852478+0000","last_became_peered":"2026-02-17T05:56:43.852478+0000","last_unstale":"2026-02-17T05:58:57.003241+0000","last_undegraded":"2026-02-17T05:58:57.003241+0000","last_fullsized":"2026-02-17T05:58:57.003241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T08:21:55.695169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031385049999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:54.575851+0000","last_change":"2026-02-17T05:56:46.700244+0000","last_active":"2026-02-17T05:58:54.575851+0000","last_peered":"2026-02-17T05:58:54.575851+0000","last_clean":"2026-02-17T05:58:54.575851+0000","last_became_active":"2026-02-17T05:56:43.182525+0000","last_became_peered":"2026-02-17T05:56:43.182525+0000","last_unstale":"2026-02-17T05:58:54.575851+0000","last_undegraded":"2026-02-17T05:58:54.575851+0000","last_fullsized":"2026-02-17T05:58:54.575851+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:38:44.190148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038112767999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:47.494561+0000","last_change":"2026-02-17T05:56:45.501489+0000","last_active":"2026-02-17T05:58:47.494561+0000","last_peered":"2026-02-17T05:58:47.494561+0000","last_clean":"2026-02-17T05:58:47.494561+0000","last_became_active":"2026-02-17T05:56:43.381684+0000","last_became_peered":"2026-02-17T05:56:43.381684+0000","last_unstale":"2026-02-17T05:58:47.494561+0000","last_undegraded":"2026-02-17T05:58:47.494561+0000","last_fullsized":"2026-02-17T05:58:47.494561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:07:04.503400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031884497999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:50.945819+0000","last_change":"2026-02-17T05:56:47.091287+0000","last_active":"2026-02-17T05:58:50.945819+0000","last_peered":"2026-02-17T05:58:50.945819+0000","last_clean":"2026-02-17T05:58:50.945819+0000","last_became_active":"2026-02-17T05:56:43.290452+0000","last_became_peered":"2026-02-17T05:56:43.290452+0000","last_unstale":"2026-02-17T05:58:50.945819+0000","last_undegraded":"2026-02-17T05:58:50.945819+0000","last_fullsized":"2026-02-17T05:58:50.945819+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:12:29.979912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032515719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"33'2","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:47.312723+0000","last_change":"2026-02-17T05:56:45.527362+0000","last_active":"2026-02-17T05:58:47.312723+0000","last_peered":"2026-02-17T05:58:47.312723+0000","last_clean":"2026-02-17T05:58:47.312723+0000","last_became_active":"2026-02-17T05:56:43.262979+0000","last_became_peered":"2026-02-17T05:56:43.262979+0000","last_unstale":"2026-02-17T05:58:47.312723+0000","last_undegraded":"2026-02-17T05:58:47.312723+0000","last_fullsized":"2026-02-17T05:58:47.312723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:30:08.337496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.056715723000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:50.698151+0000","last_change":"2026-02-17T05:56:46.695943+0000","last_active":"2026-02-17T05:58:50.698151+0000","last_peered":"2026-02-17T05:58:50.698151+0000","last_clean":"2026-02-17T05:58:50.698151+0000","last_became_active":"2026-02-17T05:56:43.075477+0000","last_became_peered":"2026-02-17T05:56:43.075477+0000","last_unstale":"2026-02-17T05:58:50.698151+0000","last_undegraded":"2026-02-17T05:58:50.698151+0000","last_fullsized":"2026-02-17T05:58:50.698151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T07:20:23.191397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.030611670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:42.310803+0000","last_change":"2026-02-17T05:56:45.522106+0000","last_active":"2026-02-17T05:58:42.310803+0000","last_peered":"2026-02-17T05:58:42.310803+0000","last_clean":"2026-02-17T05:58:42.310803+0000","last_became_active":"2026-02-17T05:56:43.142816+0000","last_became_peered":"2026-02-17T05:56:43.142816+0000","last_unstale":"2026-02-17T05:58:42.310803+0000","last_undegraded":"2026-02-17T05:58:42.310803+0000","last_fullsized":"2026-02-17T05:58:42.310803+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:47:19.171322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.043200879999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"31'1","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:50.430092+0000","last_change":"2026-02-17T05:56:46.694818+0000","last_active":"2026-02-17T05:58:50.430092+0000","last_peered":"2026-02-17T05:58:50.430092+0000","last_clean":"2026-02-17T05:58:50.430092+0000","last_became_active":"2026-02-17T05:56:43.121999+0000","last_became_peered":"2026-02-17T05:56:43.121999+0000","last_unstale":"2026-02-17T05:58:50.430092+0000","last_undegraded":"2026-02-17T05:58:50.430092+0000","last_fullsized":"2026-02-17T05:58:50.430092+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T17:12:50.512061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.034338280999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"32'32","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:35.692645+0000","last_change":"2026-02-17T05:56:37.927084+0000","last_active":"2026-02-17T05:58:35.692645+0000","last_peered":"2026-02-17T05:58:35.692645+0000","last_clean":"2026-02-17T05:58:35.692645+0000","last_became_active":"2026-02-17T05:56:37.919763+0000","last_became_peered":"2026-02-17T05:56:37.919763+0000","last_unstale":"2026-02-17T05:58:35.692645+0000","last_undegraded":"2026-02-17T05:58:35.692645+0000","last_fullsized":"2026-02-17T05:58:35.692645+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:30.809269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:30.809269+0000","last_clean_scrub_stamp":"2026-02-17T05:56:30.809269+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:55:23.546716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":28,"seq":120259084357,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":28,"seq":120259084356,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":27,"seq":115964117059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,3,4,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313284,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280579,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,4,5,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313284,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":372736,"data_stored":164912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[3,4,5,6,7,8,9,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":23,"seq":98784247874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":25,"seq":107374182470,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":23,"seq":98784247873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":372736,"data_stored":164912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[0,1,2,4,5,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":23,"seq":98784247873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":23,"seq":98784247874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,6,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":25,"seq":107374182466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,5,6,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":25,"seq":107374182466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":770048,"data_stored":562733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,4,5,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":27,"seq":115964117063,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":770048,"data_stored":562733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":27,"seq":115964117063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[1,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:01:53.357 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:01:53.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.423+0000 7f4d5fbb6640 10 monclient: get_monmap_and_config 2026-02-17T06:01:53.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.423+0000 7f4d5fbb6640 10 monclient: build_initial_monmap 2026-02-17T06:01:53.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.423+0000 7f4d5fbb6640 10 monclient: monmap: 2026-02-17T06:01:53.424 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:53.424 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.424 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:53.424643+0000 2026-02-17T06:01:53.424 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:53.424643+0000 2026-02-17T06:01:53.424 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:53.425 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.425 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:53.425 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:53.425 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:53.425 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:53.425 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:53.425 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:53.425 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 1 Processor -- start 2026-02-17T06:01:53.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 1 -- start start 2026-02-17T06:01:53.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient: init 2026-02-17T06:01:53.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:53.428 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient: _add_conns ranks=[4,3,5,1,2,0] 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f4d580700f0 0x7f4d580704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient(hunting): picked mon.noname-b con 0x7f4d580700f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f4d580c7fe0 0x7f4d580c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient(hunting): picked mon.noname-a con 0x7f4d580c7fe0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f4d581536f0 0x7f4d58153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient(hunting): picked mon.noname-c con 0x7f4d581536f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4d580c8970 con 0x7f4d580c7fe0 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f4d580c4820 con 0x7f4d580700f0 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f4d580c4b00 con 0x7f4d581536f0 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5fbb6640 10 monclient(hunting): authenticate will time out at 1365.993593s 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5d92b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f4d580700f0 0x7f4d580704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.424+0000 7f4d5d12a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f4d580c7fe0 0x7f4d580c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d92b640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f4d580700f0 0x7f4d580704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43204/0 (socket says 10.20.193.16:43204) 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d92b640 1 -- 10.20.193.16:0/4060158556 learned_addr learned my addr 10.20.193.16:0/4060158556 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:53.429 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d92b640 10 monclient(hunting): get_auth_request con 0x7f4d580700f0 auth_method 0 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d92b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d92b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d92b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5e12c640 1 --2- 10.20.193.16:0/4060158556 >> v2:10.20.193.148:3300/0 conn(0x7f4d581536f0 0x7f4d58153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d12a640 10 monclient(hunting): get_auth_request con 0x7f4d580c7fe0 auth_method 0 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d12a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d12a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d12a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5e12c640 10 monclient(hunting): get_auth_request con 0x7f4d581536f0 auth_method 0 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5e12c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5e12c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5e12c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d92b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d92b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d92b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d12a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d12a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5d12a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5e12c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5e12c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.425+0000 7f4d5e12c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.426+0000 7f4d5d92b640 10 monclient(hunting): handle_auth_done global_id 5110 payload 274 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.426+0000 7f4d5d92b640 1 -- 10.20.193.16:0/4060158556 >> v2:10.20.193.148:3300/0 conn(0x7f4d581536f0 msgr2=0x7f4d58153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.426+0000 7f4d5d92b640 1 --2- 10.20.193.16:0/4060158556 >> v2:10.20.193.148:3300/0 conn(0x7f4d581536f0 0x7f4d58153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.426+0000 7f4d5d92b640 1 -- 10.20.193.16:0/4060158556 >> v2:10.20.193.16:3300/0 conn(0x7f4d580c7fe0 msgr2=0x7f4d580c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.426+0000 7f4d5d92b640 1 --2- 10.20.193.16:0/4060158556 >> v2:10.20.193.16:3300/0 conn(0x7f4d580c7fe0 0x7f4d580c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.426+0000 7f4d5d92b640 10 monclient: _finish_hunting 0 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.426+0000 7f4d5d92b640 1 monclient: found mon.noname-b 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.426+0000 7f4d5d92b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.426+0000 7f4d5d92b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.426+0000 7f4d5d92b640 1 -- 10.20.193.16:0/4060158556 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d580c4de0 con 0x7f4d580700f0 2026-02-17T06:01:53.431 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.427+0000 7f4d5d92b640 1 --2- 10.20.193.16:0/4060158556 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d580700f0 0x7f4d580704d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4d58070b90 tx=0x7f4d4802edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc75f45396230ee9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:53.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 1 -- 10.20.193.16:0/4060158556 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4803c070 con 0x7f4d580700f0 2026-02-17T06:01:53.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:53.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 10 monclient: dump: 2026-02-17T06:01:53.435 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.435 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 10 monclient: _finish_auth 0 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 10 monclient: _check_auth_tickets 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5fbb6640 5 monclient: authenticate success, global_id 5110 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 1 -- 10.20.193.16:0/4060158556 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4802fab0 con 0x7f4d580700f0 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5fbb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 1 -- 10.20.193.16:0/4060158556 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48037440 con 0x7f4d580700f0 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5fbb6640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:53.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5fbb6640 4 set_mon_vals no callback set 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.428+0000 7f4d5c929640 10 monclient: dump: 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 10 monclient: shutdown 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/4060158556 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d580700f0 msgr2=0x7f4d580704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 --2- 10.20.193.16:0/4060158556 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d580700f0 0x7f4d580704d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4d58070b90 tx=0x7f4d4802edf0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/4060158556 shutdown_connections 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 --2- 10.20.193.16:0/4060158556 >> v2:10.20.193.148:3300/0 conn(0x7f4d581536f0 0x7f4d58153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 --2- 10.20.193.16:0/4060158556 >> v2:10.20.193.16:3300/0 conn(0x7f4d580c7fe0 0x7f4d580c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 --2- 10.20.193.16:0/4060158556 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d580700f0 0x7f4d580704d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/4060158556 >> 10.20.193.16:0/4060158556 conn(0x7f4d580cd770 msgr2=0x7f4d580cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/4060158556 shutdown_connections 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/4060158556 wait complete. 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 10 monclient: build_initial_monmap 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 10 monclient: monmap: 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:53.438 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 librados: starting msgr at 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 librados: starting objecter 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 Processor -- start 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 -- start start 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 librados: setting wanted keys 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 1 librados: calling monclient init 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.429+0000 7f4d5fbb6640 10 monclient: init 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4d580700f0 0x7f4d58151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 10 monclient(hunting): picked mon.noname-a con 0x7f4d580700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4d580c7fe0 0x7f4d581489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 10 monclient(hunting): picked mon.noname-c con 0x7f4d580c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d581536f0 0x7f4d581e7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 10 monclient(hunting): picked mon.noname-b con 0x7f4d581536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d580c5680 con 0x7f4d580700f0 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.439 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f4d580c53a0 con 0x7f4d581536f0 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4d58154040 con 0x7f4d580c7fe0 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5fbb6640 10 monclient(hunting): authenticate will time out at 1365.999274s 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5e12c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d581536f0 0x7f4d581e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5d12a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4d580c7fe0 0x7f4d581489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5e12c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d581536f0 0x7f4d581e7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43210/0 (socket says 10.20.193.16:43210) 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5e12c640 1 -- 10.20.193.16:0/1020189059 learned_addr learned my addr 10.20.193.16:0/1020189059 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5e12c640 10 monclient(hunting): get_auth_request con 0x7f4d581536f0 auth_method 0 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5e12c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5e12c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5e12c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5d92b640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4d580700f0 0x7f4d58151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5d12a640 10 monclient(hunting): get_auth_request con 0x7f4d580c7fe0 auth_method 0 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5d12a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5d12a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5d12a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.440 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5e12c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5e12c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5e12c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5d92b640 10 monclient(hunting): get_auth_request con 0x7f4d580700f0 auth_method 0 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5d92b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5d92b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.430+0000 7f4d5d92b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.431+0000 7f4d5d12a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.431+0000 7f4d5d12a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.431+0000 7f4d5d12a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.431+0000 7f4d5d92b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.431+0000 7f4d5d92b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.431+0000 7f4d5d92b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.432+0000 7f4d5e12c640 10 monclient(hunting): handle_auth_done global_id 5113 payload 824 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.432+0000 7f4d5e12c640 1 -- 10.20.193.16:0/1020189059 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4d580c7fe0 msgr2=0x7f4d581489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.432+0000 7f4d5e12c640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4d580c7fe0 0x7f4d581489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.432+0000 7f4d5e12c640 1 -- 10.20.193.16:0/1020189059 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4d580700f0 msgr2=0x7f4d58151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.432+0000 7f4d5e12c640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4d580700f0 0x7f4d58151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.432+0000 7f4d5e12c640 10 monclient: _finish_hunting 0 2026-02-17T06:01:53.441 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.432+0000 7f4d5e12c640 1 monclient: found mon.noname-b 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.432+0000 7f4d5e12c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.432+0000 7f4d5e12c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.432+0000 7f4d5e12c640 1 -- 10.20.193.16:0/1020189059 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d581e5de0 con 0x7f4d581536f0 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.433+0000 7f4d5e12c640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d581536f0 0x7f4d581e7680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400efc0 tx=0x7f4d5400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.433+0000 7f4d467fc640 10 client.?.objecter ms_handle_connect 0x7f4d581536f0 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.433+0000 7f4d467fc640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 1 -- 10.20.193.16:0/1020189059 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54018020 con 0x7f4d581536f0 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 10 monclient: dump: 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.442 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d5fbb6640 5 monclient: authenticate success, global_id 5113 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d5fbb6640 10 monclient: _renew_subs 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d5fbb6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/1020189059 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5806d310 con 0x7f4d581536f0 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d5fbb6640 10 client.5113.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d5fbb6640 10 monclient: _renew_subs 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d5fbb6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/1020189059 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5806d8d0 con 0x7f4d581536f0 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d5fbb6640 1 librados: init done 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 1 -- 10.20.193.16:0/1020189059 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d54014480 con 0x7f4d581536f0 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d47fff640 4 set_mon_vals no callback set 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 1 -- 10.20.193.16:0/1020189059 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54014830 con 0x7f4d581536f0 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.443 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.434+0000 7f4d467fc640 10 monclient: dump: 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.435+0000 7f4d5fbb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.435+0000 7f4d5fbb6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.435+0000 7f4d5fbb6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.435+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/1020189059 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d58074c00 con 0x7f4d581536f0 2026-02-17T06:01:53.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.446+0000 7f4d467fc640 1 -- 10.20.193.16:0/1020189059 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f4d540149d0 con 0x7f4d581536f0 2026-02-17T06:01:53.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.446+0000 7f4d467fc640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4d28079140 0x7f4d2807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.446+0000 7f4d5d92b640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4d28079140 0x7f4d2807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.446+0000 7f4d5d92b640 10 monclient: get_auth_request con 0x7f4d28079140 auth_method 0 2026-02-17T06:01:53.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.446+0000 7f4d467fc640 10 client.5113.objecter ms_handle_connect 0x7f4d28079140 2026-02-17T06:01:53.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.446+0000 7f4d5d92b640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4d28079140 0x7f4d2807b600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d580c5b80 tx=0x7f4d48039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:53.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.447+0000 7f4d467fc640 1 -- 10.20.193.16:0/1020189059 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f4d54098d10 con 0x7f4d581536f0 2026-02-17T06:01:53.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.447+0000 7f4d467fc640 10 client.5113.objecter ms_dispatch2 0x7f4d58000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:53.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.447+0000 7f4d467fc640 3 client.5113.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:53.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.447+0000 7f4d467fc640 3 client.5113.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:53.448 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.447+0000 7f4d467fc640 20 client.5113.objecter dump_active .. 0 homeless 2026-02-17T06:01:53.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.545+0000 7f4d467fc640 1 -- 10.20.193.16:0/1020189059 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4d54022030 con 0x7f4d581536f0 2026-02-17T06:01:53.545 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.545+0000 7f4d467fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:53.546 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.545+0000 7f4d467fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:53.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.627+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/1020189059 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d581f7ba0 con 0x7f4d28079140 2026-02-17T06:01:53.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.628+0000 7f4d467fc640 1 -- 10.20.193.16:0/1020189059 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44800 (secure 0 0 0) 0x7f4d4803d040 con 0x7f4d28079140 2026-02-17T06:01:53.628 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:53.628 INFO:teuthology.orchestra.run.trial016.stderr:dumped all 2026-02-17T06:01:53.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 10 librados: watch_flush enter 2026-02-17T06:01:53.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 10 librados: watch_flush exit 2026-02-17T06:01:53.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 20 client.5113.objecter shutdown clearing up homeless session... 2026-02-17T06:01:53.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 10 client.5113.objecter successfully canceled tick 2026-02-17T06:01:53.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/1020189059 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4d28079140 msgr2=0x7f4d2807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4d28079140 0x7f4d2807b600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d580c5b80 tx=0x7f4d48039040 comp rx=0 tx=0).stop 2026-02-17T06:01:53.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 10 monclient: shutdown 2026-02-17T06:01:53.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:53.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/1020189059 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d581536f0 msgr2=0x7f4d581e7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d581536f0 0x7f4d581e7680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400efc0 tx=0x7f4d5400c490 comp rx=0 tx=0).stop 2026-02-17T06:01:53.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/1020189059 shutdown_connections 2026-02-17T06:01:53.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4d28079140 0x7f4d2807b600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4d581536f0 0x7f4d581e7680 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4d580c7fe0 0x7f4d581489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 --2- 10.20.193.16:0/1020189059 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4d580700f0 0x7f4d58151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/1020189059 >> 10.20.193.16:0/1020189059 conn(0x7f4d580cd770 msgr2=0x7f4d5814e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:53.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/1020189059 shutdown_connections 2026-02-17T06:01:53.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 -- 10.20.193.16:0/1020189059 wait complete. 2026-02-17T06:01:53.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.629+0000 7f4d5fbb6640 1 librados: shutdown 2026-02-17T06:01:53.637 INFO:teuthology.orchestra.run.trial016.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2026-02-17T06:01:51.925515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437976,"kb_used_data":6552,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509511464,"statfs":{"total":1546188226560,"available":1545739739136,"internally_reserved":0,"allocated":6709248,"data_stored":3434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25505,"internal_metadata":441424991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002029"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:57.003241+0000","last_change":"2026-02-17T05:56:45.946554+0000","last_active":"2026-02-17T05:58:57.003241+0000","last_peered":"2026-02-17T05:58:57.003241+0000","last_clean":"2026-02-17T05:58:57.003241+0000","last_became_active":"2026-02-17T05:56:43.852478+0000","last_became_peered":"2026-02-17T05:56:43.852478+0000","last_unstale":"2026-02-17T05:58:57.003241+0000","last_undegraded":"2026-02-17T05:58:57.003241+0000","last_fullsized":"2026-02-17T05:58:57.003241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T08:21:55.695169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031385049999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:54.575851+0000","last_change":"2026-02-17T05:56:46.700244+0000","last_active":"2026-02-17T05:58:54.575851+0000","last_peered":"2026-02-17T05:58:54.575851+0000","last_clean":"2026-02-17T05:58:54.575851+0000","last_became_active":"2026-02-17T05:56:43.182525+0000","last_became_peered":"2026-02-17T05:56:43.182525+0000","last_unstale":"2026-02-17T05:58:54.575851+0000","last_undegraded":"2026-02-17T05:58:54.575851+0000","last_fullsized":"2026-02-17T05:58:54.575851+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:38:44.190148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038112767999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:47.494561+0000","last_change":"2026-02-17T05:56:45.501489+0000","last_active":"2026-02-17T05:58:47.494561+0000","last_peered":"2026-02-17T05:58:47.494561+0000","last_clean":"2026-02-17T05:58:47.494561+0000","last_became_active":"2026-02-17T05:56:43.381684+0000","last_became_peered":"2026-02-17T05:56:43.381684+0000","last_unstale":"2026-02-17T05:58:47.494561+0000","last_undegraded":"2026-02-17T05:58:47.494561+0000","last_fullsized":"2026-02-17T05:58:47.494561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:07:04.503400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031884497999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:50.945819+0000","last_change":"2026-02-17T05:56:47.091287+0000","last_active":"2026-02-17T05:58:50.945819+0000","last_peered":"2026-02-17T05:58:50.945819+0000","last_clean":"2026-02-17T05:58:50.945819+0000","last_became_active":"2026-02-17T05:56:43.290452+0000","last_became_peered":"2026-02-17T05:56:43.290452+0000","last_unstale":"2026-02-17T05:58:50.945819+0000","last_undegraded":"2026-02-17T05:58:50.945819+0000","last_fullsized":"2026-02-17T05:58:50.945819+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:12:29.979912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032515719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"33'2","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:47.312723+0000","last_change":"2026-02-17T05:56:45.527362+0000","last_active":"2026-02-17T05:58:47.312723+0000","last_peered":"2026-02-17T05:58:47.312723+0000","last_clean":"2026-02-17T05:58:47.312723+0000","last_became_active":"2026-02-17T05:56:43.262979+0000","last_became_peered":"2026-02-17T05:56:43.262979+0000","last_unstale":"2026-02-17T05:58:47.312723+0000","last_undegraded":"2026-02-17T05:58:47.312723+0000","last_fullsized":"2026-02-17T05:58:47.312723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:30:08.337496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.056715723000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:50.698151+0000","last_change":"2026-02-17T05:56:46.695943+0000","last_active":"2026-02-17T05:58:50.698151+0000","last_peered":"2026-02-17T05:58:50.698151+0000","last_clean":"2026-02-17T05:58:50.698151+0000","last_became_active":"2026-02-17T05:56:43.075477+0000","last_became_peered":"2026-02-17T05:56:43.075477+0000","last_unstale":"2026-02-17T05:58:50.698151+0000","last_undegraded":"2026-02-17T05:58:50.698151+0000","last_fullsized":"2026-02-17T05:58:50.698151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T07:20:23.191397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.030611670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:42.310803+0000","last_change":"2026-02-17T05:56:45.522106+0000","last_active":"2026-02-17T05:58:42.310803+0000","last_peered":"2026-02-17T05:58:42.310803+0000","last_clean":"2026-02-17T05:58:42.310803+0000","last_became_active":"2026-02-17T05:56:43.142816+0000","last_became_peered":"2026-02-17T05:56:43.142816+0000","last_unstale":"2026-02-17T05:58:42.310803+0000","last_undegraded":"2026-02-17T05:58:42.310803+0000","last_fullsized":"2026-02-17T05:58:42.310803+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:47:19.171322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.043200879999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"31'1","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:50.430092+0000","last_change":"2026-02-17T05:56:46.694818+0000","last_active":"2026-02-17T05:58:50.430092+0000","last_peered":"2026-02-17T05:58:50.430092+0000","last_clean":"2026-02-17T05:58:50.430092+0000","last_became_active":"2026-02-17T05:56:43.121999+0000","last_became_peered":"2026-02-17T05:56:43.121999+0000","last_unstale":"2026-02-17T05:58:50.430092+0000","last_undegraded":"2026-02-17T05:58:50.430092+0000","last_fullsized":"2026-02-17T05:58:50.430092+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T17:12:50.512061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.034338280999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"32'32","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:35.692645+0000","last_change":"2026-02-17T05:56:37.927084+0000","last_active":"2026-02-17T05:58:35.692645+0000","last_peered":"2026-02-17T05:58:35.692645+0000","last_clean":"2026-02-17T05:58:35.692645+0000","last_became_active":"2026-02-17T05:56:37.919763+0000","last_became_peered":"2026-02-17T05:56:37.919763+0000","last_unstale":"2026-02-17T05:58:35.692645+0000","last_undegraded":"2026-02-17T05:58:35.692645+0000","last_fullsized":"2026-02-17T05:58:35.692645+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:30.809269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:30.809269+0000","last_clean_scrub_stamp":"2026-02-17T05:56:30.809269+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:55:23.546716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":28,"seq":120259084357,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":28,"seq":120259084356,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":27,"seq":115964117059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,3,4,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313284,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280579,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,4,5,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313284,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":372736,"data_stored":164912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[3,4,5,6,7,8,9,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":23,"seq":98784247874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":25,"seq":107374182470,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":23,"seq":98784247873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":372736,"data_stored":164912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[0,1,2,4,5,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":23,"seq":98784247873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":23,"seq":98784247874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,6,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":25,"seq":107374182466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,5,6,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":25,"seq":107374182466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":770048,"data_stored":562733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,4,5,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":27,"seq":115964117063,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":770048,"data_stored":562733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":27,"seq":115964117063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[1,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:01:53.638 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-17T06:01:53.638 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.704+0000 7f6e0a4b7640 10 monclient: get_monmap_and_config 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient: build_initial_monmap 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient: monmap: 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:53.706016+0000 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:53.706016+0000 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:53.705 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 1 Processor -- start 2026-02-17T06:01:53.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 1 -- start start 2026-02-17T06:01:53.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient: init 2026-02-17T06:01:53.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:53.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient: _add_conns ranks=[4,2,5,1,3,0] 2026-02-17T06:01:53.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f6e040700f0 0x7f6e040704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient(hunting): picked mon.noname-b con 0x7f6e040700f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:53.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6e040c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:53.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6e041536f0 0x7f6e04153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient(hunting): picked mon.noname-c con 0x7f6e041536f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:53.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e040c4820 con 0x7f6e040c7fe0 2026-02-17T06:01:53.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f6e040c4b00 con 0x7f6e040700f0 2026-02-17T06:01:53.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6e040c4de0 con 0x7f6e041536f0 2026-02-17T06:01:53.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:53.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.705+0000 7f6e0a4b7640 10 monclient(hunting): authenticate will time out at 1366.274953s 2026-02-17T06:01:53.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e03fff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f6e040700f0 0x7f6e040704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e08a2d640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f6e041536f0 0x7f6e04153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e03fff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f6e040700f0 0x7f6e040704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43214/0 (socket says 10.20.193.16:43214) 2026-02-17T06:01:53.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e03fff640 1 -- 10.20.193.16:0/3693315379 learned_addr learned my addr 10.20.193.16:0/3693315379 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:53.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e03fff640 10 monclient(hunting): get_auth_request con 0x7f6e040700f0 auth_method 0 2026-02-17T06:01:53.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e03fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e03fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e08a2d640 10 monclient(hunting): get_auth_request con 0x7f6e041536f0 auth_method 0 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e08a2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e08a2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e08a2d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.706+0000 7f6e03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e08a2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e08a2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e08a2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e03fff640 10 monclient(hunting): handle_auth_done global_id 5116 payload 274 2026-02-17T06:01:53.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e03fff640 1 -- 10.20.193.16:0/3693315379 >> v2:10.20.193.148:3300/0 conn(0x7f6e041536f0 msgr2=0x7f6e04153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:53.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e03fff640 1 --2- 10.20.193.16:0/3693315379 >> v2:10.20.193.148:3300/0 conn(0x7f6e041536f0 0x7f6e04153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e03fff640 1 -- 10.20.193.16:0/3693315379 >> v1:10.20.193.148:6789/0 conn(0x7f6e040c7fe0 legacy=0x7f6e040c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:53.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e03fff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:53.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e03fff640 1 monclient: found mon.noname-b 2026-02-17T06:01:53.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:53.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e03fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.707+0000 7f6e03fff640 1 -- 10.20.193.16:0/3693315379 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e040c50c0 con 0x7f6e040700f0 2026-02-17T06:01:53.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.708+0000 7f6e03fff640 1 --2- 10.20.193.16:0/3693315379 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e040700f0 0x7f6e040704d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6e040c9540 tx=0x7f6df002ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f144f74c887f6b3b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:53.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 1 -- 10.20.193.16:0/3693315379 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df003c070 con 0x7f6e040700f0 2026-02-17T06:01:53.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 10 monclient: dump: 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.718 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 5 monclient: authenticate success, global_id 5116 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 1 -- 10.20.193.16:0/3693315379 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df002fa90 con 0x7f6e040700f0 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 1 -- 10.20.193.16:0/3693315379 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df002fe60 con 0x7f6e040700f0 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e02ffd640 10 monclient: dump: 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.719 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 4 set_mon_vals no callback set 2026-02-17T06:01:53.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 10 monclient: shutdown 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3693315379 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e040700f0 msgr2=0x7f6e040704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 1 --2- 10.20.193.16:0/3693315379 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e040700f0 0x7f6e040704d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6e040c9540 tx=0x7f6df002ee60 comp rx=0 tx=0).stop 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3693315379 shutdown_connections 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 1 --2- 10.20.193.16:0/3693315379 >> v2:10.20.193.148:3300/0 conn(0x7f6e041536f0 0x7f6e04153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 1 --2- 10.20.193.16:0/3693315379 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e040700f0 0x7f6e040704d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3693315379 >> 10.20.193.16:0/3693315379 conn(0x7f6e040cd770 msgr2=0x7f6e040cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3693315379 shutdown_connections 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.710+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3693315379 wait complete. 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient: build_initial_monmap 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient: monmap: 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 librados: starting msgr at 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 librados: starting objecter 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 Processor -- start 2026-02-17T06:01:53.722 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 -- start start 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 librados: setting wanted keys 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 librados: calling monclient init 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient: init 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e040700f0 0x7f6e040cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient(hunting): picked mon.noname-c con 0x7f6e040700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e040c7fe0 0x7f6e041f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient(hunting): picked mon.noname-a con 0x7f6e040c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e041536f0 0x7f6e041f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient(hunting): picked mon.noname-b con 0x7f6e041536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:53.723 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e04154060 con 0x7f6e040c7fe0 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f6e040c5680 con 0x7f6e041536f0 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6e04074c00 con 0x7f6e040700f0 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.711+0000 7f6e0a4b7640 10 monclient(hunting): authenticate will time out at 1366.280976s 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e08a2d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e041536f0 0x7f6e041f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e03fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e040700f0 0x7f6e040cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e08a2d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e041536f0 0x7f6e041f2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43220/0 (socket says 10.20.193.16:43220) 2026-02-17T06:01:53.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e08a2d640 1 -- 10.20.193.16:0/3924485785 learned_addr learned my addr 10.20.193.16:0/3924485785 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e08a2d640 10 monclient(hunting): get_auth_request con 0x7f6e041536f0 auth_method 0 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e08a2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e08a2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e08a2d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e037fe640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e040c7fe0 0x7f6e041f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e03fff640 10 monclient(hunting): get_auth_request con 0x7f6e040700f0 auth_method 0 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e03fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e03fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e037fe640 10 monclient(hunting): get_auth_request con 0x7f6e040c7fe0 auth_method 0 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:53.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e08a2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e08a2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e08a2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.712+0000 7f6e037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.713+0000 7f6e08a2d640 10 monclient(hunting): handle_auth_done global_id 5119 payload 824 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.713+0000 7f6e08a2d640 1 -- 10.20.193.16:0/3924485785 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e040700f0 msgr2=0x7f6e040cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.713+0000 7f6e08a2d640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e040700f0 0x7f6e040cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.713+0000 7f6e08a2d640 1 -- 10.20.193.16:0/3924485785 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e040c7fe0 msgr2=0x7f6e041f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.713+0000 7f6e08a2d640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e040c7fe0 0x7f6e041f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.713+0000 7f6e08a2d640 10 monclient: _finish_hunting 0 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.713+0000 7f6e08a2d640 1 monclient: found mon.noname-b 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.713+0000 7f6e08a2d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.713+0000 7f6e08a2d640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.713+0000 7f6e08a2d640 1 -- 10.20.193.16:0/3924485785 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e041f15d0 con 0x7f6e041536f0 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.714+0000 7f6e08a2d640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e041536f0 0x7f6e041f2e90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6df800efc0 tx=0x7f6df800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.714+0000 7f6e00ff9640 10 client.?.objecter ms_handle_connect 0x7f6e041536f0 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.714+0000 7f6e00ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.715+0000 7f6e00ff9640 1 -- 10.20.193.16:0/3924485785 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8018020 con 0x7f6e041536f0 2026-02-17T06:01:53.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.715+0000 7f6e00ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 10 monclient: dump: 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 10 monclient: _finish_auth 0 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 5 monclient: authenticate success, global_id 5119 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 10 monclient: _renew_subs 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3924485785 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e041f18b0 con 0x7f6e041536f0 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 10 client.5119.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 10 monclient: _renew_subs 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3924485785 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e041f1e70 con 0x7f6e041536f0 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 1 librados: init done 2026-02-17T06:01:53.729 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 1 -- 10.20.193.16:0/3924485785 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df8014480 con 0x7f6e041536f0 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 1 -- 10.20.193.16:0/3924485785 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8014830 con 0x7f6e041536f0 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e00ff9640 10 monclient: dump: 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:53.730 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:53.731 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:53.731 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e027fc640 4 set_mon_vals no callback set 2026-02-17T06:01:53.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:53.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:53.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:53.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.716+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3924485785 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e040704d0 con 0x7f6e041536f0 2026-02-17T06:01:53.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.727+0000 7f6e00ff9640 1 -- 10.20.193.16:0/3924485785 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f6df80149d0 con 0x7f6e041536f0 2026-02-17T06:01:53.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.728+0000 7f6e00ff9640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dd4079140 0x7f6dd407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:53.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.728+0000 7f6e03fff640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dd4079140 0x7f6dd407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:53.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.728+0000 7f6e03fff640 10 monclient: get_auth_request con 0x7f6dd4079140 auth_method 0 2026-02-17T06:01:53.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.728+0000 7f6e03fff640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dd4079140 0x7f6dd407b600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e040c5b80 tx=0x7f6df0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:53.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.728+0000 7f6e00ff9640 10 client.5119.objecter ms_handle_connect 0x7f6dd4079140 2026-02-17T06:01:53.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.729+0000 7f6e00ff9640 1 -- 10.20.193.16:0/3924485785 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f6df805f390 con 0x7f6e041536f0 2026-02-17T06:01:53.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.729+0000 7f6e00ff9640 10 client.5119.objecter ms_dispatch2 0x7f6e04000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:53.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.729+0000 7f6e00ff9640 3 client.5119.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:53.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.729+0000 7f6e00ff9640 3 client.5119.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:53.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.729+0000 7f6e00ff9640 20 client.5119.objecter dump_active .. 0 homeless 2026-02-17T06:01:53.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.829+0000 7f6e00ff9640 1 -- 10.20.193.16:0/3924485785 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6df8022070 con 0x7f6e041536f0 2026-02-17T06:01:53.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.829+0000 7f6e00ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:53.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.829+0000 7f6e00ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:53.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.913+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3924485785 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e041573f0 con 0x7f6dd4079140 2026-02-17T06:01:53.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.914+0000 7f6e00ff9640 1 -- 10.20.193.16:0/3924485785 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44800 (secure 0 0 0) 0x7f6e041573f0 con 0x7f6dd4079140 2026-02-17T06:01:53.915 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:53.916 INFO:teuthology.orchestra.run.trial016.stderr:dumped all 2026-02-17T06:01:53.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.914+0000 7f6e0a4b7640 10 librados: watch_flush enter 2026-02-17T06:01:53.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.914+0000 7f6e0a4b7640 10 librados: watch_flush exit 2026-02-17T06:01:53.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.914+0000 7f6e0a4b7640 20 client.5119.objecter shutdown clearing up homeless session... 2026-02-17T06:01:53.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.914+0000 7f6e0a4b7640 10 client.5119.objecter successfully canceled tick 2026-02-17T06:01:53.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.914+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3924485785 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dd4079140 msgr2=0x7f6dd407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.914+0000 7f6e0a4b7640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dd4079140 0x7f6dd407b600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e040c5b80 tx=0x7f6df0033000 comp rx=0 tx=0).stop 2026-02-17T06:01:53.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.914+0000 7f6e0a4b7640 10 monclient: shutdown 2026-02-17T06:01:53.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.914+0000 7f6e0a4b7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:53.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.914+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3924485785 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e041536f0 msgr2=0x7f6e041f2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:53.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.915+0000 7f6e0a4b7640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e041536f0 0x7f6e041f2e90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6df800efc0 tx=0x7f6df800c490 comp rx=0 tx=0).stop 2026-02-17T06:01:53.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.915+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3924485785 shutdown_connections 2026-02-17T06:01:53.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.915+0000 7f6e0a4b7640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6dd4079140 0x7f6dd407b600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.915+0000 7f6e0a4b7640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f6e041536f0 0x7f6e041f2e90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.915+0000 7f6e0a4b7640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f6e040c7fe0 0x7f6e041f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.915+0000 7f6e0a4b7640 1 --2- 10.20.193.16:0/3924485785 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f6e040700f0 0x7f6e040cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:53.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.915+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3924485785 >> 10.20.193.16:0/3924485785 conn(0x7f6e040cd770 msgr2=0x7f6e0414dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:53.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.915+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3924485785 shutdown_connections 2026-02-17T06:01:53.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.915+0000 7f6e0a4b7640 1 -- 10.20.193.16:0/3924485785 wait complete. 2026-02-17T06:01:53.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.915+0000 7f6e0a4b7640 1 librados: shutdown 2026-02-17T06:01:53.924 INFO:teuthology.orchestra.run.trial016.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2026-02-17T06:01:51.925515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437976,"kb_used_data":6552,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509511464,"statfs":{"total":1546188226560,"available":1545739739136,"internally_reserved":0,"allocated":6709248,"data_stored":3434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25505,"internal_metadata":441424991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002029"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:57.003241+0000","last_change":"2026-02-17T05:56:45.946554+0000","last_active":"2026-02-17T05:58:57.003241+0000","last_peered":"2026-02-17T05:58:57.003241+0000","last_clean":"2026-02-17T05:58:57.003241+0000","last_became_active":"2026-02-17T05:56:43.852478+0000","last_became_peered":"2026-02-17T05:56:43.852478+0000","last_unstale":"2026-02-17T05:58:57.003241+0000","last_undegraded":"2026-02-17T05:58:57.003241+0000","last_fullsized":"2026-02-17T05:58:57.003241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T08:21:55.695169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031385049999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:54.575851+0000","last_change":"2026-02-17T05:56:46.700244+0000","last_active":"2026-02-17T05:58:54.575851+0000","last_peered":"2026-02-17T05:58:54.575851+0000","last_clean":"2026-02-17T05:58:54.575851+0000","last_became_active":"2026-02-17T05:56:43.182525+0000","last_became_peered":"2026-02-17T05:56:43.182525+0000","last_unstale":"2026-02-17T05:58:54.575851+0000","last_undegraded":"2026-02-17T05:58:54.575851+0000","last_fullsized":"2026-02-17T05:58:54.575851+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:38:44.190148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038112767999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:47.494561+0000","last_change":"2026-02-17T05:56:45.501489+0000","last_active":"2026-02-17T05:58:47.494561+0000","last_peered":"2026-02-17T05:58:47.494561+0000","last_clean":"2026-02-17T05:58:47.494561+0000","last_became_active":"2026-02-17T05:56:43.381684+0000","last_became_peered":"2026-02-17T05:56:43.381684+0000","last_unstale":"2026-02-17T05:58:47.494561+0000","last_undegraded":"2026-02-17T05:58:47.494561+0000","last_fullsized":"2026-02-17T05:58:47.494561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:07:04.503400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031884497999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:50.945819+0000","last_change":"2026-02-17T05:56:47.091287+0000","last_active":"2026-02-17T05:58:50.945819+0000","last_peered":"2026-02-17T05:58:50.945819+0000","last_clean":"2026-02-17T05:58:50.945819+0000","last_became_active":"2026-02-17T05:56:43.290452+0000","last_became_peered":"2026-02-17T05:56:43.290452+0000","last_unstale":"2026-02-17T05:58:50.945819+0000","last_undegraded":"2026-02-17T05:58:50.945819+0000","last_fullsized":"2026-02-17T05:58:50.945819+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:12:29.979912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032515719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"33'2","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:47.312723+0000","last_change":"2026-02-17T05:56:45.527362+0000","last_active":"2026-02-17T05:58:47.312723+0000","last_peered":"2026-02-17T05:58:47.312723+0000","last_clean":"2026-02-17T05:58:47.312723+0000","last_became_active":"2026-02-17T05:56:43.262979+0000","last_became_peered":"2026-02-17T05:56:43.262979+0000","last_unstale":"2026-02-17T05:58:47.312723+0000","last_undegraded":"2026-02-17T05:58:47.312723+0000","last_fullsized":"2026-02-17T05:58:47.312723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:30:08.337496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.056715723000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:50.698151+0000","last_change":"2026-02-17T05:56:46.695943+0000","last_active":"2026-02-17T05:58:50.698151+0000","last_peered":"2026-02-17T05:58:50.698151+0000","last_clean":"2026-02-17T05:58:50.698151+0000","last_became_active":"2026-02-17T05:56:43.075477+0000","last_became_peered":"2026-02-17T05:56:43.075477+0000","last_unstale":"2026-02-17T05:58:50.698151+0000","last_undegraded":"2026-02-17T05:58:50.698151+0000","last_fullsized":"2026-02-17T05:58:50.698151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T07:20:23.191397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.030611670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:42.310803+0000","last_change":"2026-02-17T05:56:45.522106+0000","last_active":"2026-02-17T05:58:42.310803+0000","last_peered":"2026-02-17T05:58:42.310803+0000","last_clean":"2026-02-17T05:58:42.310803+0000","last_became_active":"2026-02-17T05:56:43.142816+0000","last_became_peered":"2026-02-17T05:56:43.142816+0000","last_unstale":"2026-02-17T05:58:42.310803+0000","last_undegraded":"2026-02-17T05:58:42.310803+0000","last_fullsized":"2026-02-17T05:58:42.310803+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:47:19.171322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.043200879999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"31'1","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:50.430092+0000","last_change":"2026-02-17T05:56:46.694818+0000","last_active":"2026-02-17T05:58:50.430092+0000","last_peered":"2026-02-17T05:58:50.430092+0000","last_clean":"2026-02-17T05:58:50.430092+0000","last_became_active":"2026-02-17T05:56:43.121999+0000","last_became_peered":"2026-02-17T05:56:43.121999+0000","last_unstale":"2026-02-17T05:58:50.430092+0000","last_undegraded":"2026-02-17T05:58:50.430092+0000","last_fullsized":"2026-02-17T05:58:50.430092+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T17:12:50.512061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.034338280999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"32'32","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:35.692645+0000","last_change":"2026-02-17T05:56:37.927084+0000","last_active":"2026-02-17T05:58:35.692645+0000","last_peered":"2026-02-17T05:58:35.692645+0000","last_clean":"2026-02-17T05:58:35.692645+0000","last_became_active":"2026-02-17T05:56:37.919763+0000","last_became_peered":"2026-02-17T05:56:37.919763+0000","last_unstale":"2026-02-17T05:58:35.692645+0000","last_undegraded":"2026-02-17T05:58:35.692645+0000","last_fullsized":"2026-02-17T05:58:35.692645+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:30.809269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:30.809269+0000","last_clean_scrub_stamp":"2026-02-17T05:56:30.809269+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:55:23.546716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":28,"seq":120259084357,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":28,"seq":120259084356,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":27,"seq":115964117059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,3,4,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313284,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280579,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,4,5,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313284,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":372736,"data_stored":164912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[3,4,5,6,7,8,9,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":23,"seq":98784247874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":25,"seq":107374182470,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":23,"seq":98784247873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":372736,"data_stored":164912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[0,1,2,4,5,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":23,"seq":98784247873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":23,"seq":98784247874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,6,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":25,"seq":107374182466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,5,6,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":25,"seq":107374182466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":770048,"data_stored":562733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,4,5,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":27,"seq":115964117063,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":770048,"data_stored":562733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":27,"seq":115964117063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[1,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:01:53.927 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T06:01:53.996 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.995+0000 7fdd40091640 10 monclient: get_monmap_and_config 2026-02-17T06:01:53.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.995+0000 7fdd40091640 10 monclient: build_initial_monmap 2026-02-17T06:01:53.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.995+0000 7fdd40091640 10 monclient: monmap: 2026-02-17T06:01:53.997 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:53.997 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:53.997 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:53.996837+0000 2026-02-17T06:01:53.997 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:53.996837+0000 2026-02-17T06:01:53.997 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:53.997 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:53.997 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:53.998 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:53.998 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:53.998 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:53.998 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:53.998 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:53.998 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:53.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 1 Processor -- start 2026-02-17T06:01:53.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 1 -- start start 2026-02-17T06:01:53.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 10 monclient: init 2026-02-17T06:01:54.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:54.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 10 monclient: _add_conns ranks=[3,5,0,1,4,2] 2026-02-17T06:01:54.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fdd380700f0 0x7fdd380704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:54.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 10 monclient(hunting): picked mon.noname-a con 0x7fdd380700f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:54.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fdd380c7fe0 0x7fdd380c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:54.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 10 monclient(hunting): picked mon.noname-c con 0x7fdd380c7fe0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:54.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdd381536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:54.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd380c8970 con 0x7fdd381536f0 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fdd380c4820 con 0x7fdd380700f0 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fdd380c4b00 con 0x7fdd380c7fe0 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.996+0000 7fdd40091640 10 monclient(hunting): authenticate will time out at 1366.565810s 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3de06640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fdd380700f0 0x7fdd380704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3d605640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fdd380c7fe0 0x7fdd380c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3de06640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fdd380700f0 0x7fdd380704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33636/0 (socket says 10.20.193.16:33636) 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3de06640 1 -- 10.20.193.16:0/3170223272 learned_addr learned my addr 10.20.193.16:0/3170223272 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:54.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3de06640 10 monclient(hunting): get_auth_request con 0x7fdd380700f0 auth_method 0 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3de06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3de06640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3de06640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3d605640 10 monclient(hunting): get_auth_request con 0x7fdd380c7fe0 auth_method 0 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3d605640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3d605640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3d605640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3de06640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3de06640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3de06640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3d605640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3d605640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:54.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.997+0000 7fdd3d605640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:54.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.998+0000 7fdd3de06640 10 monclient(hunting): handle_auth_done global_id 5040 payload 274 2026-02-17T06:01:54.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.998+0000 7fdd3de06640 1 -- 10.20.193.16:0/3170223272 >> v2:10.20.193.148:3300/0 conn(0x7fdd380c7fe0 msgr2=0x7fdd380c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:54.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.998+0000 7fdd3de06640 1 --2- 10.20.193.16:0/3170223272 >> v2:10.20.193.148:3300/0 conn(0x7fdd380c7fe0 0x7fdd380c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:54.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.998+0000 7fdd3de06640 1 -- 10.20.193.16:0/3170223272 >> v1:10.20.193.16:6789/0 conn(0x7fdd381536f0 legacy=0x7fdd38153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:54.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.998+0000 7fdd3de06640 10 monclient: _finish_hunting 0 2026-02-17T06:01:54.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.998+0000 7fdd3de06640 1 monclient: found mon.noname-a 2026-02-17T06:01:54.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.998+0000 7fdd3de06640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:54.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.998+0000 7fdd3de06640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:54.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.998+0000 7fdd3de06640 1 -- 10.20.193.16:0/3170223272 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd380c4de0 con 0x7fdd380700f0 2026-02-17T06:01:54.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:53.999+0000 7fdd3de06640 1 --2- 10.20.193.16:0/3170223272 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd380700f0 0x7fdd380704d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd380c5b80 tx=0x7fdd2802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9648e688c2bd9c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:54.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 1 -- 10.20.193.16:0/3170223272 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2803c070 con 0x7fdd380700f0 2026-02-17T06:01:54.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:54.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:54.011 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 10 monclient: dump: 2026-02-17T06:01:54.011 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 10 monclient: _finish_auth 0 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 10 monclient: _check_auth_tickets 2026-02-17T06:01:54.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 1 -- 10.20.193.16:0/3170223272 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2802fa90 con 0x7fdd380700f0 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 5 monclient: authenticate success, global_id 5040 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 1 -- 10.20.193.16:0/3170223272 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2802fe60 con 0x7fdd380700f0 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd3ce04640 10 monclient: dump: 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:54.013 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 4 set_mon_vals no callback set 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 10 monclient: shutdown 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 1 -- 10.20.193.16:0/3170223272 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd380700f0 msgr2=0x7fdd380704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:54.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 1 --2- 10.20.193.16:0/3170223272 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd380700f0 0x7fdd380704d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd380c5b80 tx=0x7fdd2802ee60 comp rx=0 tx=0).stop 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 1 -- 10.20.193.16:0/3170223272 shutdown_connections 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 1 --2- 10.20.193.16:0/3170223272 >> v2:10.20.193.148:3300/0 conn(0x7fdd380c7fe0 0x7fdd380c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 1 --2- 10.20.193.16:0/3170223272 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd380700f0 0x7fdd380704d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 1 -- 10.20.193.16:0/3170223272 >> 10.20.193.16:0/3170223272 conn(0x7fdd380cd770 msgr2=0x7fdd380cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 1 -- 10.20.193.16:0/3170223272 shutdown_connections 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 1 -- 10.20.193.16:0/3170223272 wait complete. 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 10 monclient: build_initial_monmap 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.002+0000 7fdd40091640 10 monclient: monmap: 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:54.015 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 librados: starting msgr at 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 librados: starting objecter 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 Processor -- start 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 -- start start 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 librados: setting wanted keys 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 librados: calling monclient init 2026-02-17T06:01:54.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient: init 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd380700f0 0x7fdd38073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient(hunting): picked mon.noname-b con 0x7fdd380700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdd380c7fe0 0x7fdd380cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient(hunting): picked mon.noname-c con 0x7fdd380c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd381536f0 0x7fdd380cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient(hunting): picked mon.noname-a con 0x7fdd381536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd380c5680 con 0x7fdd381536f0 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fdd380c53a0 con 0x7fdd380700f0 2026-02-17T06:01:54.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdd38074c00 con 0x7fdd380c7fe0 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd40091640 10 monclient(hunting): authenticate will time out at 1366.572563s 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd3e607640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd381536f0 0x7fdd380cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd3de06640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd380700f0 0x7fdd38073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd3e607640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd381536f0 0x7fdd380cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33638/0 (socket says 10.20.193.16:33638) 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd3e607640 1 -- 10.20.193.16:0/3345981218 learned_addr learned my addr 10.20.193.16:0/3345981218 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd3e607640 10 monclient(hunting): get_auth_request con 0x7fdd381536f0 auth_method 0 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd3e607640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd3e607640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:54.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.003+0000 7fdd3e607640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3d605640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdd380c7fe0 0x7fdd380cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3de06640 10 monclient(hunting): get_auth_request con 0x7fdd380700f0 auth_method 0 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3de06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3de06640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3de06640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3e607640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3e607640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3e607640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3d605640 10 monclient(hunting): get_auth_request con 0x7fdd380c7fe0 auth_method 0 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3d605640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3d605640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3d605640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:54.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3de06640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3de06640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3de06640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3d605640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3d605640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.004+0000 7fdd3d605640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.005+0000 7fdd3e607640 10 monclient(hunting): handle_auth_done global_id 5043 payload 824 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.005+0000 7fdd3e607640 1 -- 10.20.193.16:0/3345981218 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdd380c7fe0 msgr2=0x7fdd380cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.005+0000 7fdd3e607640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdd380c7fe0 0x7fdd380cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.005+0000 7fdd3e607640 1 -- 10.20.193.16:0/3345981218 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd380700f0 msgr2=0x7fdd38073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.005+0000 7fdd3e607640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd380700f0 0x7fdd38073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.005+0000 7fdd3e607640 10 monclient: _finish_hunting 0 2026-02-17T06:01:54.020 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.005+0000 7fdd3e607640 1 monclient: found mon.noname-a 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.005+0000 7fdd3e607640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.005+0000 7fdd3e607640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.005+0000 7fdd3e607640 1 -- 10.20.193.16:0/3345981218 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd381eaff0 con 0x7fdd381536f0 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.006+0000 7fdd3e607640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd381536f0 0x7fdd380cbb20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400b990 tx=0x7fdd3400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.006+0000 7fdd267fc640 10 client.?.objecter ms_handle_connect 0x7fdd381536f0 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.007+0000 7fdd267fc640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 1 -- 10.20.193.16:0/3345981218 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34006e60 con 0x7fdd381536f0 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 10 monclient: dump: 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:54.021 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 1 -- 10.20.193.16:0/3345981218 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd34010070 con 0x7fdd381536f0 2026-02-17T06:01:54.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd40091640 5 monclient: authenticate success, global_id 5043 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd40091640 10 monclient: _renew_subs 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd40091640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd40091640 1 -- 10.20.193.16:0/3345981218 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd381f3660 con 0x7fdd381536f0 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 1 -- 10.20.193.16:0/3345981218 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3400c9d0 con 0x7fdd381536f0 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd267fc640 10 monclient: dump: 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:54.023 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.008+0000 7fdd27fff640 4 set_mon_vals no callback set 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.009+0000 7fdd40091640 10 client.5043.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.009+0000 7fdd40091640 10 monclient: _renew_subs 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.009+0000 7fdd40091640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.009+0000 7fdd40091640 1 -- 10.20.193.16:0/3345981218 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd381f3c20 con 0x7fdd381536f0 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.009+0000 7fdd40091640 1 librados: init done 2026-02-17T06:01:54.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.009+0000 7fdd40091640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:54.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.009+0000 7fdd40091640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:54.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.009+0000 7fdd40091640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:54.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.009+0000 7fdd40091640 1 -- 10.20.193.16:0/3345981218 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd380704d0 con 0x7fdd381536f0 2026-02-17T06:01:54.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.021+0000 7fdd267fc640 1 -- 10.20.193.16:0/3345981218 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fdd3400cb70 con 0x7fdd381536f0 2026-02-17T06:01:54.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.021+0000 7fdd267fc640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd08079190 0x7fdd0807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:54.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.021+0000 7fdd3de06640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd08079190 0x7fdd0807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:54.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.021+0000 7fdd3de06640 10 monclient: get_auth_request con 0x7fdd08079190 auth_method 0 2026-02-17T06:01:54.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.021+0000 7fdd267fc640 10 client.5043.objecter ms_handle_connect 0x7fdd08079190 2026-02-17T06:01:54.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.021+0000 7fdd3de06640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd08079190 0x7fdd0807b650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdd380c9b90 tx=0x7fdd28033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:54.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.022+0000 7fdd267fc640 1 -- 10.20.193.16:0/3345981218 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fdd3405dc90 con 0x7fdd381536f0 2026-02-17T06:01:54.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.022+0000 7fdd267fc640 10 client.5043.objecter ms_dispatch2 0x7fdd38000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:54.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.022+0000 7fdd267fc640 3 client.5043.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:54.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.022+0000 7fdd267fc640 3 client.5043.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:54.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.022+0000 7fdd267fc640 20 client.5043.objecter dump_active .. 0 homeless 2026-02-17T06:01:54.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.124+0000 7fdd267fc640 1 -- 10.20.193.16:0/3345981218 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdd34061c50 con 0x7fdd381536f0 2026-02-17T06:01:54.124 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.124+0000 7fdd267fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:54.125 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.124+0000 7fdd267fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:54.207 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.207+0000 7fdd40091640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T06:01:54.207 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.207+0000 7fdd40091640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T06:01:54.207 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.207+0000 7fdd40091640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:54.207 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.207+0000 7fdd40091640 1 -- 10.20.193.16:0/3345981218 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd381573f0 con 0x7fdd381536f0 2026-02-17T06:01:54.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.227+0000 7fdd267fc640 1 -- 10.20.193.16:0/3345981218 <== mon.0 v2:10.20.193.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+24724 (secure 0 0 0) 0x7fdd34065f50 con 0x7fdd381536f0 2026-02-17T06:01:54.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.227+0000 7fdd267fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T06:01:54.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.227+0000 7fdd267fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:54.228 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:01:54.228 INFO:teuthology.orchestra.run.trial016.stdout:{"epoch":34,"fsid":"bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6","created":"2026-02-17T05:54:49.129963+0000","modified":"2026-02-17T05:56:53.710237+0000","last_up_change":"2026-02-17T05:56:37.615172+0000","last_in_change":"2026-02-17T05:54:55.671400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:56:30.047883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-17T05:56:41.588944+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":33,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0100002288818359,"score_stable":4.0100002288818359,"optimal_score":0.81000000238418579,"raw_score_acting":3.25,"raw_score_stable":3.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1fd07ff-606a-4763-9d8d-5b703e21032a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6826","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6827","nonce":821011033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6828","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6829","nonce":821011033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6832","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6833","nonce":821011033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6830","nonce":821011033},{"type":"v1","addr":"10.20.193.16:6831","nonce":821011033}]},"public_addr":"10.20.193.16:6827/821011033","cluster_addr":"10.20.193.16:6829/821011033","heartbeat_back_addr":"10.20.193.16:6833/821011033","heartbeat_front_addr":"10.20.193.16:6831/821011033","state":["exists","up"]},{"osd":1,"uuid":"7ef10114-33c6-4bb2-a822-d67cb7cdf725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6807","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6809","nonce":1618049789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6810","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6811","nonce":1618049789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6814","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6815","nonce":1618049789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6812","nonce":1618049789},{"type":"v1","addr":"10.20.193.128:6813","nonce":1618049789}]},"public_addr":"10.20.193.128:6809/1618049789","cluster_addr":"10.20.193.128:6811/1618049789","heartbeat_back_addr":"10.20.193.128:6815/1618049789","heartbeat_front_addr":"10.20.193.128:6813/1618049789","state":["exists","up"]},{"osd":2,"uuid":"a077772d-ffbf-4354-bc37-84706b589239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6816","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6817","nonce":1588035471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6818","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6819","nonce":1588035471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6822","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6823","nonce":1588035471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6820","nonce":1588035471},{"type":"v1","addr":"10.20.193.148:6821","nonce":1588035471}]},"public_addr":"10.20.193.148:6817/1588035471","cluster_addr":"10.20.193.148:6819/1588035471","heartbeat_back_addr":"10.20.193.148:6823/1588035471","heartbeat_front_addr":"10.20.193.148:6821/1588035471","state":["exists","up"]},{"osd":3,"uuid":"c82441ef-121c-4d28-a591-eb6ef81200c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6824","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6825","nonce":3772404125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6826","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6827","nonce":3772404125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6830","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6831","nonce":3772404125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6828","nonce":3772404125},{"type":"v1","addr":"10.20.193.164:6829","nonce":3772404125}]},"public_addr":"10.20.193.164:6825/3772404125","cluster_addr":"10.20.193.164:6827/3772404125","heartbeat_back_addr":"10.20.193.164:6831/3772404125","heartbeat_front_addr":"10.20.193.164:6829/3772404125","state":["exists","up"]},{"osd":4,"uuid":"5bf489c2-40d8-490c-9c10-34a9c6fc407c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6818","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6819","nonce":1157611076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6820","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6821","nonce":1157611076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6824","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6825","nonce":1157611076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6822","nonce":1157611076},{"type":"v1","addr":"10.20.193.16:6823","nonce":1157611076}]},"public_addr":"10.20.193.16:6819/1157611076","cluster_addr":"10.20.193.16:6821/1157611076","heartbeat_back_addr":"10.20.193.16:6825/1157611076","heartbeat_front_addr":"10.20.193.16:6823/1157611076","state":["exists","up"]},{"osd":5,"uuid":"3b254810-f671-42ee-8ae9-45fd888d08a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6824","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6825","nonce":1593443319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6826","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6827","nonce":1593443319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6830","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6831","nonce":1593443319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6828","nonce":1593443319},{"type":"v1","addr":"10.20.193.128:6829","nonce":1593443319}]},"public_addr":"10.20.193.128:6825/1593443319","cluster_addr":"10.20.193.128:6827/1593443319","heartbeat_back_addr":"10.20.193.128:6831/1593443319","heartbeat_front_addr":"10.20.193.128:6829/1593443319","state":["exists","up"]},{"osd":6,"uuid":"b8592e91-c8a9-49a0-846c-519306dc7064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6824","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6825","nonce":2491860138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6826","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6827","nonce":2491860138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6830","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6831","nonce":2491860138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6828","nonce":2491860138},{"type":"v1","addr":"10.20.193.148:6829","nonce":2491860138}]},"public_addr":"10.20.193.148:6825/2491860138","cluster_addr":"10.20.193.148:6827/2491860138","heartbeat_back_addr":"10.20.193.148:6831/2491860138","heartbeat_front_addr":"10.20.193.148:6829/2491860138","state":["exists","up"]},{"osd":7,"uuid":"00efccbd-39d3-4135-8978-22d7614c94ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6816","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6817","nonce":3942637745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6818","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6819","nonce":3942637745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6822","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6823","nonce":3942637745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6820","nonce":3942637745},{"type":"v1","addr":"10.20.193.164:6821","nonce":3942637745}]},"public_addr":"10.20.193.164:6817/3942637745","cluster_addr":"10.20.193.164:6819/3942637745","heartbeat_back_addr":"10.20.193.164:6823/3942637745","heartbeat_front_addr":"10.20.193.164:6821/3942637745","state":["exists","up"]},{"osd":8,"uuid":"540bfc0b-4904-4286-b210-1c06f78cd6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6802","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6803","nonce":15364298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6804","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6805","nonce":15364298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6808","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6809","nonce":15364298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6806","nonce":15364298},{"type":"v1","addr":"10.20.193.16:6807","nonce":15364298}]},"public_addr":"10.20.193.16:6803/15364298","cluster_addr":"10.20.193.16:6805/15364298","heartbeat_back_addr":"10.20.193.16:6809/15364298","heartbeat_front_addr":"10.20.193.16:6807/15364298","state":["exists","up"]},{"osd":9,"uuid":"dd5ed63a-41d2-4afc-86dd-7834e0c16448","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6800","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6801","nonce":1813385411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6802","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6803","nonce":1813385411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6806","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6808","nonce":1813385411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6804","nonce":1813385411},{"type":"v1","addr":"10.20.193.128:6805","nonce":1813385411}]},"public_addr":"10.20.193.128:6801/1813385411","cluster_addr":"10.20.193.128:6803/1813385411","heartbeat_back_addr":"10.20.193.128:6808/1813385411","heartbeat_front_addr":"10.20.193.128:6805/1813385411","state":["exists","up"]},{"osd":10,"uuid":"d425da5d-7acd-48f9-abf7-fb4896df492d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6808","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6809","nonce":2531105113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6810","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6811","nonce":2531105113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6814","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6815","nonce":2531105113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6812","nonce":2531105113},{"type":"v1","addr":"10.20.193.148:6813","nonce":2531105113}]},"public_addr":"10.20.193.148:6809/2531105113","cluster_addr":"10.20.193.148:6811/2531105113","heartbeat_back_addr":"10.20.193.148:6815/2531105113","heartbeat_front_addr":"10.20.193.148:6813/2531105113","state":["exists","up"]},{"osd":11,"uuid":"32bc4c0f-3014-4c48-b6c7-f1f09db76d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6808","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6809","nonce":188912883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6810","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6811","nonce":188912883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6814","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6815","nonce":188912883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6812","nonce":188912883},{"type":"v1","addr":"10.20.193.164:6813","nonce":188912883}]},"public_addr":"10.20.193.164:6809/188912883","cluster_addr":"10.20.193.164:6811/188912883","heartbeat_back_addr":"10.20.193.164:6815/188912883","heartbeat_front_addr":"10.20.193.164:6813/188912883","state":["exists","up"]},{"osd":12,"uuid":"9e9e82f6-9b4c-44cd-a18e-7089cc27b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6810","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6811","nonce":3960967243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6812","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6813","nonce":3960967243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6816","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6817","nonce":3960967243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.16:6814","nonce":3960967243},{"type":"v1","addr":"10.20.193.16:6815","nonce":3960967243}]},"public_addr":"10.20.193.16:6811/3960967243","cluster_addr":"10.20.193.16:6813/3960967243","heartbeat_back_addr":"10.20.193.16:6817/3960967243","heartbeat_front_addr":"10.20.193.16:6815/3960967243","state":["exists","up"]},{"osd":13,"uuid":"e1bec267-a188-44ef-952c-6b9a39a2265a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6816","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6817","nonce":1434515412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6818","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6819","nonce":1434515412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6822","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6823","nonce":1434515412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.128:6820","nonce":1434515412},{"type":"v1","addr":"10.20.193.128:6821","nonce":1434515412}]},"public_addr":"10.20.193.128:6817/1434515412","cluster_addr":"10.20.193.128:6819/1434515412","heartbeat_back_addr":"10.20.193.128:6823/1434515412","heartbeat_front_addr":"10.20.193.128:6821/1434515412","state":["exists","up"]},{"osd":14,"uuid":"c733624a-c696-405a-a41d-17481c672857","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6800","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6801","nonce":1855767579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6802","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6803","nonce":1855767579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6806","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6807","nonce":1855767579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.148:6804","nonce":1855767579},{"type":"v1","addr":"10.20.193.148:6805","nonce":1855767579}]},"public_addr":"10.20.193.148:6801/1855767579","cluster_addr":"10.20.193.148:6803/1855767579","heartbeat_back_addr":"10.20.193.148:6807/1855767579","heartbeat_front_addr":"10.20.193.148:6805/1855767579","state":["exists","up"]},{"osd":15,"uuid":"8fa42467-586f-4e64-b428-9afd5dd14b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6800","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6801","nonce":1881720827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6802","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6803","nonce":1881720827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6806","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6807","nonce":1881720827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.164:6804","nonce":1881720827},{"type":"v1","addr":"10.20.193.164:6805","nonce":1881720827}]},"public_addr":"10.20.193.164:6801/1881720827","cluster_addr":"10.20.193.164:6803/1881720827","heartbeat_back_addr":"10.20.193.164:6807/1881720827","heartbeat_front_addr":"10.20.193.164:6805/1881720827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.650447+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.248604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:27.916170+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.151671+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.881661+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.409956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:28.238512+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.558291+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.920088+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.224894+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:27.660706+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.933443+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:31.293415+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:30.603293+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:28.448673+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:56:29.312258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.228+0000 7fdd40091640 10 librados: watch_flush enter 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.228+0000 7fdd40091640 10 librados: watch_flush exit 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.228+0000 7fdd40091640 20 client.5043.objecter shutdown clearing up homeless session... 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.228+0000 7fdd40091640 10 client.5043.objecter successfully canceled tick 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.228+0000 7fdd40091640 1 -- 10.20.193.16:0/3345981218 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd08079190 msgr2=0x7fdd0807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.228+0000 7fdd40091640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd08079190 0x7fdd0807b650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdd380c9b90 tx=0x7fdd28033000 comp rx=0 tx=0).stop 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.228+0000 7fdd40091640 10 monclient: shutdown 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.228+0000 7fdd40091640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.228+0000 7fdd40091640 1 -- 10.20.193.16:0/3345981218 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd381536f0 msgr2=0x7fdd380cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.228+0000 7fdd40091640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd381536f0 0x7fdd380cbb20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400b990 tx=0x7fdd3400be60 comp rx=0 tx=0).stop 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.229+0000 7fdd40091640 1 -- 10.20.193.16:0/3345981218 shutdown_connections 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.229+0000 7fdd40091640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fdd08079190 0x7fdd0807b650 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.229+0000 7fdd40091640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fdd381536f0 0x7fdd380cbb20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.229+0000 7fdd40091640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fdd380c7fe0 0x7fdd380cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.229+0000 7fdd40091640 1 --2- 10.20.193.16:0/3345981218 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fdd380700f0 0x7fdd38073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:54.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.229+0000 7fdd40091640 1 -- 10.20.193.16:0/3345981218 >> 10.20.193.16:0/3345981218 conn(0x7fdd380cd770 msgr2=0x7fdd38152450 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:54.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.229+0000 7fdd40091640 1 -- 10.20.193.16:0/3345981218 shutdown_connections 2026-02-17T06:01:54.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.229+0000 7fdd40091640 1 -- 10.20.193.16:0/3345981218 wait complete. 2026-02-17T06:01:54.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:54.229+0000 7fdd40091640 1 librados: shutdown 2026-02-17T06:01:55.238 INFO:tasks.ceph:Scrubbing osd.0 2026-02-17T06:01:55.239 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:55.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.310+0000 7fe68e13e640 10 monclient: get_monmap_and_config 2026-02-17T06:01:55.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.310+0000 7fe68e13e640 10 monclient: build_initial_monmap 2026-02-17T06:01:55.312 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.310+0000 7fe68e13e640 10 monclient: monmap: 2026-02-17T06:01:55.312 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:55.312 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.312 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:55.311955+0000 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:55.311955+0000 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.311+0000 7fe68e13e640 1 Processor -- start 2026-02-17T06:01:55.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.311+0000 7fe68e13e640 1 -- start start 2026-02-17T06:01:55.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.311+0000 7fe68e13e640 10 monclient: init 2026-02-17T06:01:55.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.311+0000 7fe68e13e640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:55.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 10 monclient: _add_conns ranks=[5,4,0,3,1,2] 2026-02-17T06:01:55.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fe688155ce0 0x7fe68806e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 10 monclient(hunting): picked mon.noname-c con 0x7fe688155ce0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:55.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fe6881560c0 0x7fe68806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 10 monclient(hunting): picked mon.noname-b con 0x7fe6881560c0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:55.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe6880703e0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:55.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68806cbb0 con 0x7fe6880703e0 2026-02-17T06:01:55.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fe68806ce90 con 0x7fe6881560c0 2026-02-17T06:01:55.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe68806d170 con 0x7fe688155ce0 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe68e13e640 10 monclient(hunting): authenticate will time out at 1367.881074s 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe687fff640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fe6880703e0 0x7fe6880c9f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:41820/0 (socket says 10.20.193.16:41820) 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe687fff640 1 -- 10.20.193.16:0/3806759254 learned_addr learned my addr 10.20.193.16:0/3806759254 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe6877fe640 1 --2- 10.20.193.16:0/3806759254 >> v2:10.20.193.148:3300/0 conn(0x7fe688155ce0 0x7fe68806e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe686ffd640 1 --2- 10.20.193.16:0/3806759254 >> v2:10.20.193.128:3300/0 conn(0x7fe6881560c0 0x7fe68806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe686ffd640 10 monclient(hunting): get_auth_request con 0x7fe6881560c0 auth_method 0 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe686ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe686ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe686ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe6877fe640 10 monclient(hunting): get_auth_request con 0x7fe688155ce0 auth_method 0 2026-02-17T06:01:55.324 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe6877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe6877fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.312+0000 7fe6877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe686ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe686ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe686ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6867fc640 1 -- 10.20.193.16:0/3806759254 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1222523314 0 0) 0x7fe68806cbb0 con 0x7fe6880703e0 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6867fc640 10 monclient(hunting): _negotiate 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6867fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6867fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6867fc640 10 monclient(hunting): my global_id is 5046 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6867fc640 1 -- 10.20.193.16:0/3806759254 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe66c003620 con 0x7fe6880703e0 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 10 monclient(hunting): handle_auth_done global_id 5186 payload 274 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 1 -- 10.20.193.16:0/3806759254 >> v2:10.20.193.128:3300/0 conn(0x7fe6881560c0 msgr2=0x7fe68806fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 1 --2- 10.20.193.16:0/3806759254 >> v2:10.20.193.128:3300/0 conn(0x7fe6881560c0 0x7fe68806fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 1 -- 10.20.193.16:0/3806759254 >> v1:10.20.193.16:6789/0 conn(0x7fe6880703e0 legacy=0x7fe6880c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 1 monclient: found mon.noname-c 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.313+0000 7fe6877fe640 1 -- 10.20.193.16:0/3806759254 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68806d450 con 0x7fe688155ce0 2026-02-17T06:01:55.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.314+0000 7fe6877fe640 1 --2- 10.20.193.16:0/3806759254 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe688155ce0 0x7fe68806e430 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe688072d40 tx=0x7fe67402eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaca1f8dcc9543c7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 1 -- 10.20.193.16:0/3806759254 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67403c070 con 0x7fe688155ce0 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 10 monclient: dump: 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.327 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 1 -- 10.20.193.16:0/3806759254 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67402fb80 con 0x7fe688155ce0 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 1 -- 10.20.193.16:0/3806759254 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674037440 con 0x7fe688155ce0 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe68e13e640 5 monclient: authenticate success, global_id 5186 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe68e13e640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:55.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe68e13e640 4 set_mon_vals no callback set 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe6867fc640 10 monclient: dump: 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.329 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe68e13e640 10 monclient: shutdown 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe68e13e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe68e13e640 1 -- 10.20.193.16:0/3806759254 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe688155ce0 msgr2=0x7fe68806e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe68e13e640 1 --2- 10.20.193.16:0/3806759254 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe688155ce0 0x7fe68806e430 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe688072d40 tx=0x7fe67402eeb0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe68e13e640 1 -- 10.20.193.16:0/3806759254 shutdown_connections 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe68e13e640 1 --2- 10.20.193.16:0/3806759254 >> v2:10.20.193.128:3300/0 conn(0x7fe6881560c0 0x7fe68806fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.316+0000 7fe68e13e640 1 --2- 10.20.193.16:0/3806759254 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe688155ce0 0x7fe68806e430 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 1 -- 10.20.193.16:0/3806759254 >> 10.20.193.16:0/3806759254 conn(0x7fe6880cf600 msgr2=0x7fe6880cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 1 -- 10.20.193.16:0/3806759254 shutdown_connections 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 1 -- 10.20.193.16:0/3806759254 wait complete. 2026-02-17T06:01:55.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 10 monclient: build_initial_monmap 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 10 monclient: monmap: 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 1 librados: starting msgr at 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 1 librados: starting objecter 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 1 Processor -- start 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 1 -- start start 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 1 librados: setting wanted keys 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 1 librados: calling monclient init 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.317+0000 7fe68e13e640 10 monclient: init 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe6880703e0 0x7fe6881f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 10 monclient(hunting): picked mon.noname-c con 0x7fe6880703e0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe688155ce0 0x7fe6881e9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 10 monclient(hunting): picked mon.noname-a con 0x7fe688155ce0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe6881560c0 0x7fe6880c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 10 monclient(hunting): picked mon.noname-b con 0x7fe6881560c0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe68806dcf0 con 0x7fe688155ce0 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fe68806da10 con 0x7fe6881560c0 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe68806c870 con 0x7fe6880703e0 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe68e13e640 10 monclient(hunting): authenticate will time out at 1367.887223s 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe686ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe688155ce0 0x7fe6881e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe6877fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe6880703e0 0x7fe6881f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe687fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe6881560c0 0x7fe6880c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe686ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe688155ce0 0x7fe6881e9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33640/0 (socket says 10.20.193.16:33640) 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe686ffd640 1 -- 10.20.193.16:0/2322591858 learned_addr learned my addr 10.20.193.16:0/2322591858 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe686ffd640 10 monclient(hunting): get_auth_request con 0x7fe688155ce0 auth_method 0 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe686ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe686ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe686ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe6877fe640 10 monclient(hunting): get_auth_request con 0x7fe6880703e0 auth_method 0 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe6877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe6877fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe6877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe687fff640 10 monclient(hunting): get_auth_request con 0x7fe6881560c0 auth_method 0 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe687fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe687fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe687fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe686ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe686ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe686ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe6877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe6877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.318+0000 7fe6877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.319+0000 7fe687fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.319+0000 7fe687fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.319+0000 7fe687fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.320+0000 7fe686ffd640 10 monclient(hunting): handle_auth_done global_id 5049 payload 824 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.320+0000 7fe686ffd640 1 -- 10.20.193.16:0/2322591858 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe6880703e0 msgr2=0x7fe6881f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.320+0000 7fe686ffd640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe6880703e0 0x7fe6881f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.320+0000 7fe686ffd640 1 -- 10.20.193.16:0/2322591858 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe6881560c0 msgr2=0x7fe6880c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.320+0000 7fe686ffd640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe6881560c0 0x7fe6880c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.320+0000 7fe686ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.320+0000 7fe686ffd640 1 monclient: found mon.noname-a 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.320+0000 7fe686ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.320+0000 7fe686ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.320+0000 7fe686ffd640 1 -- 10.20.193.16:0/2322591858 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6881f4a50 con 0x7fe688155ce0 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.321+0000 7fe686ffd640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe688155ce0 0x7fe6881e9680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe67800ec10 tx=0x7fe67800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.321+0000 7fe663fff640 10 client.?.objecter ms_handle_connect 0x7fe688155ce0 2026-02-17T06:01:55.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.321+0000 7fe663fff640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:55.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 1 -- 10.20.193.16:0/2322591858 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678017070 con 0x7fe688155ce0 2026-02-17T06:01:55.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 10 monclient: dump: 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.341 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 10 monclient: _finish_auth 0 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 1 -- 10.20.193.16:0/2322591858 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67800cc40 con 0x7fe688155ce0 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 1 -- 10.20.193.16:0/2322591858 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678013470 con 0x7fe688155ce0 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe68e13e640 5 monclient: authenticate success, global_id 5049 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe68e13e640 10 monclient: _renew_subs 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe685ffb640 4 set_mon_vals no callback set 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe68e13e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:55.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6880c3c70 con 0x7fe688155ce0 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe663fff640 10 monclient: dump: 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.343 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe68e13e640 10 client.5049.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe68e13e640 10 monclient: _renew_subs 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe68e13e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6880c4230 con 0x7fe688155ce0 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.322+0000 7fe68e13e640 1 librados: init done 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.323+0000 7fe68e13e640 10 client.5049.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.323+0000 7fe68e13e640 15 client.5049.objecter _session_command_op_assign -1 1 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.323+0000 7fe68e13e640 20 client.5049.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.323+0000 7fe68e13e640 10 client.5049.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.323+0000 7fe68e13e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.323+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe68806fea0 con 0x7fe688155ce0 2026-02-17T06:01:55.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.334+0000 7fe663fff640 1 -- 10.20.193.16:0/2322591858 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fe678013630 con 0x7fe688155ce0 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.335+0000 7fe663fff640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe658079190 0x7fe65807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.335+0000 7fe6877fe640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe658079190 0x7fe65807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.335+0000 7fe6877fe640 10 monclient: get_auth_request con 0x7fe658079190 auth_method 0 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.335+0000 7fe6877fe640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe658079190 0x7fe65807b650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe674039000 tx=0x7fe674033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.335+0000 7fe663fff640 10 client.5049.objecter ms_handle_connect 0x7fe658079190 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 1 -- 10.20.193.16:0/2322591858 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fe67805e6e0 con 0x7fe688155ce0 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 10 client.5049.objecter ms_dispatch2 0x7fe688000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 3 client.5049.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 3 client.5049.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 10 client.5049.objecter checking command 1 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] conn(0x7fe6580816d0 0x7fe658083af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 20 client.5049.objecter _get_session s=0x7fe658081000 osd=0 3 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 20 client.5049.objecter put_session s=0x7fe658081000 osd=0 3 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 15 client.5049.objecter _session_command_op_remove -1 1 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 20 client.5049.objecter _get_session s=0x7fe658081000 osd=0 3 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 20 client.5049.objecter get_session s=0x7fe658081000 osd=0 3 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 15 client.5049.objecter _session_command_op_assign 0 1 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 20 client.5049.objecter put_session s=0x7fe658081000 osd=0 4 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 10 client.5049.objecter _send_command 1 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 1 -- 10.20.193.16:0/2322591858 --> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6580841c0 con 0x7fe6580816d0 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.336+0000 7fe663fff640 20 client.5049.objecter dump_active .. 0 homeless 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.337+0000 7fe687fff640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] conn(0x7fe6580816d0 0x7fe658083af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.337+0000 7fe687fff640 10 monclient: get_auth_request con 0x7fe6580816d0 auth_method 0 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.337+0000 7fe663fff640 1 -- 10.20.193.16:0/2322591858 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe67805eaa0 con 0x7fe688155ce0 2026-02-17T06:01:55.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.337+0000 7fe663fff640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:55.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.338+0000 7fe687fff640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] conn(0x7fe6580816d0 0x7fe658083af0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.338+0000 7fe663fff640 10 client.5049.objecter ms_handle_connect 0x7fe6580816d0 2026-02-17T06:01:55.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.348+0000 7fe663fff640 1 -- 10.20.193.16:0/2322591858 <== osd.0 v2:10.20.193.16:6826/821011033 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fe6580841c0 con 0x7fe6580816d0 2026-02-17T06:01:55.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.348+0000 7fe663fff640 10 client.5049.objecter ms_dispatch2 0x7fe688000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:55.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.348+0000 7fe663fff640 10 client.5049.objecter _finish_command 1 = system:0 2026-02-17T06:01:55.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.348+0000 7fe663fff640 20 client.5049.objecter put_session s=0x7fe658081000 osd=0 4 2026-02-17T06:01:55.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.348+0000 7fe663fff640 15 client.5049.objecter _session_command_op_remove 0 1 2026-02-17T06:01:55.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 10 client.5049.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:55.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 15 client.5049.objecter _session_command_op_assign -1 2 2026-02-17T06:01:55.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 20 client.5049.objecter _get_session s=0x7fe658081000 osd=0 3 2026-02-17T06:01:55.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 20 client.5049.objecter put_session s=0x7fe658081000 osd=0 3 2026-02-17T06:01:55.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 20 client.5049.objecter _get_session s=0x7fe658081000 osd=0 3 2026-02-17T06:01:55.355 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 15 client.5049.objecter _session_command_op_remove -1 2 2026-02-17T06:01:55.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 20 client.5049.objecter get_session s=0x7fe658081000 osd=0 3 2026-02-17T06:01:55.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 15 client.5049.objecter _session_command_op_assign 0 2 2026-02-17T06:01:55.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 20 client.5049.objecter put_session s=0x7fe658081000 osd=0 4 2026-02-17T06:01:55.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 10 client.5049.objecter _send_command 2 2026-02-17T06:01:55.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.354+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 --> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe6881596c0 con 0x7fe6580816d0 2026-02-17T06:01:55.362 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe663fff640 1 -- 10.20.193.16:0/2322591858 <== osd.0 v2:10.20.193.16:6826/821011033 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe6881596c0 con 0x7fe6580816d0 2026-02-17T06:01:55.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe663fff640 10 client.5049.objecter ms_dispatch2 0x7fe688000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:55.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe663fff640 10 client.5049.objecter _finish_command 2 = system:0 2026-02-17T06:01:55.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe663fff640 20 client.5049.objecter put_session s=0x7fe658081000 osd=0 4 2026-02-17T06:01:55.363 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe663fff640 15 client.5049.objecter _session_command_op_remove 0 2 2026-02-17T06:01:55.364 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:55.364 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:55.364 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:55.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 10 librados: watch_flush enter 2026-02-17T06:01:55.367 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 10 librados: watch_flush exit 2026-02-17T06:01:55.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 10 client.5049.objecter close_session for osd.0 2026-02-17T06:01:55.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] conn(0x7fe6580816d0 msgr2=0x7fe658083af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:6826/821011033,v1:10.20.193.16:6827/821011033] conn(0x7fe6580816d0 0x7fe658083af0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 20 client.5049.objecter put_session s=0x7fe658081000 osd=0 1 2026-02-17T06:01:55.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 20 client.5049.objecter shutdown clearing up homeless session... 2026-02-17T06:01:55.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 10 client.5049.objecter successfully canceled tick 2026-02-17T06:01:55.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe658079190 msgr2=0x7fe65807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe658079190 0x7fe65807b650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe674039000 tx=0x7fe674033000 comp rx=0 tx=0).stop 2026-02-17T06:01:55.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 10 monclient: shutdown 2026-02-17T06:01:55.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:55.369 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe688155ce0 msgr2=0x7fe6881e9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 1 --2- 10.20.193.16:0/2322591858 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe688155ce0 0x7fe6881e9680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe67800ec10 tx=0x7fe67800c520 comp rx=0 tx=0).stop 2026-02-17T06:01:55.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe687fff640 1 -- 10.20.193.16:0/2322591858 reap_dead start 2026-02-17T06:01:55.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 shutdown_connections 2026-02-17T06:01:55.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.361+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 >> 10.20.193.16:0/2322591858 conn(0x7fe6880cf600 msgr2=0x7fe68806f260 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:55.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.362+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 shutdown_connections 2026-02-17T06:01:55.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.362+0000 7fe68e13e640 1 -- 10.20.193.16:0/2322591858 wait complete. 2026-02-17T06:01:55.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.362+0000 7fe68e13e640 1 librados: shutdown 2026-02-17T06:01:55.373 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-17T06:01:55.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.442+0000 7f3e8a553640 10 monclient: get_monmap_and_config 2026-02-17T06:01:55.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.442+0000 7f3e8a553640 10 monclient: build_initial_monmap 2026-02-17T06:01:55.442 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.442+0000 7f3e8a553640 10 monclient: monmap: 2026-02-17T06:01:55.442 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:55.442 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.442 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:55.443449+0000 2026-02-17T06:01:55.443 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:55.443449+0000 2026-02-17T06:01:55.443 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:55.443 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.443 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:55.443 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:55.443 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:55.443 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:55.443 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:55.443 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:55.443 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 1 Processor -- start 2026-02-17T06:01:55.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 1 -- start start 2026-02-17T06:01:55.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 10 monclient: init 2026-02-17T06:01:55.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:55.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 10 monclient: _add_conns ranks=[2,3,0,4,5,1] 2026-02-17T06:01:55.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3e84155e00 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:55.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f3e841561e0 0x7f3e840ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 10 monclient(hunting): picked mon.noname-a con 0x7f3e841561e0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:55.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3e84074dc0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:55.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e840c96d0 con 0x7f3e84074dc0 2026-02-17T06:01:55.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e84074860 con 0x7f3e84155e00 2026-02-17T06:01:55.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3e840c9940 con 0x7f3e841561e0 2026-02-17T06:01:55.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:55.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.443+0000 7f3e8a553640 10 monclient(hunting): authenticate will time out at 1368.012692s 2026-02-17T06:01:55.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.444+0000 7f3e83fff640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f3e84155e00 0x7f3e84074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:51284/0 (socket says 10.20.193.16:51284) 2026-02-17T06:01:55.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.444+0000 7f3e83fff640 1 -- 10.20.193.16:0/3641583207 learned_addr learned my addr 10.20.193.16:0/3641583207 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:55.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.446+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2060464787 0 0) 0x7f3e70002920 con 0x7f3e84155e00 2026-02-17T06:01:55.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.446+0000 7f3e82ffd640 10 monclient(hunting): _negotiate 2026-02-17T06:01:55.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.446+0000 7f3e82ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.446+0000 7f3e82ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.446+0000 7f3e82ffd640 10 monclient(hunting): my global_id is 5192 2026-02-17T06:01:55.447 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.446+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e68003620 con 0x7f3e84155e00 2026-02-17T06:01:55.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 <== mon.2 v1:10.20.193.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1251316810 0 0) 0x7f3e68003620 con 0x7f3e84155e00 2026-02-17T06:01:55.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 >> v2:10.20.193.16:3300/0 conn(0x7f3e841561e0 msgr2=0x7f3e840ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:55.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 --2- 10.20.193.16:0/3641583207 >> v2:10.20.193.16:3300/0 conn(0x7f3e841561e0 0x7f3e840ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 >> v1:10.20.193.16:6789/0 conn(0x7f3e84074dc0 legacy=0x7f3e8415f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 monclient: found mon.noname-c-legacy 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.148:6789/0 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 --> v1:10.20.193.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e840c9c20 con 0x7f3e84155e00 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 <== mon.2 v1:10.20.193.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 146676893 0 0) 0x7f3e700030d0 con 0x7f3e84155e00 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient: dump: 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] but i'm connected to v1:10.20.193.148:6789/0, reconnecting 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 >> v1:10.20.193.148:6789/0 conn(0x7f3e84155e00 legacy=0x7f3e84074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 --2- 10.20.193.16:0/3641583207 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3e68003bf0 0x7f3e68004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient(hunting): picked mon.c con 0x7f3e68003bf0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 --2- 10.20.193.16:0/3641583207 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3e68004900 0x7f3e68004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient(hunting): picked mon.a con 0x7f3e68004900 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 --2- 10.20.193.16:0/3641583207 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e68009ab0 0x7f3e6800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient(hunting): picked mon.b con 0x7f3e68009ab0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e840c9940 con 0x7f3e68004900 2026-02-17T06:01:55.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6800c4c0 con 0x7f3e68009ab0 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6800c6c0 con 0x7f3e68003bf0 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e82ffd640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e88ac9640 1 --2- 10.20.193.16:0/3641583207 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e68009ab0 0x7f3e6800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e88ac9640 10 monclient(hunting): get_auth_request con 0x7f3e68009ab0 auth_method 0 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e88ac9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e88ac9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.448+0000 7f3e88ac9640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e83fff640 1 --2- 10.20.193.16:0/3641583207 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3e68003bf0 0x7f3e68004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e83fff640 10 monclient(hunting): get_auth_request con 0x7f3e68003bf0 auth_method 0 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e83fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e83fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e83fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 10 monclient(hunting): handle_auth_done global_id 5192 payload 293 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 1 -- 10.20.193.16:0/3641583207 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3e68003bf0 msgr2=0x7f3e68004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 1 --2- 10.20.193.16:0/3641583207 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3e68003bf0 0x7f3e68004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 1 -- 10.20.193.16:0/3641583207 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3e68004900 msgr2=0x7f3e68004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 1 --2- 10.20.193.16:0/3641583207 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3e68004900 0x7f3e68004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 10 monclient: _finish_hunting 0 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 1 monclient: found mon.b 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 1 -- 10.20.193.16:0/3641583207 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e68003620 con 0x7f3e68009ab0 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 10 monclient: _finish_auth 0 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 10 monclient: _check_auth_tickets 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e88ac9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e8a553640 5 monclient: authenticate success, global_id 5192 2026-02-17T06:01:55.457 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.449+0000 7f3e8a553640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:55.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.450+0000 7f3e88ac9640 1 -- 10.20.193.16:0/3641583207 reap_dead start 2026-02-17T06:01:55.458 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.450+0000 7f3e88ac9640 1 --2- 10.20.193.16:0/3641583207 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e68009ab0 0x7f3e6800bea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c240 tx=0x7f3e7802f540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8abb98c199678648 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78003760 con 0x7f3e68009ab0 2026-02-17T06:01:55.463 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e82ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e82ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e82ffd640 10 monclient: dump: 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.464 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e78003c70 con 0x7f3e68009ab0 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e82ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e82ffd640 1 -- 10.20.193.16:0/3641583207 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78005310 con 0x7f3e68009ab0 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 4 set_mon_vals no callback set 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e82ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e82ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e82ffd640 10 monclient: dump: 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.465 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 10 monclient: shutdown 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 1 -- 10.20.193.16:0/3641583207 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e68009ab0 msgr2=0x7f3e6800bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.466 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 1 --2- 10.20.193.16:0/3641583207 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e68009ab0 0x7f3e6800bea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c240 tx=0x7f3e7802f540 comp rx=0 tx=0).stop 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 1 -- 10.20.193.16:0/3641583207 shutdown_connections 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 1 --2- 10.20.193.16:0/3641583207 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e68009ab0 0x7f3e6800bea0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 1 -- 10.20.193.16:0/3641583207 >> 10.20.193.16:0/3641583207 conn(0x7f3e840ce6c0 msgr2=0x7f3e840cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 1 -- 10.20.193.16:0/3641583207 shutdown_connections 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.452+0000 7f3e8a553640 1 -- 10.20.193.16:0/3641583207 wait complete. 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.453+0000 7f3e8a553640 10 monclient: build_initial_monmap 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.453+0000 7f3e8a553640 10 monclient: monmap: 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:55.467 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.453+0000 7f3e8a553640 1 librados: starting msgr at 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.453+0000 7f3e8a553640 1 librados: starting objecter 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.453+0000 7f3e8a553640 1 Processor -- start 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.453+0000 7f3e8a553640 1 -- start start 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.453+0000 7f3e8a553640 1 librados: setting wanted keys 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.453+0000 7f3e8a553640 1 librados: calling monclient init 2026-02-17T06:01:55.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.453+0000 7f3e8a553640 10 monclient: init 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3e68009ab0 0x7f3e840d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 10 monclient(hunting): picked mon.noname-c con 0x7f3e68009ab0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3e840d12e0 0x7f3e841f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 10 monclient(hunting): picked mon.noname-a con 0x7f3e840d12e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e841f69c0 0x7f3e841f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 10 monclient(hunting): picked mon.noname-b con 0x7f3e841f69c0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e840c9360 con 0x7f3e840d12e0 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f3e840ca1e0 con 0x7f3e841f69c0 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3e840ca700 con 0x7f3e68009ab0 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e8a553640 10 monclient(hunting): authenticate will time out at 1368.023179s 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e88ac9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e841f69c0 0x7f3e841f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e83fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3e68009ab0 0x7f3e840d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e88ac9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e841f69c0 0x7f3e841f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43288/0 (socket says 10.20.193.16:43288) 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e88ac9640 1 -- 10.20.193.16:0/2391435294 learned_addr learned my addr 10.20.193.16:0/2391435294 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e88ac9640 10 monclient(hunting): get_auth_request con 0x7f3e841f69c0 auth_method 0 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e88ac9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e88ac9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e88ac9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e83fff640 10 monclient(hunting): get_auth_request con 0x7f3e68009ab0 auth_method 0 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e83fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e83fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e88ac9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e88ac9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.454+0000 7f3e88ac9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.455+0000 7f3e83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.455+0000 7f3e83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.455+0000 7f3e83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.456+0000 7f3e88ac9640 10 monclient(hunting): handle_auth_done global_id 5131 payload 824 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.456+0000 7f3e88ac9640 1 -- 10.20.193.16:0/2391435294 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3e68009ab0 msgr2=0x7f3e840d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.456+0000 7f3e88ac9640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3e68009ab0 0x7f3e840d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.456+0000 7f3e88ac9640 1 -- 10.20.193.16:0/2391435294 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3e840d12e0 msgr2=0x7f3e841f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.456+0000 7f3e88ac9640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3e840d12e0 0x7f3e841f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.456+0000 7f3e88ac9640 10 monclient: _finish_hunting 0 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.456+0000 7f3e88ac9640 1 monclient: found mon.noname-b 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.456+0000 7f3e88ac9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.456+0000 7f3e88ac9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.456+0000 7f3e88ac9640 1 -- 10.20.193.16:0/2391435294 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e841f6e40 con 0x7f3e841f69c0 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.457+0000 7f3e88ac9640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e841f69c0 0x7f3e841f7a20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c370 tx=0x7f3e78002cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.457+0000 7f3e80ff9640 10 client.?.objecter ms_handle_connect 0x7f3e841f69c0 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.457+0000 7f3e80ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 1 -- 10.20.193.16:0/2391435294 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e780030b0 con 0x7f3e841f69c0 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 10 monclient: dump: 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 10 monclient: _finish_auth 0 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:01:55.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e8a553640 5 monclient: authenticate success, global_id 5131 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e8a553640 10 monclient: _renew_subs 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e8a553640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e8a553640 1 -- 10.20.193.16:0/2391435294 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e841f59c0 con 0x7f3e841f69c0 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e8a553640 10 client.5131.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e8a553640 10 monclient: _renew_subs 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e8a553640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e8a553640 1 -- 10.20.193.16:0/2391435294 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e841e9ba0 con 0x7f3e841f69c0 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e8a553640 1 librados: init done 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 1 -- 10.20.193.16:0/2391435294 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e78003a70 con 0x7f3e841f69c0 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 1 -- 10.20.193.16:0/2391435294 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78036610 con 0x7f3e841f69c0 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e827fc640 4 set_mon_vals no callback set 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e80ff9640 10 monclient: dump: 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.458+0000 7f3e8a553640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:55.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.459+0000 7f3e8a553640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:55.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.459+0000 7f3e8a553640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:55.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.459+0000 7f3e8a553640 1 -- 10.20.193.16:0/2391435294 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e84159730 con 0x7f3e841f69c0 2026-02-17T06:01:55.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.469+0000 7f3e80ff9640 1 -- 10.20.193.16:0/2391435294 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f3e780367b0 con 0x7f3e841f69c0 2026-02-17T06:01:55.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.470+0000 7f3e80ff9640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3e54079170 0x7f3e5407b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.470+0000 7f3e83fff640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3e54079170 0x7f3e5407b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.470+0000 7f3e83fff640 10 monclient: get_auth_request con 0x7f3e54079170 auth_method 0 2026-02-17T06:01:55.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.470+0000 7f3e80ff9640 10 client.5131.objecter ms_handle_connect 0x7f3e54079170 2026-02-17T06:01:55.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.470+0000 7f3e83fff640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3e54079170 0x7f3e5407b630 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e70002410 tx=0x7f3e70007af0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.472+0000 7f3e80ff9640 1 -- 10.20.193.16:0/2391435294 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f3e78082930 con 0x7f3e841f69c0 2026-02-17T06:01:55.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.472+0000 7f3e80ff9640 10 client.5131.objecter ms_dispatch2 0x7f3e84000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:55.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.472+0000 7f3e80ff9640 3 client.5131.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:55.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.472+0000 7f3e80ff9640 3 client.5131.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:55.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.472+0000 7f3e80ff9640 20 client.5131.objecter dump_active .. 0 homeless 2026-02-17T06:01:55.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.572+0000 7f3e80ff9640 1 -- 10.20.193.16:0/2391435294 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3e841f6e40 con 0x7f3e841f69c0 2026-02-17T06:01:55.571 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.572+0000 7f3e80ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:55.572 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.572+0000 7f3e80ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:55.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.653+0000 7f3e8a553640 1 -- 10.20.193.16:0/2391435294 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f3e840caae0 con 0x7f3e54079170 2026-02-17T06:01:55.654 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.654+0000 7f3e80ff9640 1 -- 10.20.193.16:0/2391435294 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3e840caae0 con 0x7f3e54079170 2026-02-17T06:01:55.654 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 0 to deep-scrub 2026-02-17T06:01:55.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 10 librados: watch_flush enter 2026-02-17T06:01:55.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 10 librados: watch_flush exit 2026-02-17T06:01:55.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 20 client.5131.objecter shutdown clearing up homeless session... 2026-02-17T06:01:55.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 10 client.5131.objecter successfully canceled tick 2026-02-17T06:01:55.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 -- 10.20.193.16:0/2391435294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3e54079170 msgr2=0x7f3e5407b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.657 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3e54079170 0x7f3e5407b630 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e70002410 tx=0x7f3e70007af0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 10 monclient: shutdown 2026-02-17T06:01:55.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:55.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 -- 10.20.193.16:0/2391435294 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e841f69c0 msgr2=0x7f3e841f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e841f69c0 0x7f3e841f7a20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c370 tx=0x7f3e78002cb0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 -- 10.20.193.16:0/2391435294 shutdown_connections 2026-02-17T06:01:55.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f3e54079170 0x7f3e5407b630 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f3e841f69c0 0x7f3e841f7a20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f3e840d12e0 0x7f3e841f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 --2- 10.20.193.16:0/2391435294 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f3e68009ab0 0x7f3e840d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 -- 10.20.193.16:0/2391435294 >> 10.20.193.16:0/2391435294 conn(0x7f3e840ce6c0 msgr2=0x7f3e841519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:55.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 -- 10.20.193.16:0/2391435294 shutdown_connections 2026-02-17T06:01:55.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 -- 10.20.193.16:0/2391435294 wait complete. 2026-02-17T06:01:55.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.655+0000 7f3e8a553640 1 librados: shutdown 2026-02-17T06:01:55.664 INFO:tasks.ceph:Scrubbing osd.4 2026-02-17T06:01:55.664 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.732+0000 7f5aa82bf640 10 monclient: get_monmap_and_config 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.732+0000 7f5aa82bf640 10 monclient: build_initial_monmap 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.733+0000 7f5aa82bf640 10 monclient: monmap: 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:55.734002+0000 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:55.734002+0000 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:55.733 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.733+0000 7f5aa82bf640 1 Processor -- start 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.733+0000 7f5aa82bf640 1 -- start start 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.733+0000 7f5aa82bf640 10 monclient: init 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.733+0000 7f5aa82bf640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.733+0000 7f5aa82bf640 10 monclient: _add_conns ranks=[1,5,0,3,4,2] 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.733+0000 7f5aa82bf640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5aa00700f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.733+0000 7f5aa82bf640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f5aa00c86f0 0x7f5aa00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.733+0000 7f5aa82bf640 10 monclient(hunting): picked mon.noname-c con 0x7f5aa00c86f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa82bf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5aa01536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa82bf640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aa00c4820 con 0x7f5aa01536f0 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa82bf640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aa00c4b00 con 0x7f5aa00700f0 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa82bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa82bf640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f5aa00c4de0 con 0x7f5aa00c86f0 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa82bf640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa82bf640 10 monclient(hunting): authenticate will time out at 1368.303027s 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa6034640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f5aa00700f0 0x7f5aa00c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:51266/0 (socket says 10.20.193.16:51266) 2026-02-17T06:01:55.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa6034640 1 -- 10.20.193.16:0/2074987112 learned_addr learned my addr 10.20.193.16:0/2074987112 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:55.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa5833640 1 --2- 10.20.193.16:0/2074987112 >> v2:10.20.193.148:3300/0 conn(0x7f5aa00c86f0 0x7f5aa00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa5833640 10 monclient(hunting): get_auth_request con 0x7f5aa00c86f0 auth_method 0 2026-02-17T06:01:55.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa5833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa5833640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.734+0000 7f5aa5833640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.735 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 10 monclient(hunting): handle_auth_done global_id 5198 payload 274 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 1 -- 10.20.193.16:0/2074987112 >> v1:10.20.193.128:6789/0 conn(0x7f5aa00700f0 legacy=0x7f5aa00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 1 -- 10.20.193.16:0/2074987112 >> v1:10.20.193.16:6789/0 conn(0x7f5aa01536f0 legacy=0x7f5aa0153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5032640 1 -- 10.20.193.16:0/2074987112 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2447846764 0 0) 0x7f5aa00c4820 con 0x7f5aa01536f0 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 10 monclient: _finish_hunting 0 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 1 monclient: found mon.noname-c 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5833640 1 -- 10.20.193.16:0/2074987112 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa00c50c0 con 0x7f5aa00c86f0 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.735+0000 7f5aa5032640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T06:01:55.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.736+0000 7f5aa5833640 1 --2- 10.20.193.16:0/2074987112 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5aa00c86f0 0x7f5aa00704d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a940098a0 tx=0x7f5a9402edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7facbc464c83464 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 1 -- 10.20.193.16:0/2074987112 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9403c070 con 0x7f5aa00c86f0 2026-02-17T06:01:55.738 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 10 monclient: dump: 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 10 monclient: _finish_auth 0 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 10 monclient: _check_auth_tickets 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 1 -- 10.20.193.16:0/2074987112 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9402fae0 con 0x7f5aa00c86f0 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa82bf640 5 monclient: authenticate success, global_id 5198 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 1 -- 10.20.193.16:0/2074987112 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94037440 con 0x7f5aa00c86f0 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa82bf640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa82bf640 4 set_mon_vals no callback set 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:55.739 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.738+0000 7f5aa5032640 10 monclient: dump: 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 10 monclient: shutdown 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 1 -- 10.20.193.16:0/2074987112 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5aa00c86f0 msgr2=0x7f5aa00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 1 --2- 10.20.193.16:0/2074987112 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5aa00c86f0 0x7f5aa00704d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a940098a0 tx=0x7f5a9402edd0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 1 -- 10.20.193.16:0/2074987112 shutdown_connections 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 1 --2- 10.20.193.16:0/2074987112 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5aa00c86f0 0x7f5aa00704d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 1 -- 10.20.193.16:0/2074987112 >> 10.20.193.16:0/2074987112 conn(0x7f5aa00cd770 msgr2=0x7f5aa00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 1 -- 10.20.193.16:0/2074987112 shutdown_connections 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 1 -- 10.20.193.16:0/2074987112 wait complete. 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 10 monclient: build_initial_monmap 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 10 monclient: monmap: 2026-02-17T06:01:55.740 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 1 librados: starting msgr at 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.739+0000 7f5aa82bf640 1 librados: starting objecter 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 1 Processor -- start 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 1 -- start start 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 1 librados: setting wanted keys 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 1 librados: calling monclient init 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient: init 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5aa00700f0 0x7f5aa01e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient(hunting): picked mon.noname-b con 0x7f5aa00700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5aa00c86f0 0x7f5aa013b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient(hunting): picked mon.noname-a con 0x7f5aa00c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5aa01536f0 0x7f5aa01e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient(hunting): picked mon.noname-c con 0x7f5aa01536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0074c00 con 0x7f5aa00c86f0 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa00c5680 con 0x7f5aa00700f0 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0074d80 con 0x7f5aa01536f0 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa82bf640 10 monclient(hunting): authenticate will time out at 1368.309461s 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa5833640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5aa00c86f0 0x7f5aa013b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa6034640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5aa00700f0 0x7f5aa01e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.741 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa6835640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5aa01536f0 0x7f5aa01e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa5833640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5aa00c86f0 0x7f5aa013b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33684/0 (socket says 10.20.193.16:33684) 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa5833640 1 -- 10.20.193.16:0/3528215 learned_addr learned my addr 10.20.193.16:0/3528215 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa5833640 10 monclient(hunting): get_auth_request con 0x7f5aa00c86f0 auth_method 0 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa5833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa5833640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa5833640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa6835640 10 monclient(hunting): get_auth_request con 0x7f5aa01536f0 auth_method 0 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa6835640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa6835640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa6835640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa6034640 10 monclient(hunting): get_auth_request con 0x7f5aa00700f0 auth_method 0 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa6034640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa6034640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.740+0000 7f5aa6034640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.741+0000 7f5aa5833640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.741+0000 7f5aa5833640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.741+0000 7f5aa5833640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.741+0000 7f5aa6835640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.741+0000 7f5aa6835640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.741+0000 7f5aa6835640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.741+0000 7f5aa6034640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.741+0000 7f5aa6034640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.741+0000 7f5aa6034640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.742 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.742+0000 7f5aa6835640 10 monclient(hunting): handle_auth_done global_id 5201 payload 824 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.742+0000 7f5aa6835640 1 -- 10.20.193.16:0/3528215 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5aa00700f0 msgr2=0x7f5aa01e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.742+0000 7f5aa6835640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f5aa00700f0 0x7f5aa01e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.742+0000 7f5aa6835640 1 -- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5aa00c86f0 msgr2=0x7f5aa013b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.742+0000 7f5aa6835640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f5aa00c86f0 0x7f5aa013b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.742+0000 7f5aa6835640 10 monclient: _finish_hunting 0 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.742+0000 7f5aa6835640 1 monclient: found mon.noname-c 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.742+0000 7f5aa6835640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.742+0000 7f5aa6835640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.742+0000 7f5aa6835640 1 -- 10.20.193.16:0/3528215 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa01f5cb0 con 0x7f5aa01536f0 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.743+0000 7f5aa6835640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5aa01536f0 0x7f5aa01e73b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00ca70 tx=0x7f5a9c00cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.743+0000 7f5a86ffd640 10 client.?.objecter ms_handle_connect 0x7f5aa01536f0 2026-02-17T06:01:55.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.743+0000 7f5a86ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 1 -- 10.20.193.16:0/3528215 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c017070 con 0x7f5aa01536f0 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 10 monclient: dump: 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 1 -- 10.20.193.16:0/3528215 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c00a7d0 con 0x7f5aa01536f0 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 1 -- 10.20.193.16:0/3528215 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00abc0 con 0x7f5aa01536f0 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5a86ffd640 10 monclient: dump: 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa4831640 4 set_mon_vals no callback set 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 5 monclient: authenticate success, global_id 5201 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 10 monclient: _renew_subs 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa0151de0 con 0x7f5aa01536f0 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 10 client.5201.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 10 monclient: _renew_subs 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa01523a0 con 0x7f5aa01536f0 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 1 librados: init done 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 10 client.5201.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 15 client.5201.objecter _session_command_op_assign -1 1 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 20 client.5201.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 10 client.5201.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:55.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.745+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5aa00704d0 con 0x7f5aa01536f0 2026-02-17T06:01:55.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.757+0000 7f5a86ffd640 1 -- 10.20.193.16:0/3528215 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f5a9c005070 con 0x7f5aa01536f0 2026-02-17T06:01:55.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.757+0000 7f5a86ffd640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5a70079170 0x7f5a7007b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.757+0000 7f5aa6034640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5a70079170 0x7f5a7007b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.757 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.757+0000 7f5aa6034640 10 monclient: get_auth_request con 0x7f5a70079170 auth_method 0 2026-02-17T06:01:55.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.757+0000 7f5aa6034640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5a70079170 0x7f5a7007b630 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5a90002770 tx=0x7f5a90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.758 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.757+0000 7f5a86ffd640 10 client.5201.objecter ms_handle_connect 0x7f5a70079170 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 1 -- 10.20.193.16:0/3528215 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f5a9c097920 con 0x7f5aa01536f0 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 10 client.5201.objecter ms_dispatch2 0x7f5aa0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 3 client.5201.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 3 client.5201.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 10 client.5201.objecter checking command 1 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x7f5a70081650 0x7f5a70083a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 20 client.5201.objecter _get_session s=0x7f5a70080f80 osd=4 3 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 20 client.5201.objecter put_session s=0x7f5a70080f80 osd=4 3 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 15 client.5201.objecter _session_command_op_remove -1 1 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 20 client.5201.objecter _get_session s=0x7f5a70080f80 osd=4 3 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 20 client.5201.objecter get_session s=0x7f5a70080f80 osd=4 3 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 15 client.5201.objecter _session_command_op_assign 4 1 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 20 client.5201.objecter put_session s=0x7f5a70080f80 osd=4 4 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 10 client.5201.objecter _send_command 1 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 1 -- 10.20.193.16:0/3528215 --> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a70084140 con 0x7f5a70081650 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 20 client.5201.objecter dump_active .. 0 homeless 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 1 -- 10.20.193.16:0/3528215 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5a9c05e240 con 0x7f5aa01536f0 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.758+0000 7f5a86ffd640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.759+0000 7f5aa5833640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x7f5a70081650 0x7f5a70083a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.759 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.759+0000 7f5aa5833640 10 monclient: get_auth_request con 0x7f5a70081650 auth_method 0 2026-02-17T06:01:55.760 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.760+0000 7f5a86ffd640 10 client.5201.objecter ms_handle_connect 0x7f5a70081650 2026-02-17T06:01:55.761 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.760+0000 7f5aa5833640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x7f5a70081650 0x7f5a70083a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.770+0000 7f5a86ffd640 1 -- 10.20.193.16:0/3528215 <== osd.4 v2:10.20.193.16:6818/1157611076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f5a70084140 con 0x7f5a70081650 2026-02-17T06:01:55.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.770+0000 7f5a86ffd640 10 client.5201.objecter ms_dispatch2 0x7f5aa0000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:55.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.770+0000 7f5a86ffd640 10 client.5201.objecter _finish_command 1 = system:0 2026-02-17T06:01:55.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.770+0000 7f5a86ffd640 20 client.5201.objecter put_session s=0x7f5a70080f80 osd=4 4 2026-02-17T06:01:55.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.770+0000 7f5a86ffd640 15 client.5201.objecter _session_command_op_remove 4 1 2026-02-17T06:01:55.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 10 client.5201.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:55.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 15 client.5201.objecter _session_command_op_assign -1 2 2026-02-17T06:01:55.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 20 client.5201.objecter _get_session s=0x7f5a70080f80 osd=4 3 2026-02-17T06:01:55.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 20 client.5201.objecter put_session s=0x7f5a70080f80 osd=4 3 2026-02-17T06:01:55.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 20 client.5201.objecter _get_session s=0x7f5a70080f80 osd=4 3 2026-02-17T06:01:55.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 15 client.5201.objecter _session_command_op_remove -1 2 2026-02-17T06:01:55.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 20 client.5201.objecter get_session s=0x7f5a70080f80 osd=4 3 2026-02-17T06:01:55.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 15 client.5201.objecter _session_command_op_assign 4 2 2026-02-17T06:01:55.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 20 client.5201.objecter put_session s=0x7f5a70080f80 osd=4 4 2026-02-17T06:01:55.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 10 client.5201.objecter _send_command 2 2026-02-17T06:01:55.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.776+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 --> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5aa006be30 con 0x7f5a70081650 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.783+0000 7f5a86ffd640 1 -- 10.20.193.16:0/3528215 <== osd.4 v2:10.20.193.16:6818/1157611076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5aa006be30 con 0x7f5a70081650 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.783+0000 7f5a86ffd640 10 client.5201.objecter ms_dispatch2 0x7f5aa0000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.783+0000 7f5a86ffd640 10 client.5201.objecter _finish_command 2 = system:0 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.783+0000 7f5a86ffd640 20 client.5201.objecter put_session s=0x7f5a70080f80 osd=4 4 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.783+0000 7f5a86ffd640 15 client.5201.objecter _session_command_op_remove 4 2 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 10 librados: watch_flush enter 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 10 librados: watch_flush exit 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 10 client.5201.objecter close_session for osd.4 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x7f5a70081650 msgr2=0x7f5a70083a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:6818/1157611076,v1:10.20.193.16:6819/1157611076] conn(0x7f5a70081650 0x7f5a70083a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 20 client.5201.objecter put_session s=0x7f5a70080f80 osd=4 1 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 20 client.5201.objecter shutdown clearing up homeless session... 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 10 client.5201.objecter successfully canceled tick 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5a70079170 msgr2=0x7f5a7007b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5a70079170 0x7f5a7007b630 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5a90002770 tx=0x7f5a90009040 comp rx=0 tx=0).stop 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 10 monclient: shutdown 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5aa01536f0 msgr2=0x7f5aa01e73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 --2- 10.20.193.16:0/3528215 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f5aa01536f0 0x7f5aa01e73b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00ca70 tx=0x7f5a9c00cf40 comp rx=0 tx=0).stop 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa6835640 1 -- 10.20.193.16:0/3528215 reap_dead start 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 shutdown_connections 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 >> 10.20.193.16:0/3528215 conn(0x7f5aa00cd770 msgr2=0x7f5aa0153b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 shutdown_connections 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 -- 10.20.193.16:0/3528215 wait complete. 2026-02-17T06:01:55.785 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.784+0000 7f5aa82bf640 1 librados: shutdown 2026-02-17T06:01:55.791 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.858+0000 7f677dd9e640 10 monclient: get_monmap_and_config 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.858+0000 7f677dd9e640 10 monclient: build_initial_monmap 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.858+0000 7f677dd9e640 10 monclient: monmap: 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:55.859882+0000 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:55.859882+0000 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 1 Processor -- start 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 1 -- start start 2026-02-17T06:01:55.859 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 10 monclient: init 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 10 monclient: _add_conns ranks=[3,2,4,0,1,5] 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f67780700f0 0x7f67780704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 10 monclient(hunting): picked mon.noname-a con 0x7f67780700f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f67780c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f67781536f0 0x7f6778153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 10 monclient(hunting): picked mon.noname-b con 0x7f67781536f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67780c4820 con 0x7f67780c7fe0 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f67780c4b00 con 0x7f67780700f0 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f67780c4de0 con 0x7f67781536f0 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.859+0000 7f677dd9e640 10 monclient(hunting): authenticate will time out at 1368.428818s 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.860+0000 7f6777fff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f67781536f0 0x7f6778153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.860+0000 7f6777fff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f67781536f0 0x7f6778153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43314/0 (socket says 10.20.193.16:43314) 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.860+0000 7f6777fff640 1 -- 10.20.193.16:0/2238287571 learned_addr learned my addr 10.20.193.16:0/2238287571 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.860+0000 7f6777fff640 10 monclient(hunting): get_auth_request con 0x7f67781536f0 auth_method 0 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.860+0000 7f6777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.860+0000 7f6777fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.860+0000 7f6777fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.860 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.860+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.860+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.860+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.861+0000 7f6777fff640 10 monclient(hunting): handle_auth_done global_id 5140 payload 274 2026-02-17T06:01:55.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.861+0000 7f6777fff640 1 -- 10.20.193.16:0/2238287571 >> v2:10.20.193.16:3300/0 conn(0x7f67780700f0 msgr2=0x7f67780704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:55.861 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.861+0000 7f6777fff640 1 --2- 10.20.193.16:0/2238287571 >> v2:10.20.193.16:3300/0 conn(0x7f67780700f0 0x7f67780704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.861+0000 7f6777fff640 1 -- 10.20.193.16:0/2238287571 >> v1:10.20.193.148:6789/0 conn(0x7f67780c7fe0 legacy=0x7f67780c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:55.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.861+0000 7f6777fff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:55.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.861+0000 7f6777fff640 1 monclient: found mon.noname-b 2026-02-17T06:01:55.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.861+0000 7f6777fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:55.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.861+0000 7f6777fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:55.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.861+0000 7f6777fff640 1 -- 10.20.193.16:0/2238287571 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67780c50c0 con 0x7f67781536f0 2026-02-17T06:01:55.862 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.862+0000 7f6777fff640 1 --2- 10.20.193.16:0/2238287571 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 0x7f6778153b20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f67780c9540 tx=0x7f676c02f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dec28711ef73ca9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 1 -- 10.20.193.16:0/2238287571 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c02fc50 con 0x7f67781536f0 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 10 monclient: dump: 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 5 monclient: authenticate success, global_id 5140 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 1 -- 10.20.193.16:0/2238287571 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676c02fdf0 con 0x7f67781536f0 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 1 -- 10.20.193.16:0/2238287571 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c037610 con 0x7f67781536f0 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f67767fc640 10 monclient: dump: 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 4 set_mon_vals no callback set 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 10 monclient: shutdown 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 1 -- 10.20.193.16:0/2238287571 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 msgr2=0x7f6778153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 1 --2- 10.20.193.16:0/2238287571 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 0x7f6778153b20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f67780c9540 tx=0x7f676c02f230 comp rx=0 tx=0).stop 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 1 -- 10.20.193.16:0/2238287571 shutdown_connections 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 1 --2- 10.20.193.16:0/2238287571 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 0x7f6778153b20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 1 --2- 10.20.193.16:0/2238287571 >> v2:10.20.193.16:3300/0 conn(0x7f67780700f0 0x7f67780704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 1 -- 10.20.193.16:0/2238287571 >> 10.20.193.16:0/2238287571 conn(0x7f67780cd770 msgr2=0x7f67780cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 1 -- 10.20.193.16:0/2238287571 shutdown_connections 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.864+0000 7f677dd9e640 1 -- 10.20.193.16:0/2238287571 wait complete. 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 10 monclient: build_initial_monmap 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 10 monclient: monmap: 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 1 librados: starting msgr at 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 1 librados: starting objecter 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 1 Processor -- start 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 1 -- start start 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 1 librados: setting wanted keys 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 1 librados: calling monclient init 2026-02-17T06:01:55.866 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 10 monclient: init 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.865+0000 7f677dd9e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67780700f0 0x7f67781f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 10 monclient(hunting): picked mon.noname-a con 0x7f67780700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f67780c7fe0 0x7f67781f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 10 monclient(hunting): picked mon.noname-c con 0x7f67780c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 0x7f677814ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 10 monclient(hunting): picked mon.noname-b con 0x7f67781536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6778154060 con 0x7f67780700f0 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f67780c5680 con 0x7f67781536f0 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6778074c00 con 0x7f67780c7fe0 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f677dd9e640 10 monclient(hunting): authenticate will time out at 1368.435033s 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6777fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 0x7f677814ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6776ffd640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f67780c7fe0 0x7f67781f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6777fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 0x7f677814ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43322/0 (socket says 10.20.193.16:43322) 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6777fff640 1 -- 10.20.193.16:0/4115681359 learned_addr learned my addr 10.20.193.16:0/4115681359 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6777fff640 10 monclient(hunting): get_auth_request con 0x7f67781536f0 auth_method 0 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6777fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6777fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6776ffd640 10 monclient(hunting): get_auth_request con 0x7f67780c7fe0 auth_method 0 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6776ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6776ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6776ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.867 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6776ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6776ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.866+0000 7f6776ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.867+0000 7f67777fe640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67780700f0 0x7f67781f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.867+0000 7f67777fe640 10 monclient(hunting): get_auth_request con 0x7f67780700f0 auth_method 0 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.867+0000 7f67777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.867+0000 7f67777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.867+0000 7f67777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.867+0000 7f67777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.867+0000 7f67777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f67777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f6777fff640 10 monclient(hunting): handle_auth_done global_id 5143 payload 824 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f6777fff640 1 -- 10.20.193.16:0/4115681359 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f67780c7fe0 msgr2=0x7f67781f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f6777fff640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f67780c7fe0 0x7f67781f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f6777fff640 1 -- 10.20.193.16:0/4115681359 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67780700f0 msgr2=0x7f67781f40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f6777fff640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67780700f0 0x7f67781f40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f6777fff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f6777fff640 1 monclient: found mon.noname-b 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f6777fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f6777fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:55.868 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.868+0000 7f6777fff640 1 -- 10.20.193.16:0/4115681359 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6778144d70 con 0x7f67781536f0 2026-02-17T06:01:55.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.869+0000 7f6777fff640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 0x7f677814ffa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f676c02f270 tx=0x7f676c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.869+0000 7f6753fff640 10 client.?.objecter ms_handle_connect 0x7f67781536f0 2026-02-17T06:01:55.869 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.869+0000 7f6753fff640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 1 -- 10.20.193.16:0/4115681359 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c046070 con 0x7f67781536f0 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 10 monclient: dump: 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.870 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 10 monclient: _finish_auth 0 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f677dd9e640 5 monclient: authenticate success, global_id 5143 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f677dd9e640 10 monclient: _renew_subs 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f677dd9e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f677dd9e640 1 -- 10.20.193.16:0/4115681359 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67781f3910 con 0x7f67781536f0 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f677dd9e640 10 client.5143.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f677dd9e640 10 monclient: _renew_subs 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f677dd9e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f677dd9e640 1 -- 10.20.193.16:0/4115681359 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67781f7620 con 0x7f67781536f0 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f677dd9e640 1 librados: init done 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 1 -- 10.20.193.16:0/4115681359 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676c042640 con 0x7f67781536f0 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 1 -- 10.20.193.16:0/4115681359 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c050060 con 0x7f67781536f0 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6753fff640 10 monclient: dump: 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f6775ffb640 4 set_mon_vals no callback set 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.870+0000 7f677dd9e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:55.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.871+0000 7f677dd9e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:55.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.871+0000 7f677dd9e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:55.872 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.871+0000 7f677dd9e640 1 -- 10.20.193.16:0/4115681359 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67780704d0 con 0x7f67781536f0 2026-02-17T06:01:55.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.882+0000 7f6753fff640 1 -- 10.20.193.16:0/4115681359 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f676c0429f0 con 0x7f67781536f0 2026-02-17T06:01:55.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.882+0000 7f6753fff640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6748079140 0x7f674807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.882+0000 7f67777fe640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6748079140 0x7f674807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.882+0000 7f67777fe640 10 monclient: get_auth_request con 0x7f6748079140 auth_method 0 2026-02-17T06:01:55.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.883+0000 7f6753fff640 10 client.5143.objecter ms_handle_connect 0x7f6748079140 2026-02-17T06:01:55.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.883+0000 7f67777fe640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6748079140 0x7f674807b600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6764006fd0 tx=0x7f6764009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:55.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.883+0000 7f6753fff640 1 -- 10.20.193.16:0/4115681359 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f676c0841f0 con 0x7f67781536f0 2026-02-17T06:01:55.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.883+0000 7f6753fff640 10 client.5143.objecter ms_dispatch2 0x7f6778000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:55.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.883+0000 7f6753fff640 3 client.5143.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:55.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.883+0000 7f6753fff640 3 client.5143.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:55.883 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.883+0000 7f6753fff640 20 client.5143.objecter dump_active .. 0 homeless 2026-02-17T06:01:55.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.984+0000 7f6753fff640 1 -- 10.20.193.16:0/4115681359 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f676c0881b0 con 0x7f67781536f0 2026-02-17T06:01:55.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.984+0000 7f6753fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:55.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:55.984+0000 7f6753fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:56.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.065+0000 7f677dd9e640 1 -- 10.20.193.16:0/4115681359 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f67781573f0 con 0x7f6748079140 2026-02-17T06:01:56.066 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.066+0000 7f6753fff640 1 -- 10.20.193.16:0/4115681359 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f67781573f0 con 0x7f6748079140 2026-02-17T06:01:56.066 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 4 to deep-scrub 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 10 librados: watch_flush enter 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 10 librados: watch_flush exit 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 20 client.5143.objecter shutdown clearing up homeless session... 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 10 client.5143.objecter successfully canceled tick 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 -- 10.20.193.16:0/4115681359 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6748079140 msgr2=0x7f674807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6748079140 0x7f674807b600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6764006fd0 tx=0x7f6764009190 comp rx=0 tx=0).stop 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 10 monclient: shutdown 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 -- 10.20.193.16:0/4115681359 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 msgr2=0x7f677814ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 0x7f677814ffa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f676c02f270 tx=0x7f676c004b40 comp rx=0 tx=0).stop 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 -- 10.20.193.16:0/4115681359 shutdown_connections 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f6748079140 0x7f674807b600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f67781536f0 0x7f677814ffa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f67780c7fe0 0x7f67781f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 --2- 10.20.193.16:0/4115681359 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f67780700f0 0x7f67781f40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 -- 10.20.193.16:0/4115681359 >> 10.20.193.16:0/4115681359 conn(0x7f67780cd770 msgr2=0x7f67780cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 -- 10.20.193.16:0/4115681359 shutdown_connections 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 -- 10.20.193.16:0/4115681359 wait complete. 2026-02-17T06:01:56.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.067+0000 7f677dd9e640 1 librados: shutdown 2026-02-17T06:01:56.075 INFO:tasks.ceph:Scrubbing osd.8 2026-02-17T06:01:56.075 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:56.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.143+0000 7f1d8e6de640 10 monclient: get_monmap_and_config 2026-02-17T06:01:56.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.143+0000 7f1d8e6de640 10 monclient: build_initial_monmap 2026-02-17T06:01:56.143 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.143+0000 7f1d8e6de640 10 monclient: monmap: 2026-02-17T06:01:56.143 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:56.143 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.143 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:56.144484+0000 2026-02-17T06:01:56.143 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:56.144484+0000 2026-02-17T06:01:56.143 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:56.143 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.144 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:56.144 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:56.144 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:56.144 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:56.144 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:56.144 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:56.144 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 1 Processor -- start 2026-02-17T06:01:56.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 1 -- start start 2026-02-17T06:01:56.144 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 10 monclient: init 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 10 monclient: _add_conns ranks=[5,2,4,0,3,1] 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f1d88152660 0x7f1d88159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 10 monclient(hunting): picked mon.noname-c con 0x7f1d88152660 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1d88152a40 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1d880743a0 0x7f1d880747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 10 monclient(hunting): picked mon.noname-b con 0x7f1d880743a0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d8815d940 con 0x7f1d88152a40 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f1d8815eaf0 con 0x7f1d880743a0 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f1d88074cf0 con 0x7f1d88152660 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.144+0000 7f1d8e6de640 10 monclient(hunting): authenticate will time out at 1368.713857s 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d87fff640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f1d88152660 0x7f1d88159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d8cc54640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1d880743a0 0x7f1d880747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d8cc54640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1d880743a0 0x7f1d880747b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43326/0 (socket says 10.20.193.16:43326) 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d8cc54640 1 -- 10.20.193.16:0/1727456631 learned_addr learned my addr 10.20.193.16:0/1727456631 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d8cc54640 10 monclient(hunting): get_auth_request con 0x7f1d880743a0 auth_method 0 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d8cc54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.145 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d8cc54640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d8cc54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d87fff640 10 monclient(hunting): get_auth_request con 0x7f1d88152660 auth_method 0 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d87fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d87fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d8cc54640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d8cc54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d8cc54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.145+0000 7f1d87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.146 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.146+0000 7f1d87fff640 10 monclient(hunting): handle_auth_done global_id 5207 payload 274 2026-02-17T06:01:56.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.146+0000 7f1d87fff640 1 -- 10.20.193.16:0/1727456631 >> v2:10.20.193.128:3300/0 conn(0x7f1d880743a0 msgr2=0x7f1d880747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:56.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.146+0000 7f1d87fff640 1 --2- 10.20.193.16:0/1727456631 >> v2:10.20.193.128:3300/0 conn(0x7f1d880743a0 0x7f1d880747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.146+0000 7f1d87fff640 1 -- 10.20.193.16:0/1727456631 >> v1:10.20.193.148:6789/0 conn(0x7f1d88152a40 legacy=0x7f1d8815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:56.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.146+0000 7f1d87fff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:56.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.146+0000 7f1d87fff640 1 monclient: found mon.noname-c 2026-02-17T06:01:56.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.146+0000 7f1d87fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:56.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.146+0000 7f1d87fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:56.147 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.146+0000 7f1d87fff640 1 -- 10.20.193.16:0/1727456631 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8815dc00 con 0x7f1d88152660 2026-02-17T06:01:56.148 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.148+0000 7f1d87fff640 1 --2- 10.20.193.16:0/1727456631 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d88152660 0x7f1d88159cb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1d8806e820 tx=0x7f1d7402ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa900f6e858cbd96 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 1 -- 10.20.193.16:0/1727456631 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7403c070 con 0x7f1d88152660 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 10 monclient: dump: 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 1 -- 10.20.193.16:0/1727456631 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d7402fa50 con 0x7f1d88152660 2026-02-17T06:01:56.150 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 5 monclient: authenticate success, global_id 5207 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 1 -- 10.20.193.16:0/1727456631 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7402fe20 con 0x7f1d88152660 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 4 set_mon_vals no callback set 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d86ffd640 10 monclient: dump: 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.151 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 10 monclient: shutdown 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 1 -- 10.20.193.16:0/1727456631 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d88152660 msgr2=0x7f1d88159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 1 --2- 10.20.193.16:0/1727456631 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d88152660 0x7f1d88159cb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1d8806e820 tx=0x7f1d7402ede0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 1 -- 10.20.193.16:0/1727456631 shutdown_connections 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 1 --2- 10.20.193.16:0/1727456631 >> v2:10.20.193.128:3300/0 conn(0x7f1d880743a0 0x7f1d880747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 1 --2- 10.20.193.16:0/1727456631 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d88152660 0x7f1d88159cb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 1 -- 10.20.193.16:0/1727456631 >> 10.20.193.16:0/1727456631 conn(0x7f1d880ce480 msgr2=0x7f1d880ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 1 -- 10.20.193.16:0/1727456631 shutdown_connections 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.150+0000 7f1d8e6de640 1 -- 10.20.193.16:0/1727456631 wait complete. 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient: build_initial_monmap 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient: monmap: 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 librados: starting msgr at 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 librados: starting objecter 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 Processor -- start 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 -- start start 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 librados: setting wanted keys 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 librados: calling monclient init 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient: init 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1d880743a0 0x7f1d881f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient(hunting): picked mon.noname-a con 0x7f1d880743a0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d88152660 0x7f1d88158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient(hunting): picked mon.noname-c con 0x7f1d88152660 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d88152a40 0x7f1d88157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient(hunting): picked mon.noname-b con 0x7f1d88152a40 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:56.152 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8815e4a0 con 0x7f1d880743a0 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8815e1c0 con 0x7f1d88152a40 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8815e780 con 0x7f1d88152660 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.151+0000 7f1d8e6de640 10 monclient(hunting): authenticate will time out at 1368.720838s 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d8cc54640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d88152a40 0x7f1d88157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d8cc54640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d88152a40 0x7f1d88157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43342/0 (socket says 10.20.193.16:43342) 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d8cc54640 1 -- 10.20.193.16:0/4255600755 learned_addr learned my addr 10.20.193.16:0/4255600755 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d8cc54640 10 monclient(hunting): get_auth_request con 0x7f1d88152a40 auth_method 0 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d8cc54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d8cc54640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d8cc54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d877fe640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d88152660 0x7f1d88158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d87fff640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1d880743a0 0x7f1d881f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d877fe640 10 monclient(hunting): get_auth_request con 0x7f1d88152660 auth_method 0 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d877fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.153 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d8cc54640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d8cc54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d8cc54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d87fff640 10 monclient(hunting): get_auth_request con 0x7f1d880743a0 auth_method 0 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d87fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d87fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.152+0000 7f1d87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.153+0000 7f1d877fe640 10 monclient(hunting): handle_auth_done global_id 5210 payload 824 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.153+0000 7f1d877fe640 1 -- 10.20.193.16:0/4255600755 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d88152a40 msgr2=0x7f1d88157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.153+0000 7f1d877fe640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1d88152a40 0x7f1d88157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.153+0000 7f1d877fe640 1 -- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1d880743a0 msgr2=0x7f1d881f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.153+0000 7f1d877fe640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1d880743a0 0x7f1d881f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.153+0000 7f1d877fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.153+0000 7f1d877fe640 1 monclient: found mon.noname-c 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.153+0000 7f1d877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.153+0000 7f1d877fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:56.154 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.153+0000 7f1d877fe640 1 -- 10.20.193.16:0/4255600755 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d88158e60 con 0x7f1d88152660 2026-02-17T06:01:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.154+0000 7f1d877fe640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d88152660 0x7f1d88158420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800b550 tx=0x7f1d7800ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.154+0000 7f1d84ff9640 10 client.?.objecter ms_handle_connect 0x7f1d88152660 2026-02-17T06:01:56.155 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.154+0000 7f1d84ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 1 -- 10.20.193.16:0/4255600755 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78004500 con 0x7f1d88152660 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 10 monclient: dump: 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 10 monclient: _finish_auth 0 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d8e6de640 5 monclient: authenticate success, global_id 5210 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 1 -- 10.20.193.16:0/4255600755 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d78015070 con 0x7f1d88152660 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 1 -- 10.20.193.16:0/4255600755 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d780029c0 con 0x7f1d88152660 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d867fc640 4 set_mon_vals no callback set 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d84ff9640 10 monclient: dump: 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.157 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d8e6de640 10 monclient: _renew_subs 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d8e6de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d88159140 con 0x7f1d88152660 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d8e6de640 10 client.5210.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d8e6de640 10 monclient: _renew_subs 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d8e6de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d880c5940 con 0x7f1d88152660 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.156+0000 7f1d8e6de640 1 librados: init done 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.157+0000 7f1d8e6de640 10 client.5210.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.157+0000 7f1d8e6de640 15 client.5210.objecter _session_command_op_assign -1 1 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.157+0000 7f1d8e6de640 20 client.5210.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.157+0000 7f1d8e6de640 10 client.5210.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.157+0000 7f1d8e6de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:56.158 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.157+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d880ced20 con 0x7f1d88152660 2026-02-17T06:01:56.168 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.168+0000 7f1d84ff9640 1 -- 10.20.193.16:0/4255600755 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1d78010480 con 0x7f1d88152660 2026-02-17T06:01:56.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.169+0000 7f1d84ff9640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1d58079190 0x7f1d5807b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.169+0000 7f1d87fff640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1d58079190 0x7f1d5807b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.169+0000 7f1d87fff640 10 monclient: get_auth_request con 0x7f1d58079190 auth_method 0 2026-02-17T06:01:56.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.169+0000 7f1d84ff9640 10 client.5210.objecter ms_handle_connect 0x7f1d58079190 2026-02-17T06:01:56.169 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.169+0000 7f1d87fff640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1d58079190 0x7f1d5807b650 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1d74039000 tx=0x7f1d74033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 1 -- 10.20.193.16:0/4255600755 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1d78097d30 con 0x7f1d88152660 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 10 client.5210.objecter ms_dispatch2 0x7f1d88000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 3 client.5210.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 3 client.5210.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 10 client.5210.objecter checking command 1 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] conn(0x7f1d58081670 0x7f1d58083a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 20 client.5210.objecter _get_session s=0x7f1d58080fa0 osd=8 3 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 20 client.5210.objecter put_session s=0x7f1d58080fa0 osd=8 3 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 15 client.5210.objecter _session_command_op_remove -1 1 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 20 client.5210.objecter _get_session s=0x7f1d58080fa0 osd=8 3 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 20 client.5210.objecter get_session s=0x7f1d58080fa0 osd=8 3 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 15 client.5210.objecter _session_command_op_assign 8 1 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 20 client.5210.objecter put_session s=0x7f1d58080fa0 osd=8 4 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 10 client.5210.objecter _send_command 1 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 1 -- 10.20.193.16:0/4255600755 --> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d58084160 con 0x7f1d58081670 2026-02-17T06:01:56.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d84ff9640 20 client.5210.objecter dump_active .. 0 homeless 2026-02-17T06:01:56.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.170+0000 7f1d8cc54640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] conn(0x7f1d58081670 0x7f1d58083a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.171+0000 7f1d84ff9640 1 -- 10.20.193.16:0/4255600755 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1d7805e760 con 0x7f1d88152660 2026-02-17T06:01:56.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.171+0000 7f1d84ff9640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:56.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.171+0000 7f1d8cc54640 10 monclient: get_auth_request con 0x7f1d58081670 auth_method 0 2026-02-17T06:01:56.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.172+0000 7f1d84ff9640 10 client.5210.objecter ms_handle_connect 0x7f1d58081670 2026-02-17T06:01:56.173 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.172+0000 7f1d8cc54640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] conn(0x7f1d58081670 0x7f1d58083a90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.183+0000 7f1d84ff9640 1 -- 10.20.193.16:0/4255600755 <== osd.8 v2:10.20.193.16:6802/15364298 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f1d58084160 con 0x7f1d58081670 2026-02-17T06:01:56.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.183+0000 7f1d84ff9640 10 client.5210.objecter ms_dispatch2 0x7f1d88000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:56.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.183+0000 7f1d84ff9640 10 client.5210.objecter _finish_command 1 = system:0 2026-02-17T06:01:56.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.183+0000 7f1d84ff9640 20 client.5210.objecter put_session s=0x7f1d58080fa0 osd=8 4 2026-02-17T06:01:56.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.183+0000 7f1d84ff9640 15 client.5210.objecter _session_command_op_remove 8 1 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 10 client.5210.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 15 client.5210.objecter _session_command_op_assign -1 2 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 20 client.5210.objecter _get_session s=0x7f1d58080fa0 osd=8 3 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 20 client.5210.objecter put_session s=0x7f1d58080fa0 osd=8 3 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 20 client.5210.objecter _get_session s=0x7f1d58080fa0 osd=8 3 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 15 client.5210.objecter _session_command_op_remove -1 2 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 20 client.5210.objecter get_session s=0x7f1d58080fa0 osd=8 3 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 15 client.5210.objecter _session_command_op_assign 8 2 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 20 client.5210.objecter put_session s=0x7f1d58080fa0 osd=8 4 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 10 client.5210.objecter _send_command 2 2026-02-17T06:01:56.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.189+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 --> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1d88148570 con 0x7f1d58081670 2026-02-17T06:01:56.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d84ff9640 1 -- 10.20.193.16:0/4255600755 <== osd.8 v2:10.20.193.16:6802/15364298 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1d88148570 con 0x7f1d58081670 2026-02-17T06:01:56.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d84ff9640 10 client.5210.objecter ms_dispatch2 0x7f1d88000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d84ff9640 10 client.5210.objecter _finish_command 2 = system:0 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d84ff9640 20 client.5210.objecter put_session s=0x7f1d58080fa0 osd=8 4 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d84ff9640 15 client.5210.objecter _session_command_op_remove 8 2 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d8e6de640 10 librados: watch_flush enter 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d8e6de640 10 librados: watch_flush exit 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d8e6de640 10 client.5210.objecter close_session for osd.8 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] conn(0x7f1d58081670 msgr2=0x7f1d58083a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d8e6de640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:6802/15364298,v1:10.20.193.16:6803/15364298] conn(0x7f1d58081670 0x7f1d58083a90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d8e6de640 20 client.5210.objecter put_session s=0x7f1d58080fa0 osd=8 1 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d8e6de640 20 client.5210.objecter shutdown clearing up homeless session... 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.195+0000 7f1d8e6de640 10 client.5210.objecter successfully canceled tick 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1d58079190 msgr2=0x7f1d5807b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1d58079190 0x7f1d5807b650 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1d74039000 tx=0x7f1d74033000 comp rx=0 tx=0).stop 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 10 monclient: shutdown 2026-02-17T06:01:56.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:56.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d88152660 msgr2=0x7f1d88158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 1 --2- 10.20.193.16:0/4255600755 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1d88152660 0x7f1d88158420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800b550 tx=0x7f1d7800ba20 comp rx=0 tx=0).stop 2026-02-17T06:01:56.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8cc54640 1 -- 10.20.193.16:0/4255600755 reap_dead start 2026-02-17T06:01:56.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 shutdown_connections 2026-02-17T06:01:56.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 >> 10.20.193.16:0/4255600755 conn(0x7f1d880ce480 msgr2=0x7f1d8815f190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:56.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 shutdown_connections 2026-02-17T06:01:56.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 1 -- 10.20.193.16:0/4255600755 wait complete. 2026-02-17T06:01:56.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.196+0000 7f1d8e6de640 1 librados: shutdown 2026-02-17T06:01:56.204 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2026-02-17T06:01:56.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.270+0000 7fcdbd0de640 10 monclient: get_monmap_and_config 2026-02-17T06:01:56.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.270+0000 7fcdbd0de640 10 monclient: build_initial_monmap 2026-02-17T06:01:56.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.270+0000 7fcdbd0de640 10 monclient: monmap: 2026-02-17T06:01:56.270 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:56.271602+0000 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:56.271602+0000 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 1 Processor -- start 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 1 -- start start 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 10 monclient: init 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 10 monclient: _add_conns ranks=[0,3,4,1,5,2] 2026-02-17T06:01:56.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcdb80700f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fcdb80c86f0 0x7fcdb80704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 10 monclient(hunting): picked mon.noname-a con 0x7fcdb80c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fcdb81536f0 0x7fcdb8153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 10 monclient(hunting): picked mon.noname-b con 0x7fcdb81536f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb80c4820 con 0x7fcdb80700f0 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb80c4b00 con 0x7fcdb80c86f0 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb80c4de0 con 0x7fcdb81536f0 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.271+0000 7fcdbd0de640 10 monclient(hunting): authenticate will time out at 1368.840659s 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb7577640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fcdb81536f0 0x7fcdb8153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb6575640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fcdb80c86f0 0x7fcdb80704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb7577640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fcdb81536f0 0x7fcdb8153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43358/0 (socket says 10.20.193.16:43358) 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb7577640 1 -- 10.20.193.16:0/1993542413 learned_addr learned my addr 10.20.193.16:0/1993542413 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb7577640 10 monclient(hunting): get_auth_request con 0x7fcdb81536f0 auth_method 0 2026-02-17T06:01:56.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb7577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.273 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb7577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.273 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb7577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.274 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb6575640 10 monclient(hunting): get_auth_request con 0x7fcdb80c86f0 auth_method 0 2026-02-17T06:01:56.274 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.274 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.274 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.274 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb7577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.274 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb7577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.272+0000 7fcdb7577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb7577640 10 monclient(hunting): handle_auth_done global_id 5152 payload 274 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb7577640 1 -- 10.20.193.16:0/1993542413 >> v2:10.20.193.16:3300/0 conn(0x7fcdb80c86f0 msgr2=0x7fcdb80704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb7577640 1 --2- 10.20.193.16:0/1993542413 >> v2:10.20.193.16:3300/0 conn(0x7fcdb80c86f0 0x7fcdb80704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb7577640 1 -- 10.20.193.16:0/1993542413 >> v1:10.20.193.16:6789/0 conn(0x7fcdb80700f0 legacy=0x7fcdb80c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb7577640 10 monclient: _finish_hunting 0 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb7577640 1 monclient: found mon.noname-b 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb7577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:56.275 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb7577640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:56.276 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.273+0000 7fcdb7577640 1 -- 10.20.193.16:0/1993542413 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb80c50c0 con 0x7fcdb81536f0 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.274+0000 7fcdb7577640 1 --2- 10.20.193.16:0/1993542413 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb81536f0 0x7fcdb8153b20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80c9540 tx=0x7fcda802f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=807f7989e00ccd98 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 1 -- 10.20.193.16:0/1993542413 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda802fcf0 con 0x7fcdb81536f0 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 10 monclient: dump: 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.278 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 10 monclient: _finish_auth 0 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 10 monclient: _check_auth_tickets 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 1 -- 10.20.193.16:0/1993542413 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda80373f0 con 0x7fcdb81536f0 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 5 monclient: authenticate success, global_id 5152 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 1 -- 10.20.193.16:0/1993542413 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda80377c0 con 0x7fcdb81536f0 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:56.279 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 4 set_mon_vals no callback set 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdb5d74640 10 monclient: dump: 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.280 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 10 monclient: shutdown 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 1 -- 10.20.193.16:0/1993542413 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb81536f0 msgr2=0x7fcdb8153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 1 --2- 10.20.193.16:0/1993542413 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb81536f0 0x7fcdb8153b20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80c9540 tx=0x7fcda802f230 comp rx=0 tx=0).stop 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 1 -- 10.20.193.16:0/1993542413 shutdown_connections 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 1 --2- 10.20.193.16:0/1993542413 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb81536f0 0x7fcdb8153b20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 1 --2- 10.20.193.16:0/1993542413 >> v2:10.20.193.16:3300/0 conn(0x7fcdb80c86f0 0x7fcdb80704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 1 -- 10.20.193.16:0/1993542413 >> 10.20.193.16:0/1993542413 conn(0x7fcdb80cd770 msgr2=0x7fcdb80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 1 -- 10.20.193.16:0/1993542413 shutdown_connections 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.276+0000 7fcdbd0de640 1 -- 10.20.193.16:0/1993542413 wait complete. 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.277+0000 7fcdbd0de640 10 monclient: build_initial_monmap 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.277+0000 7fcdbd0de640 10 monclient: monmap: 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:56.281 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.277+0000 7fcdbd0de640 1 librados: starting msgr at 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.277+0000 7fcdbd0de640 1 librados: starting objecter 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.277+0000 7fcdbd0de640 1 Processor -- start 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.277+0000 7fcdbd0de640 1 -- start start 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.277+0000 7fcdbd0de640 1 librados: setting wanted keys 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.277+0000 7fcdbd0de640 1 librados: calling monclient init 2026-02-17T06:01:56.282 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.277+0000 7fcdbd0de640 10 monclient: init 2026-02-17T06:01:56.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:56.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:56.286 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb80700f0 0x7fcdb813e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 10 monclient(hunting): picked mon.noname-b con 0x7fcdb80700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcdb80c86f0 0x7fcdb8152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 10 monclient(hunting): picked mon.noname-c con 0x7fcdb80c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcdb81536f0 0x7fcdb8152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 10 monclient(hunting): picked mon.noname-a con 0x7fcdb81536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8154060 con 0x7fcdb81536f0 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb80c5680 con 0x7fcdb80700f0 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8074c00 con 0x7fcdb80c86f0 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdbd0de640 10 monclient(hunting): authenticate will time out at 1368.847152s 2026-02-17T06:01:56.287 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6d76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb80700f0 0x7fcdb813e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb7577640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcdb81536f0 0x7fcdb8152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6575640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcdb80c86f0 0x7fcdb8152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6d76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb80700f0 0x7fcdb813e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43368/0 (socket says 10.20.193.16:43368) 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6d76640 1 -- 10.20.193.16:0/3759593329 learned_addr learned my addr 10.20.193.16:0/3759593329 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6d76640 10 monclient(hunting): get_auth_request con 0x7fcdb80700f0 auth_method 0 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb7577640 10 monclient(hunting): get_auth_request con 0x7fcdb81536f0 auth_method 0 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb7577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb7577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb7577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6575640 10 monclient(hunting): get_auth_request con 0x7fcdb80c86f0 auth_method 0 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.288 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb7577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb7577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.278+0000 7fcdb7577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6d76640 10 monclient(hunting): handle_auth_done global_id 5155 payload 824 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6d76640 1 -- 10.20.193.16:0/3759593329 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcdb80c86f0 msgr2=0x7fcdb8152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6d76640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcdb80c86f0 0x7fcdb8152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6d76640 1 -- 10.20.193.16:0/3759593329 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcdb81536f0 msgr2=0x7fcdb8152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6d76640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcdb81536f0 0x7fcdb8152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6d76640 10 monclient: _finish_hunting 0 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6d76640 1 monclient: found mon.noname-b 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:56.289 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.279+0000 7fcdb6d76640 1 -- 10.20.193.16:0/3759593329 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb81411f0 con 0x7fcdb80700f0 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.281+0000 7fcdb6d76640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb80700f0 0x7fcdb813e030 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00c5b0 tx=0x7fcdac00ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.281+0000 7fcd977fe640 10 client.?.objecter ms_handle_connect 0x7fcdb80700f0 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.281+0000 7fcd977fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 1 -- 10.20.193.16:0/3759593329 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac0052d0 con 0x7fcdb80700f0 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 10 monclient: dump: 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.290 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 5 monclient: authenticate success, global_id 5155 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 10 monclient: _renew_subs 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 1 -- 10.20.193.16:0/3759593329 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb8151640 con 0x7fcdb80700f0 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 10 client.5155.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 10 monclient: _renew_subs 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 1 -- 10.20.193.16:0/3759593329 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb8151c00 con 0x7fcdb80700f0 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 1 librados: init done 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 1 -- 10.20.193.16:0/3759593329 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac005470 con 0x7fcdb80700f0 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 1 -- 10.20.193.16:0/3759593329 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac0038b0 con 0x7fcdb80700f0 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcd977fe640 10 monclient: dump: 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdb5573640 4 set_mon_vals no callback set 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:56.292 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.282+0000 7fcdbd0de640 1 -- 10.20.193.16:0/3759593329 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb80704d0 con 0x7fcdb80700f0 2026-02-17T06:01:56.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.386+0000 7fcd977fe640 1 -- 10.20.193.16:0/3759593329 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fcdac003a50 con 0x7fcdb80700f0 2026-02-17T06:01:56.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.387+0000 7fcd977fe640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcd84079140 0x7fcd8407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.387+0000 7fcdb6575640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcd84079140 0x7fcd8407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.387+0000 7fcdb6575640 10 monclient: get_auth_request con 0x7fcd84079140 auth_method 0 2026-02-17T06:01:56.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.387+0000 7fcd977fe640 10 client.5155.objecter ms_handle_connect 0x7fcd84079140 2026-02-17T06:01:56.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.387+0000 7fcdb6575640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcd84079140 0x7fcd8407b600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8141ab0 tx=0x7fcda0009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.388+0000 7fcd977fe640 1 -- 10.20.193.16:0/3759593329 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fcdac097c20 con 0x7fcdb80700f0 2026-02-17T06:01:56.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.388+0000 7fcd977fe640 10 client.5155.objecter ms_dispatch2 0x7fcdb8000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:56.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.388+0000 7fcd977fe640 3 client.5155.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:56.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.388+0000 7fcd977fe640 3 client.5155.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:56.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.388+0000 7fcd977fe640 20 client.5155.objecter dump_active .. 0 homeless 2026-02-17T06:01:56.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.394+0000 7fcd977fe640 1 -- 10.20.193.16:0/3759593329 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcdac062250 con 0x7fcdb80700f0 2026-02-17T06:01:56.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.394+0000 7fcd977fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:56.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.394+0000 7fcd977fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:56.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.480+0000 7fcdbd0de640 1 -- 10.20.193.16:0/3759593329 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "8", "target": ["mon-mgr", ""]}) -- 0x7fcdb81573f0 con 0x7fcd84079140 2026-02-17T06:01:56.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.480+0000 7fcd977fe640 1 -- 10.20.193.16:0/3759593329 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 8 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcdb81573f0 con 0x7fcd84079140 2026-02-17T06:01:56.480 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 8 to deep-scrub 2026-02-17T06:01:56.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.481+0000 7fcdbd0de640 10 librados: watch_flush enter 2026-02-17T06:01:56.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.481+0000 7fcdbd0de640 10 librados: watch_flush exit 2026-02-17T06:01:56.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.481+0000 7fcdbd0de640 20 client.5155.objecter shutdown clearing up homeless session... 2026-02-17T06:01:56.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.481+0000 7fcdbd0de640 10 client.5155.objecter successfully canceled tick 2026-02-17T06:01:56.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.481+0000 7fcdbd0de640 1 -- 10.20.193.16:0/3759593329 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcd84079140 msgr2=0x7fcd8407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.481+0000 7fcdbd0de640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcd84079140 0x7fcd8407b600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8141ab0 tx=0x7fcda0009290 comp rx=0 tx=0).stop 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.481+0000 7fcdbd0de640 10 monclient: shutdown 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.481+0000 7fcdbd0de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.481+0000 7fcdbd0de640 1 -- 10.20.193.16:0/3759593329 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb80700f0 msgr2=0x7fcdb813e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.481+0000 7fcdbd0de640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb80700f0 0x7fcdb813e030 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00c5b0 tx=0x7fcdac00ca80 comp rx=0 tx=0).stop 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.482+0000 7fcdbd0de640 1 -- 10.20.193.16:0/3759593329 shutdown_connections 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.482+0000 7fcdbd0de640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcd84079140 0x7fcd8407b600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.482+0000 7fcdbd0de640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcdb81536f0 0x7fcdb8152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.482+0000 7fcdbd0de640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcdb80c86f0 0x7fcdb8152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.482+0000 7fcdbd0de640 1 --2- 10.20.193.16:0/3759593329 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcdb80700f0 0x7fcdb813e030 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.482+0000 7fcdbd0de640 1 -- 10.20.193.16:0/3759593329 >> 10.20.193.16:0/3759593329 conn(0x7fcdb80cd770 msgr2=0x7fcdb80c6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.482+0000 7fcdbd0de640 1 -- 10.20.193.16:0/3759593329 shutdown_connections 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.482+0000 7fcdbd0de640 1 -- 10.20.193.16:0/3759593329 wait complete. 2026-02-17T06:01:56.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.482+0000 7fcdbd0de640 1 librados: shutdown 2026-02-17T06:01:56.491 INFO:tasks.ceph:Scrubbing osd.12 2026-02-17T06:01:56.491 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:56.561 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.560+0000 7f574823a640 10 monclient: get_monmap_and_config 2026-02-17T06:01:56.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.560+0000 7f574823a640 10 monclient: build_initial_monmap 2026-02-17T06:01:56.562 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.560+0000 7f574823a640 10 monclient: monmap: 2026-02-17T06:01:56.562 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:56.562 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.562 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:56.561844+0000 2026-02-17T06:01:56.562 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:56.561844+0000 2026-02-17T06:01:56.562 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:56.562 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.563 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:56.563 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:56.563 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:56.563 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:56.563 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:56.563 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:56.563 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.561+0000 7f574823a640 1 Processor -- start 2026-02-17T06:01:56.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.561+0000 7f574823a640 1 -- start start 2026-02-17T06:01:56.563 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.561+0000 7f574823a640 10 monclient: init 2026-02-17T06:01:56.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:56.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 10 monclient: _add_conns ranks=[3,0,2,4,5,1] 2026-02-17T06:01:56.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f574014f020 0x7f574006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.566 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 10 monclient(hunting): picked mon.noname-a con 0x7f574014f020 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f574006e120 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f574015caf0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57400c5b10 con 0x7f574006e120 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574006e620 con 0x7f574015caf0 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f574006e900 con 0x7f574014f020 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f574823a640 10 monclient(hunting): authenticate will time out at 1369.131101s 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f5745faf640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f574014f020 0x7f574006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f57467b0640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f574015caf0 0x7f57400c6010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:51300/0 (socket says 10.20.193.16:51300) 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f57467b0640 1 -- 10.20.193.16:0/3329335468 learned_addr learned my addr 10.20.193.16:0/3329335468 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:56.567 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f5745faf640 10 monclient(hunting): get_auth_request con 0x7f574014f020 auth_method 0 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f5745faf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f5745faf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.562+0000 7f5745faf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.563+0000 7f5745faf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.563+0000 7f5745faf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.563+0000 7f5745faf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5744fad640 1 -- 10.20.193.16:0/3329335468 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1570532689 0 0) 0x7f574006e620 con 0x7f574015caf0 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5744fad640 10 monclient(hunting): _negotiate 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5744fad640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5744fad640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5744fad640 10 monclient(hunting): my global_id is 5216 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5744fad640 1 -- 10.20.193.16:0/3329335468 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5728003620 con 0x7f574015caf0 2026-02-17T06:01:56.568 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5745faf640 10 monclient(hunting): handle_auth_done global_id 5073 payload 274 2026-02-17T06:01:56.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5745faf640 1 -- 10.20.193.16:0/3329335468 >> v1:10.20.193.148:6789/0 conn(0x7f574015caf0 legacy=0x7f57400c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5745faf640 1 -- 10.20.193.16:0/3329335468 >> v1:10.20.193.16:6789/0 conn(0x7f574006e120 legacy=0x7f574015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5745faf640 10 monclient: _finish_hunting 0 2026-02-17T06:01:56.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5745faf640 1 monclient: found mon.noname-a 2026-02-17T06:01:56.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5745faf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:56.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5745faf640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:56.569 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.564+0000 7f5745faf640 1 -- 10.20.193.16:0/3329335468 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574006ebe0 con 0x7f574014f020 2026-02-17T06:01:56.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.566+0000 7f5745faf640 1 --2- 10.20.193.16:0/3329335468 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f574014f020 0x7f574006dbe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f57300098a0 tx=0x7f573002ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507e666b72f3f6bd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.568+0000 7f5744fad640 1 -- 10.20.193.16:0/3329335468 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573003c070 con 0x7f574014f020 2026-02-17T06:01:56.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.568+0000 7f5744fad640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.568+0000 7f5744fad640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:56.577 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.568+0000 7f5744fad640 10 monclient: dump: 2026-02-17T06:01:56.577 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.577 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.577 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.578 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.578 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.578 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.578 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.578 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.578 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.578 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.568+0000 7f5744fad640 10 monclient: _finish_auth 0 2026-02-17T06:01:56.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.568+0000 7f5744fad640 10 monclient: _check_auth_tickets 2026-02-17T06:01:56.578 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.568+0000 7f5744fad640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.568+0000 7f5744fad640 1 -- 10.20.193.16:0/3329335468 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573002fc10 con 0x7f574014f020 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 5 monclient: authenticate success, global_id 5073 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f5744fad640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 4 set_mon_vals no callback set 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f5744fad640 1 -- 10.20.193.16:0/3329335468 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730037440 con 0x7f574014f020 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f5744fad640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f5744fad640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f5744fad640 10 monclient: dump: 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.579 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 10 monclient: shutdown 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:56.580 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 1 -- 10.20.193.16:0/3329335468 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f574014f020 msgr2=0x7f574006dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 1 --2- 10.20.193.16:0/3329335468 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f574014f020 0x7f574006dbe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f57300098a0 tx=0x7f573002ef00 comp rx=0 tx=0).stop 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 1 -- 10.20.193.16:0/3329335468 shutdown_connections 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 1 --2- 10.20.193.16:0/3329335468 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f574014f020 0x7f574006dbe0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 1 -- 10.20.193.16:0/3329335468 >> 10.20.193.16:0/3329335468 conn(0x7f57400cd770 msgr2=0x7f57400cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 1 -- 10.20.193.16:0/3329335468 shutdown_connections 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 1 -- 10.20.193.16:0/3329335468 wait complete. 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 10 monclient: build_initial_monmap 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 10 monclient: monmap: 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:56.581 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 1 librados: starting msgr at 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.569+0000 7f574823a640 1 librados: starting objecter 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 1 Processor -- start 2026-02-17T06:01:56.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 1 -- start start 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 1 librados: setting wanted keys 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 1 librados: calling monclient init 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient: init 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f574006e120 0x7f57401449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient(hunting): picked mon.noname-c con 0x7f574006e120 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f574014f020 0x7f5740073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient(hunting): picked mon.noname-b con 0x7f574014f020 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f574015caf0 0x7f57400cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.583 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient(hunting): picked mon.noname-a con 0x7f574015caf0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57400c6720 con 0x7f574015caf0 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f574006f1a0 con 0x7f574014f020 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f57400c68a0 con 0x7f574006e120 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f574823a640 10 monclient(hunting): authenticate will time out at 1369.139465s 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f5745faf640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f574006e120 0x7f57401449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f57467b0640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f574015caf0 0x7f57400cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.584 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.570+0000 7f57457ae640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f574014f020 0x7f5740073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f5745faf640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f574006e120 0x7f57401449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:53222/0 (socket says 10.20.193.16:53222) 2026-02-17T06:01:56.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f5745faf640 1 -- 10.20.193.16:0/987541413 learned_addr learned my addr 10.20.193.16:0/987541413 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:56.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f5745faf640 10 monclient(hunting): get_auth_request con 0x7f574006e120 auth_method 0 2026-02-17T06:01:56.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f5745faf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f5745faf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f5745faf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57467b0640 10 monclient(hunting): get_auth_request con 0x7f574015caf0 auth_method 0 2026-02-17T06:01:56.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57467b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57467b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57467b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57457ae640 10 monclient(hunting): get_auth_request con 0x7f574014f020 auth_method 0 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57457ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57457ae640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57457ae640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f5745faf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f5745faf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f5745faf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57467b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57467b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57467b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57457ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57457ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.586 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.571+0000 7f57457ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.572+0000 7f5745faf640 10 monclient(hunting): handle_auth_done global_id 5219 payload 824 2026-02-17T06:01:56.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.572+0000 7f5745faf640 1 -- 10.20.193.16:0/987541413 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f574014f020 msgr2=0x7f5740073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.572+0000 7f5745faf640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f574014f020 0x7f5740073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.572+0000 7f5745faf640 1 -- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f574015caf0 msgr2=0x7f57400cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.572+0000 7f5745faf640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f574015caf0 0x7f57400cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.572+0000 7f5745faf640 10 monclient: _finish_hunting 0 2026-02-17T06:01:56.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.572+0000 7f5745faf640 1 monclient: found mon.noname-c 2026-02-17T06:01:56.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.572+0000 7f5745faf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:56.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.572+0000 7f5745faf640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:56.587 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.572+0000 7f5745faf640 1 -- 10.20.193.16:0/987541413 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5740072b80 con 0x7f574006e120 2026-02-17T06:01:56.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.573+0000 7f5745faf640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f574006e120 0x7f57401449b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5730009130 tx=0x7f57300047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.573+0000 7f572e7fc640 10 client.?.objecter ms_handle_connect 0x7f574006e120 2026-02-17T06:01:56.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.573+0000 7f572e7fc640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:56.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 1 -- 10.20.193.16:0/987541413 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573003c040 con 0x7f574006e120 2026-02-17T06:01:56.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:56.588 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 10 monclient: dump: 2026-02-17T06:01:56.588 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.588 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:56.589 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 5 monclient: authenticate success, global_id 5219 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 10 monclient: _renew_subs 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5740072e40 con 0x7f574006e120 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 10 client.5219.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 10 monclient: _renew_subs 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5740073400 con 0x7f574006e120 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 1 librados: init done 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 1 -- 10.20.193.16:0/987541413 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573002fb50 con 0x7f574006e120 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572ffff640 4 set_mon_vals no callback set 2026-02-17T06:01:56.590 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 1 -- 10.20.193.16:0/987541413 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730042810 con 0x7f574006e120 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f572e7fc640 10 monclient: dump: 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.591 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.592 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.592 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 10 client.5219.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:56.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 15 client.5219.objecter _session_command_op_assign -1 1 2026-02-17T06:01:56.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 20 client.5219.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:56.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 10 client.5219.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:56.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:56.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.575+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57400cded0 con 0x7f574006e120 2026-02-17T06:01:56.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.586+0000 7f572e7fc640 1 -- 10.20.193.16:0/987541413 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f57300429b0 con 0x7f574006e120 2026-02-17T06:01:56.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.586+0000 7f572e7fc640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5710079170 0x7f571007b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.586+0000 7f57457ae640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5710079170 0x7f571007b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.586+0000 7f57457ae640 10 monclient: get_auth_request con 0x7f5710079170 auth_method 0 2026-02-17T06:01:56.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.587+0000 7f572e7fc640 10 client.5219.objecter ms_handle_connect 0x7f5710079170 2026-02-17T06:01:56.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.587+0000 7f57457ae640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5710079170 0x7f571007b630 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57400cd040 tx=0x7f5734007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 1 -- 10.20.193.16:0/987541413 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f57300bc6a0 con 0x7f574006e120 2026-02-17T06:01:56.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 10 client.5219.objecter ms_dispatch2 0x7f5740000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 3 client.5219.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 3 client.5219.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 10 client.5219.objecter checking command 1 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] conn(0x7f57100816b0 0x7f5710083ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 20 client.5219.objecter _get_session s=0x7f5710080fe0 osd=12 3 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 20 client.5219.objecter put_session s=0x7f5710080fe0 osd=12 3 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 15 client.5219.objecter _session_command_op_remove -1 1 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 20 client.5219.objecter _get_session s=0x7f5710080fe0 osd=12 3 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 20 client.5219.objecter get_session s=0x7f5710080fe0 osd=12 3 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 15 client.5219.objecter _session_command_op_assign 12 1 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 20 client.5219.objecter put_session s=0x7f5710080fe0 osd=12 4 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 10 client.5219.objecter _send_command 1 2026-02-17T06:01:56.596 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 1 -- 10.20.193.16:0/987541413 --> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f573004be50 con 0x7f57100816b0 2026-02-17T06:01:56.597 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 20 client.5219.objecter dump_active .. 0 homeless 2026-02-17T06:01:56.597 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 1 -- 10.20.193.16:0/987541413 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5730083160 con 0x7f574006e120 2026-02-17T06:01:56.597 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.588+0000 7f572e7fc640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:56.597 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.589+0000 7f57467b0640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] conn(0x7f57100816b0 0x7f5710083ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.597 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.589+0000 7f57467b0640 10 monclient: get_auth_request con 0x7f57100816b0 auth_method 0 2026-02-17T06:01:56.597 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.590+0000 7f572e7fc640 10 client.5219.objecter ms_handle_connect 0x7f57100816b0 2026-02-17T06:01:56.597 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.590+0000 7f57467b0640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] conn(0x7f57100816b0 0x7f5710083ad0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.12 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.601+0000 7f572e7fc640 1 -- 10.20.193.16:0/987541413 <== osd.12 v2:10.20.193.16:6810/3960967243 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f573004be50 con 0x7f57100816b0 2026-02-17T06:01:56.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.601+0000 7f572e7fc640 10 client.5219.objecter ms_dispatch2 0x7f5740000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:56.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.601+0000 7f572e7fc640 10 client.5219.objecter _finish_command 1 = system:0 2026-02-17T06:01:56.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.601+0000 7f572e7fc640 20 client.5219.objecter put_session s=0x7f5710080fe0 osd=12 4 2026-02-17T06:01:56.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.601+0000 7f572e7fc640 15 client.5219.objecter _session_command_op_remove 12 1 2026-02-17T06:01:56.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 10 client.5219.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:56.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 15 client.5219.objecter _session_command_op_assign -1 2 2026-02-17T06:01:56.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 20 client.5219.objecter _get_session s=0x7f5710080fe0 osd=12 3 2026-02-17T06:01:56.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 20 client.5219.objecter put_session s=0x7f5710080fe0 osd=12 3 2026-02-17T06:01:56.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 20 client.5219.objecter _get_session s=0x7f5710080fe0 osd=12 3 2026-02-17T06:01:56.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 15 client.5219.objecter _session_command_op_remove -1 2 2026-02-17T06:01:56.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 20 client.5219.objecter get_session s=0x7f5710080fe0 osd=12 3 2026-02-17T06:01:56.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 15 client.5219.objecter _session_command_op_assign 12 2 2026-02-17T06:01:56.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 20 client.5219.objecter put_session s=0x7f5710080fe0 osd=12 4 2026-02-17T06:01:56.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 10 client.5219.objecter _send_command 2 2026-02-17T06:01:56.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.607+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 --> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f574014f400 con 0x7f57100816b0 2026-02-17T06:01:56.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f572e7fc640 1 -- 10.20.193.16:0/987541413 <== osd.12 v2:10.20.193.16:6810/3960967243 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f574014f400 con 0x7f57100816b0 2026-02-17T06:01:56.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f572e7fc640 10 client.5219.objecter ms_dispatch2 0x7f5740000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:56.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f572e7fc640 10 client.5219.objecter _finish_command 2 = system:0 2026-02-17T06:01:56.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f572e7fc640 20 client.5219.objecter put_session s=0x7f5710080fe0 osd=12 4 2026-02-17T06:01:56.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f572e7fc640 15 client.5219.objecter _session_command_op_remove 12 2 2026-02-17T06:01:56.615 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:56.615 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:56.615 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:56.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f574823a640 10 librados: watch_flush enter 2026-02-17T06:01:56.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f574823a640 10 librados: watch_flush exit 2026-02-17T06:01:56.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f574823a640 10 client.5219.objecter close_session for osd.12 2026-02-17T06:01:56.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] conn(0x7f57100816b0 msgr2=0x7f5710083ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f574823a640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:6810/3960967243,v1:10.20.193.16:6811/3960967243] conn(0x7f57100816b0 0x7f5710083ad0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f574823a640 20 client.5219.objecter put_session s=0x7f5710080fe0 osd=12 1 2026-02-17T06:01:56.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f574823a640 20 client.5219.objecter shutdown clearing up homeless session... 2026-02-17T06:01:56.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.613+0000 7f574823a640 10 client.5219.objecter successfully canceled tick 2026-02-17T06:01:56.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5710079170 msgr2=0x7f571007b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f5710079170 0x7f571007b630 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57400cd040 tx=0x7f5734007ef0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 10 monclient: shutdown 2026-02-17T06:01:56.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:56.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f574006e120 msgr2=0x7f57401449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 1 --2- 10.20.193.16:0/987541413 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f574006e120 0x7f57401449b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5730009130 tx=0x7f57300047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f57467b0640 1 -- 10.20.193.16:0/987541413 reap_dead start 2026-02-17T06:01:56.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 shutdown_connections 2026-02-17T06:01:56.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 >> 10.20.193.16:0/987541413 conn(0x7f57400cd770 msgr2=0x7f57400c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:56.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 shutdown_connections 2026-02-17T06:01:56.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 1 -- 10.20.193.16:0/987541413 wait complete. 2026-02-17T06:01:56.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.614+0000 7f574823a640 1 librados: shutdown 2026-02-17T06:01:56.622 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2026-02-17T06:01:56.690 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.689+0000 7ff384b4a640 10 monclient: get_monmap_and_config 2026-02-17T06:01:56.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.689+0000 7ff384b4a640 10 monclient: build_initial_monmap 2026-02-17T06:01:56.691 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.689+0000 7ff384b4a640 10 monclient: monmap: 2026-02-17T06:01:56.691 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:56.691 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.691 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:56.690808+0000 2026-02-17T06:01:56.691 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:56.690808+0000 2026-02-17T06:01:56.691 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:56.692 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.692 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:56.692 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:56.692 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:56.692 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:56.692 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:56.692 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:56.692 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 1 Processor -- start 2026-02-17T06:01:56.692 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 1 -- start start 2026-02-17T06:01:56.693 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 10 monclient: init 2026-02-17T06:01:56.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:56.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 10 monclient: _add_conns ranks=[3,0,4,5,2,1] 2026-02-17T06:01:56.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7ff380153590 0x7ff380153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 10 monclient(hunting): picked mon.noname-a con 0x7ff380153590 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:56.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff3800722b0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:56.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7ff380072ff0 0x7ff380073480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 10 monclient(hunting): picked mon.noname-b con 0x7ff380072ff0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:56.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3800739c0 con 0x7ff3800722b0 2026-02-17T06:01:56.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff3800c3fe0 con 0x7ff380153590 2026-02-17T06:01:56.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.705 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7ff38015a7f0 con 0x7ff380072ff0 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.690+0000 7ff384b4a640 10 monclient(hunting): authenticate will time out at 1369.259859s 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37f7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7ff380153590 0x7ff380153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37ffff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7ff380072ff0 0x7ff380073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37effd640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7ff3800722b0 0x7ff3800c38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:41870/0 (socket says 10.20.193.16:41870) 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37effd640 1 -- 10.20.193.16:0/3472102038 learned_addr learned my addr 10.20.193.16:0/3472102038 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37ffff640 10 monclient(hunting): get_auth_request con 0x7ff380072ff0 auth_method 0 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.706 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37f7fe640 10 monclient(hunting): get_auth_request con 0x7ff380153590 auth_method 0 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.691+0000 7ff37f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.692+0000 7ff37ffff640 10 monclient(hunting): handle_auth_done global_id 5161 payload 274 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.692+0000 7ff37ffff640 1 -- 10.20.193.16:0/3472102038 >> v2:10.20.193.16:3300/0 conn(0x7ff380153590 msgr2=0x7ff380153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:56.707 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.692+0000 7ff37ffff640 1 --2- 10.20.193.16:0/3472102038 >> v2:10.20.193.16:3300/0 conn(0x7ff380153590 0x7ff380153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.692+0000 7ff37ffff640 1 -- 10.20.193.16:0/3472102038 >> v1:10.20.193.16:6789/0 conn(0x7ff3800722b0 legacy=0x7ff3800c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.692+0000 7ff37ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.692+0000 7ff37ffff640 1 monclient: found mon.noname-b 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.692+0000 7ff37ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.692+0000 7ff37ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.692+0000 7ff37ffff640 1 -- 10.20.193.16:0/3472102038 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff380072a10 con 0x7ff380072ff0 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.693+0000 7ff37ffff640 1 --2- 10.20.193.16:0/3472102038 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff380072ff0 0x7ff380073480 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff374009d90 tx=0x7ff37402f190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76b9e406f16d410d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 1 -- 10.20.193.16:0/3472102038 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37403c070 con 0x7ff380072ff0 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 10 monclient: dump: 2026-02-17T06:01:56.708 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:56.709 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 1 -- 10.20.193.16:0/3472102038 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff374037440 con 0x7ff380072ff0 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 5 monclient: authenticate success, global_id 5161 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 1 -- 10.20.193.16:0/3472102038 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37402fe60 con 0x7ff380072ff0 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff37e7fc640 10 monclient: dump: 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.710 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 4 set_mon_vals no callback set 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 10 monclient: shutdown 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 1 -- 10.20.193.16:0/3472102038 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff380072ff0 msgr2=0x7ff380073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 1 --2- 10.20.193.16:0/3472102038 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff380072ff0 0x7ff380073480 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff374009d90 tx=0x7ff37402f190 comp rx=0 tx=0).stop 2026-02-17T06:01:56.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 1 -- 10.20.193.16:0/3472102038 shutdown_connections 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 1 --2- 10.20.193.16:0/3472102038 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff380072ff0 0x7ff380073480 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 1 --2- 10.20.193.16:0/3472102038 >> v2:10.20.193.16:3300/0 conn(0x7ff380153590 0x7ff380153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 1 -- 10.20.193.16:0/3472102038 >> 10.20.193.16:0/3472102038 conn(0x7ff3800c9b60 msgr2=0x7ff3800c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 1 -- 10.20.193.16:0/3472102038 shutdown_connections 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 1 -- 10.20.193.16:0/3472102038 wait complete. 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 10 monclient: build_initial_monmap 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 10 monclient: monmap: 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:56.712 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 1 librados: starting msgr at 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.695+0000 7ff384b4a640 1 librados: starting objecter 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 1 Processor -- start 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 1 -- start start 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 1 librados: setting wanted keys 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 1 librados: calling monclient init 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient: init 2026-02-17T06:01:56.713 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff3800722b0 0x7ff38015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient(hunting): picked mon.noname-c con 0x7ff3800722b0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff380072ff0 0x7ff3801f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient(hunting): picked mon.noname-a con 0x7ff380072ff0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff380153590 0x7ff3801f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient(hunting): picked mon.noname-b con 0x7ff380153590 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff380159f50 con 0x7ff380072ff0 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801599d0 con 0x7ff380153590 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.714 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff38015a230 con 0x7ff3800722b0 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff384b4a640 10 monclient(hunting): authenticate will time out at 1369.265643s 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff37effd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff380072ff0 0x7ff3801f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.696+0000 7ff37ffff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff380153590 0x7ff3801f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37effd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff380072ff0 0x7ff3801f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33776/0 (socket says 10.20.193.16:33776) 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37effd640 1 -- 10.20.193.16:0/3876987440 learned_addr learned my addr 10.20.193.16:0/3876987440 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37effd640 10 monclient(hunting): get_auth_request con 0x7ff380072ff0 auth_method 0 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37ffff640 10 monclient(hunting): get_auth_request con 0x7ff380153590 auth_method 0 2026-02-17T06:01:56.715 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37f7fe640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff3800722b0 0x7ff38015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37f7fe640 10 monclient(hunting): get_auth_request con 0x7ff3800722b0 auth_method 0 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.716 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.697+0000 7ff37f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.698+0000 7ff37ffff640 10 monclient(hunting): handle_auth_done global_id 5164 payload 824 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.698+0000 7ff37ffff640 1 -- 10.20.193.16:0/3876987440 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff3800722b0 msgr2=0x7ff38015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.698+0000 7ff37ffff640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff3800722b0 0x7ff38015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.698+0000 7ff37ffff640 1 -- 10.20.193.16:0/3876987440 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff380072ff0 msgr2=0x7ff3801f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.698+0000 7ff37ffff640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff380072ff0 0x7ff3801f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.698+0000 7ff37ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.698+0000 7ff37ffff640 1 monclient: found mon.noname-b 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.698+0000 7ff37ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:56.717 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.698+0000 7ff37ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.698+0000 7ff37ffff640 1 -- 10.20.193.16:0/3876987440 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3800c5990 con 0x7ff380153590 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.699+0000 7ff37ffff640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff380153590 0x7ff3801f4f40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff374036000 tx=0x7ff374004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.699+0000 7ff35bfff640 10 client.?.objecter ms_handle_connect 0x7ff380153590 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.699+0000 7ff35bfff640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 1 -- 10.20.193.16:0/3876987440 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374047020 con 0x7ff380153590 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 10 monclient: dump: 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.718 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 10 monclient: _finish_auth 0 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 1 -- 10.20.193.16:0/3876987440 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff37403d040 con 0x7ff380153590 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:56.719 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 1 -- 10.20.193.16:0/3876987440 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37403c050 con 0x7ff380153590 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff35bfff640 10 monclient: dump: 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.720 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff37dffb640 4 set_mon_vals no callback set 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 5 monclient: authenticate success, global_id 5164 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 10 monclient: _renew_subs 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 1 -- 10.20.193.16:0/3876987440 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3800c5c30 con 0x7ff380153590 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 10 client.5164.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 10 monclient: _renew_subs 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 1 -- 10.20.193.16:0/3876987440 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3800c61f0 con 0x7ff380153590 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 1 librados: init done 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:56.721 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.701+0000 7ff384b4a640 1 -- 10.20.193.16:0/3876987440 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3800726f0 con 0x7ff380153590 2026-02-17T06:01:56.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.808+0000 7ff35bfff640 1 -- 10.20.193.16:0/3876987440 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7ff374041610 con 0x7ff380153590 2026-02-17T06:01:56.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.808+0000 7ff35bfff640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff350079140 0x7ff35007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.809+0000 7ff37f7fe640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff350079140 0x7ff35007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.809+0000 7ff37f7fe640 10 monclient: get_auth_request con 0x7ff350079140 auth_method 0 2026-02-17T06:01:56.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.809+0000 7ff37f7fe640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff350079140 0x7ff35007b600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff3800c7360 tx=0x7ff36c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.809+0000 7ff35bfff640 10 client.5164.objecter ms_handle_connect 0x7ff350079140 2026-02-17T06:01:56.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.810+0000 7ff35bfff640 1 -- 10.20.193.16:0/3876987440 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7ff374083d60 con 0x7ff380153590 2026-02-17T06:01:56.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.810+0000 7ff35bfff640 10 client.5164.objecter ms_dispatch2 0x7ff380000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:56.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.810+0000 7ff35bfff640 3 client.5164.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:56.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.810+0000 7ff35bfff640 3 client.5164.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:56.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.810+0000 7ff35bfff640 20 client.5164.objecter dump_active .. 0 homeless 2026-02-17T06:01:56.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.816+0000 7ff35bfff640 1 -- 10.20.193.16:0/3876987440 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff374051060 con 0x7ff380153590 2026-02-17T06:01:56.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.816+0000 7ff35bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:56.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.816+0000 7ff35bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:56.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.900+0000 7ff384b4a640 1 -- 10.20.193.16:0/3876987440 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "12", "target": ["mon-mgr", ""]}) -- 0x7ff380157250 con 0x7ff350079140 2026-02-17T06:01:56.901 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.900+0000 7ff35bfff640 1 -- 10.20.193.16:0/3876987440 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 12 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7ff380157250 con 0x7ff350079140 2026-02-17T06:01:56.901 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 12 to deep-scrub 2026-02-17T06:01:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 10 librados: watch_flush enter 2026-02-17T06:01:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 10 librados: watch_flush exit 2026-02-17T06:01:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 20 client.5164.objecter shutdown clearing up homeless session... 2026-02-17T06:01:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 10 client.5164.objecter successfully canceled tick 2026-02-17T06:01:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 -- 10.20.193.16:0/3876987440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff350079140 msgr2=0x7ff35007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff350079140 0x7ff35007b600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff3800c7360 tx=0x7ff36c009290 comp rx=0 tx=0).stop 2026-02-17T06:01:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 10 monclient: shutdown 2026-02-17T06:01:56.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 -- 10.20.193.16:0/3876987440 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff380153590 msgr2=0x7ff3801f4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff380153590 0x7ff3801f4f40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff374036000 tx=0x7ff374004b40 comp rx=0 tx=0).stop 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 -- 10.20.193.16:0/3876987440 shutdown_connections 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff350079140 0x7ff35007b600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ff380153590 0x7ff3801f4f40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ff380072ff0 0x7ff3801f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 --2- 10.20.193.16:0/3876987440 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ff3800722b0 0x7ff38015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 -- 10.20.193.16:0/3876987440 >> 10.20.193.16:0/3876987440 conn(0x7ff3800c9b60 msgr2=0x7ff380073b90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 -- 10.20.193.16:0/3876987440 shutdown_connections 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 -- 10.20.193.16:0/3876987440 wait complete. 2026-02-17T06:01:56.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.901+0000 7ff384b4a640 1 librados: shutdown 2026-02-17T06:01:56.911 INFO:tasks.ceph:Scrubbing osd.1 2026-02-17T06:01:56.912 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:56.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.982+0000 7f35a5066640 10 monclient: get_monmap_and_config 2026-02-17T06:01:56.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.982+0000 7f35a5066640 10 monclient: build_initial_monmap 2026-02-17T06:01:56.983 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.982+0000 7f35a5066640 10 monclient: monmap: 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:56.983539+0000 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:56.983539+0000 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:56.984 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:56.985 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.982+0000 7f35a5066640 1 Processor -- start 2026-02-17T06:01:56.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 1 -- start start 2026-02-17T06:01:56.985 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 10 monclient: init 2026-02-17T06:01:56.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:56.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 10 monclient: _add_conns ranks=[5,2,4,0,3,1] 2026-02-17T06:01:56.987 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f35a00700f0 0x7f35a00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 10 monclient(hunting): picked mon.noname-c con 0x7f35a00700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f35a00c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f35a01536f0 0x7f35a0153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 10 monclient(hunting): picked mon.noname-b con 0x7f35a01536f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a00c4820 con 0x7f35a00c7fe0 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f35a00c4b00 con 0x7f35a01536f0 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f35a00c4de0 con 0x7f35a00700f0 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:56.988 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f35a5066640 10 monclient(hunting): authenticate will time out at 1369.552547s 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f359ed76640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f35a00700f0 0x7f35a00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.983+0000 7f359f577640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f35a01536f0 0x7f35a0153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359f577640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f35a01536f0 0x7f35a0153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43412/0 (socket says 10.20.193.16:43412) 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359f577640 1 -- 10.20.193.16:0/2110279848 learned_addr learned my addr 10.20.193.16:0/2110279848 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359f577640 10 monclient(hunting): get_auth_request con 0x7f35a01536f0 auth_method 0 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359f577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359f577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359f577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359ed76640 10 monclient(hunting): get_auth_request con 0x7f35a00700f0 auth_method 0 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.989 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359f577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359f577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359f577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.984+0000 7f359ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.985+0000 7f359ed76640 10 monclient(hunting): handle_auth_done global_id 5225 payload 274 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.985+0000 7f359ed76640 1 -- 10.20.193.16:0/2110279848 >> v2:10.20.193.128:3300/0 conn(0x7f35a01536f0 msgr2=0x7f35a0153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.985+0000 7f359ed76640 1 --2- 10.20.193.16:0/2110279848 >> v2:10.20.193.128:3300/0 conn(0x7f35a01536f0 0x7f35a0153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.985+0000 7f359ed76640 1 -- 10.20.193.16:0/2110279848 >> v1:10.20.193.148:6789/0 conn(0x7f35a00c7fe0 legacy=0x7f35a00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.985+0000 7f359ed76640 10 monclient: _finish_hunting 0 2026-02-17T06:01:56.990 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.985+0000 7f359ed76640 1 monclient: found mon.noname-c 2026-02-17T06:01:56.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.985+0000 7f359ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:56.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.985+0000 7f359ed76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:56.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.985+0000 7f359ed76640 1 -- 10.20.193.16:0/2110279848 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a00c50c0 con 0x7f35a00700f0 2026-02-17T06:01:56.991 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.986+0000 7f359ed76640 1 --2- 10.20.193.16:0/2110279848 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35a00700f0 0x7f35a00704d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f35a0070b90 tx=0x7f359402ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a4f377f84891aa server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:56.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.987+0000 7f359dd74640 1 -- 10.20.193.16:0/2110279848 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359403c070 con 0x7f35a00700f0 2026-02-17T06:01:56.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.987+0000 7f359dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.987+0000 7f359dd74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:56.997 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.987+0000 7f359dd74640 10 monclient: dump: 2026-02-17T06:01:56.997 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.987+0000 7f359dd74640 10 monclient: _finish_auth 0 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.987+0000 7f359dd74640 10 monclient: _check_auth_tickets 2026-02-17T06:01:56.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.987+0000 7f359dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.987+0000 7f35a5066640 5 monclient: authenticate success, global_id 5225 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.987+0000 7f35a5066640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f359dd74640 1 -- 10.20.193.16:0/2110279848 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359402f9c0 con 0x7f35a00700f0 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f359dd74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f359dd74640 1 -- 10.20.193.16:0/2110279848 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359402fd90 con 0x7f35a00700f0 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 4 set_mon_vals no callback set 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f359dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f359dd74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f359dd74640 10 monclient: dump: 2026-02-17T06:01:56.999 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 10 monclient: shutdown 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 1 -- 10.20.193.16:0/2110279848 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35a00700f0 msgr2=0x7f35a00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 1 --2- 10.20.193.16:0/2110279848 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35a00700f0 0x7f35a00704d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f35a0070b90 tx=0x7f359402ed50 comp rx=0 tx=0).stop 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 1 -- 10.20.193.16:0/2110279848 shutdown_connections 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 1 --2- 10.20.193.16:0/2110279848 >> v2:10.20.193.128:3300/0 conn(0x7f35a01536f0 0x7f35a0153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 1 --2- 10.20.193.16:0/2110279848 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35a00700f0 0x7f35a00704d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 1 -- 10.20.193.16:0/2110279848 >> 10.20.193.16:0/2110279848 conn(0x7f35a00cd770 msgr2=0x7f35a00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 1 -- 10.20.193.16:0/2110279848 shutdown_connections 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.988+0000 7f35a5066640 1 -- 10.20.193.16:0/2110279848 wait complete. 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient: monmap: 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:57.001 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 librados: starting msgr at 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 librados: starting objecter 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 Processor -- start 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 -- start start 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 librados: setting wanted keys 2026-02-17T06:01:57.002 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 librados: calling monclient init 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient: init 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f35a00700f0 0x7f35a01f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient(hunting): picked mon.noname-a con 0x7f35a00700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35a00c7fe0 0x7f35a0142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient(hunting): picked mon.noname-c con 0x7f35a00c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f35a01536f0 0x7f35a0072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient(hunting): picked mon.noname-b con 0x7f35a01536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0154060 con 0x7f35a00700f0 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.003 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f35a00c5680 con 0x7f35a01536f0 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0074c00 con 0x7f35a00c7fe0 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.989+0000 7f35a5066640 10 monclient(hunting): authenticate will time out at 1369.558727s 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359e575640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35a00c7fe0 0x7f35a0142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359f577640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f35a01536f0 0x7f35a0072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359ed76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f35a00700f0 0x7f35a01f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359e575640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35a00c7fe0 0x7f35a0142970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:53250/0 (socket says 10.20.193.16:53250) 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359e575640 1 -- 10.20.193.16:0/339446743 learned_addr learned my addr 10.20.193.16:0/339446743 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359e575640 10 monclient(hunting): get_auth_request con 0x7f35a00c7fe0 auth_method 0 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.004 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359f577640 10 monclient(hunting): get_auth_request con 0x7f35a01536f0 auth_method 0 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359f577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359f577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359f577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359ed76640 10 monclient(hunting): get_auth_request con 0x7f35a00700f0 auth_method 0 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359f577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.005 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359f577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359f577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.990+0000 7f359ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.991+0000 7f359e575640 10 monclient(hunting): handle_auth_done global_id 5228 payload 824 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.991+0000 7f359e575640 1 -- 10.20.193.16:0/339446743 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f35a01536f0 msgr2=0x7f35a0072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.991+0000 7f359e575640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f35a01536f0 0x7f35a0072a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.991+0000 7f359e575640 1 -- 10.20.193.16:0/339446743 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f35a00700f0 msgr2=0x7f35a01f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.991+0000 7f359e575640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f35a00700f0 0x7f35a01f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.991+0000 7f359e575640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.991+0000 7f359e575640 1 monclient: found mon.noname-c 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.991+0000 7f359e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.991+0000 7f359e575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:57.006 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.991+0000 7f359e575640 1 -- 10.20.193.16:0/339446743 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a0142490 con 0x7f35a00c7fe0 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.992+0000 7f359e575640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35a00c7fe0 0x7f35a0142970 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f358800b5a0 tx=0x7f358800ba70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.992+0000 7f357f7fe640 10 client.?.objecter ms_handle_connect 0x7f35a00c7fe0 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.992+0000 7f357f7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 1 -- 10.20.193.16:0/339446743 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588012070 con 0x7f35a00c7fe0 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 10 monclient: dump: 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.007 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 5 monclient: authenticate success, global_id 5228 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 10 monclient: _renew_subs 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a00cb220 con 0x7f35a00c7fe0 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 10 client.5228.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 10 monclient: _renew_subs 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:57.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a00cb7e0 con 0x7f35a00c7fe0 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 1 librados: init done 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 1 -- 10.20.193.16:0/339446743 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3588004480 con 0x7f35a00c7fe0 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 1 -- 10.20.193.16:0/339446743 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358801c060 con 0x7f35a00c7fe0 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f359d573640 4 set_mon_vals no callback set 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f357f7fe640 10 monclient: dump: 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.009 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.010 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.010 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.010 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.010 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 10 client.5228.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 15 client.5228.objecter _session_command_op_assign -1 1 2026-02-17T06:01:57.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 20 client.5228.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:57.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 10 client.5228.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:57.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:57.010 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:56.994+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35a00704d0 con 0x7f35a00c7fe0 2026-02-17T06:01:57.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.006+0000 7f357f7fe640 1 -- 10.20.193.16:0/339446743 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f3588002980 con 0x7f35a00c7fe0 2026-02-17T06:01:57.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.006+0000 7f357f7fe640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f356c079140 0x7f356c07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.006+0000 7f359ed76640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f356c079140 0x7f356c07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.006+0000 7f359ed76640 10 monclient: get_auth_request con 0x7f356c079140 auth_method 0 2026-02-17T06:01:57.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.006+0000 7f359ed76640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f356c079140 0x7f356c07b600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3594039000 tx=0x7f3594033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.006+0000 7f357f7fe640 10 client.5228.objecter ms_handle_connect 0x7f356c079140 2026-02-17T06:01:57.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 1 -- 10.20.193.16:0/339446743 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f35880977f0 con 0x7f35a00c7fe0 2026-02-17T06:01:57.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 10 client.5228.objecter ms_dispatch2 0x7f35a0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:57.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 3 client.5228.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:57.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 3 client.5228.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:57.013 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 10 client.5228.objecter checking command 1 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] conn(0x7f356c081680 0x7f356c083aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 20 client.5228.objecter _get_session s=0x7f356c080fb0 osd=1 3 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 20 client.5228.objecter put_session s=0x7f356c080fb0 osd=1 3 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 15 client.5228.objecter _session_command_op_remove -1 1 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 20 client.5228.objecter _get_session s=0x7f356c080fb0 osd=1 3 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 20 client.5228.objecter get_session s=0x7f356c080fb0 osd=1 3 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 15 client.5228.objecter _session_command_op_assign 1 1 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 20 client.5228.objecter put_session s=0x7f356c080fb0 osd=1 4 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 10 client.5228.objecter _send_command 1 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 1 -- 10.20.193.16:0/339446743 --> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3588007e50 con 0x7f356c081680 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 20 client.5228.objecter dump_active .. 0 homeless 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 1 -- 10.20.193.16:0/339446743 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f358805e220 con 0x7f35a00c7fe0 2026-02-17T06:01:57.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.008+0000 7f357f7fe640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:57.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.009+0000 7f359f577640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] conn(0x7f356c081680 0x7f356c083aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.009+0000 7f359f577640 10 monclient: get_auth_request con 0x7f356c081680 auth_method 0 2026-02-17T06:01:57.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.011+0000 7f359f577640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] conn(0x7f356c081680 0x7f356c083aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.011+0000 7f357f7fe640 10 client.5228.objecter ms_handle_connect 0x7f356c081680 2026-02-17T06:01:57.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.020+0000 7f357f7fe640 1 -- 10.20.193.16:0/339446743 <== osd.1 v2:10.20.193.128:6807/1618049789 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f3588007e50 con 0x7f356c081680 2026-02-17T06:01:57.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.020+0000 7f357f7fe640 10 client.5228.objecter ms_dispatch2 0x7f35a0000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:57.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.020+0000 7f357f7fe640 10 client.5228.objecter _finish_command 1 = system:0 2026-02-17T06:01:57.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.020+0000 7f357f7fe640 20 client.5228.objecter put_session s=0x7f356c080fb0 osd=1 4 2026-02-17T06:01:57.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.020+0000 7f357f7fe640 15 client.5228.objecter _session_command_op_remove 1 1 2026-02-17T06:01:57.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 10 client.5228.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:57.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 15 client.5228.objecter _session_command_op_assign -1 2 2026-02-17T06:01:57.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 20 client.5228.objecter _get_session s=0x7f356c080fb0 osd=1 3 2026-02-17T06:01:57.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 20 client.5228.objecter put_session s=0x7f356c080fb0 osd=1 3 2026-02-17T06:01:57.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 20 client.5228.objecter _get_session s=0x7f356c080fb0 osd=1 3 2026-02-17T06:01:57.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 15 client.5228.objecter _session_command_op_remove -1 2 2026-02-17T06:01:57.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 20 client.5228.objecter get_session s=0x7f356c080fb0 osd=1 3 2026-02-17T06:01:57.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 15 client.5228.objecter _session_command_op_assign 1 2 2026-02-17T06:01:57.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 20 client.5228.objecter put_session s=0x7f356c080fb0 osd=1 4 2026-02-17T06:01:57.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 10 client.5228.objecter _send_command 2 2026-02-17T06:01:57.028 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.027+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 --> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f35a01467d0 con 0x7f356c081680 2026-02-17T06:01:57.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.033+0000 7f357f7fe640 1 -- 10.20.193.16:0/339446743 <== osd.1 v2:10.20.193.128:6807/1618049789 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f35a01467d0 con 0x7f356c081680 2026-02-17T06:01:57.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.033+0000 7f357f7fe640 10 client.5228.objecter ms_dispatch2 0x7f35a0000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:57.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.033+0000 7f357f7fe640 10 client.5228.objecter _finish_command 2 = system:0 2026-02-17T06:01:57.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.033+0000 7f357f7fe640 20 client.5228.objecter put_session s=0x7f356c080fb0 osd=1 4 2026-02-17T06:01:57.034 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.033+0000 7f357f7fe640 15 client.5228.objecter _session_command_op_remove 1 2 2026-02-17T06:01:57.034 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:57.034 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:57.034 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:57.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 10 librados: watch_flush enter 2026-02-17T06:01:57.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 10 librados: watch_flush exit 2026-02-17T06:01:57.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 10 client.5228.objecter close_session for osd.1 2026-02-17T06:01:57.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 >> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] conn(0x7f356c081680 msgr2=0x7f356c083aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.128:6807/1618049789,v1:10.20.193.128:6809/1618049789] conn(0x7f356c081680 0x7f356c083aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 20 client.5228.objecter put_session s=0x7f356c080fb0 osd=1 1 2026-02-17T06:01:57.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 20 client.5228.objecter shutdown clearing up homeless session... 2026-02-17T06:01:57.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 10 client.5228.objecter successfully canceled tick 2026-02-17T06:01:57.035 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f356c079140 msgr2=0x7f356c07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f356c079140 0x7f356c07b600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3594039000 tx=0x7f3594033000 comp rx=0 tx=0).stop 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 10 monclient: shutdown 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35a00c7fe0 msgr2=0x7f35a0142970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 --2- 10.20.193.16:0/339446743 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f35a00c7fe0 0x7f35a0142970 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f358800b5a0 tx=0x7f358800ba70 comp rx=0 tx=0).stop 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f359f577640 1 -- 10.20.193.16:0/339446743 reap_dead start 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 shutdown_connections 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 >> 10.20.193.16:0/339446743 conn(0x7f35a00cd770 msgr2=0x7f35a0151180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 shutdown_connections 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 -- 10.20.193.16:0/339446743 wait complete. 2026-02-17T06:01:57.036 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.034+0000 7f35a5066640 1 librados: shutdown 2026-02-17T06:01:57.041 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.107+0000 7f2ca6f23640 10 monclient: get_monmap_and_config 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.107+0000 7f2ca6f23640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.107+0000 7f2ca6f23640 10 monclient: monmap: 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:57.108655+0000 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:57.108655+0000 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:57.108 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 1 Processor -- start 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 1 -- start start 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 10 monclient: init 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 10 monclient: _add_conns ranks=[5,0,4,2,3,1] 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f2ca00700f0 0x7f2ca00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 10 monclient(hunting): picked mon.noname-c con 0x7f2ca00700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2ca00c7fe0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f2ca01536f0 0x7f2ca0153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 10 monclient(hunting): picked mon.noname-b con 0x7f2ca01536f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca00c4820 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca00c4b00 con 0x7f2ca01536f0 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca00c4de0 con 0x7f2ca00700f0 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.108+0000 7f2ca6f23640 10 monclient(hunting): authenticate will time out at 1369.677636s 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca4c98640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f2ca00700f0 0x7f2ca00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2c97fff640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f2ca00c7fe0 0x7f2ca00c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:41882/0 (socket says 10.20.193.16:41882) 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2c97fff640 1 -- 10.20.193.16:0/2328736647 learned_addr learned my addr 10.20.193.16:0/2328736647 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca4c98640 10 monclient(hunting): get_auth_request con 0x7f2ca00700f0 auth_method 0 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca4c98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca4c98640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca4c98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca5499640 1 --2- 10.20.193.16:0/2328736647 >> v2:10.20.193.128:3300/0 conn(0x7f2ca01536f0 0x7f2ca0153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca5499640 10 monclient(hunting): get_auth_request con 0x7f2ca01536f0 auth_method 0 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca5499640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca5499640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca5499640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca4c98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca4c98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca4c98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca5499640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca5499640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.109+0000 7f2ca5499640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.110+0000 7f2ca4c98640 10 monclient(hunting): handle_auth_done global_id 5231 payload 274 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.110+0000 7f2ca4c98640 1 -- 10.20.193.16:0/2328736647 >> v2:10.20.193.128:3300/0 conn(0x7f2ca01536f0 msgr2=0x7f2ca0153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.110+0000 7f2ca4c98640 1 --2- 10.20.193.16:0/2328736647 >> v2:10.20.193.128:3300/0 conn(0x7f2ca01536f0 0x7f2ca0153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.110+0000 7f2ca4c98640 1 -- 10.20.193.16:0/2328736647 >> v1:10.20.193.16:6789/0 conn(0x7f2ca00c7fe0 legacy=0x7f2ca00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.110+0000 7f2ca4c98640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.110+0000 7f2ca4c98640 1 monclient: found mon.noname-c 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.110+0000 7f2ca4c98640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.110+0000 7f2ca4c98640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.110+0000 7f2ca4c98640 1 -- 10.20.193.16:0/2328736647 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca00c50c0 con 0x7f2ca00700f0 2026-02-17T06:01:57.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.111+0000 7f2ca4c98640 1 --2- 10.20.193.16:0/2328736647 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ca00700f0 0x7f2ca00704d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0070b90 tx=0x7f2c9002ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=893361ea73f7a590 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 1 -- 10.20.193.16:0/2328736647 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9003c070 con 0x7f2ca00700f0 2026-02-17T06:01:57.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:57.113 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 10 monclient: dump: 2026-02-17T06:01:57.113 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 1 -- 10.20.193.16:0/2328736647 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9002fa60 con 0x7f2ca00700f0 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 1 -- 10.20.193.16:0/2328736647 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9002fe30 con 0x7f2ca00700f0 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2c977fe640 10 monclient: dump: 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.114 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 5 monclient: authenticate success, global_id 5231 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 4 set_mon_vals no callback set 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 10 monclient: shutdown 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 1 -- 10.20.193.16:0/2328736647 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ca00700f0 msgr2=0x7f2ca00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 1 --2- 10.20.193.16:0/2328736647 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ca00700f0 0x7f2ca00704d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0070b90 tx=0x7f2c9002ed50 comp rx=0 tx=0).stop 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 1 -- 10.20.193.16:0/2328736647 shutdown_connections 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 1 --2- 10.20.193.16:0/2328736647 >> v2:10.20.193.128:3300/0 conn(0x7f2ca01536f0 0x7f2ca0153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 1 --2- 10.20.193.16:0/2328736647 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ca00700f0 0x7f2ca00704d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 1 -- 10.20.193.16:0/2328736647 >> 10.20.193.16:0/2328736647 conn(0x7f2ca00cd770 msgr2=0x7f2ca00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 1 -- 10.20.193.16:0/2328736647 shutdown_connections 2026-02-17T06:01:57.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.113+0000 7f2ca6f23640 1 -- 10.20.193.16:0/2328736647 wait complete. 2026-02-17T06:01:57.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient: monmap: 2026-02-17T06:01:57.116 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.116 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:57.116 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:57.116 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 librados: starting msgr at 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 librados: starting objecter 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 Processor -- start 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 -- start start 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 librados: setting wanted keys 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 librados: calling monclient init 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient: init 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2ca00700f0 0x7f2ca0139b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient(hunting): picked mon.noname-b con 0x7f2ca00700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ca00c7fe0 0x7f2ca00cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient(hunting): picked mon.noname-a con 0x7f2ca00c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ca01536f0 0x7f2ca00cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient(hunting): picked mon.noname-c con 0x7f2ca01536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0154060 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca00c5680 con 0x7f2ca00700f0 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0074c00 con 0x7f2ca01536f0 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.117 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.114+0000 7f2ca6f23640 10 monclient(hunting): authenticate will time out at 1369.683856s 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2c97fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ca00c7fe0 0x7f2ca00cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca4c98640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2ca00700f0 0x7f2ca0139b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2c97fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ca00c7fe0 0x7f2ca00cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33800/0 (socket says 10.20.193.16:33800) 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2c97fff640 1 -- 10.20.193.16:0/1621340575 learned_addr learned my addr 10.20.193.16:0/1621340575 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2c97fff640 10 monclient(hunting): get_auth_request con 0x7f2ca00c7fe0 auth_method 0 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2c97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2c97fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2c97fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca4c98640 10 monclient(hunting): get_auth_request con 0x7f2ca00700f0 auth_method 0 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca4c98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca4c98640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca4c98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca5499640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ca01536f0 0x7f2ca00cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2c97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2c97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2c97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca4c98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca4c98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca4c98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca5499640 10 monclient(hunting): get_auth_request con 0x7f2ca01536f0 auth_method 0 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca5499640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca5499640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca5499640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca5499640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca5499640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.118 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.115+0000 7f2ca5499640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.116+0000 7f2c97fff640 10 monclient(hunting): handle_auth_done global_id 5097 payload 824 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.116+0000 7f2c97fff640 1 -- 10.20.193.16:0/1621340575 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ca01536f0 msgr2=0x7f2ca00cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.116+0000 7f2c97fff640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ca01536f0 0x7f2ca00cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.117+0000 7f2c97fff640 1 -- 10.20.193.16:0/1621340575 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2ca00700f0 msgr2=0x7f2ca0139b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.117+0000 7f2c97fff640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2ca00700f0 0x7f2ca0139b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.117+0000 7f2c97fff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.117+0000 7f2c97fff640 1 monclient: found mon.noname-a 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.117+0000 7f2c97fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.117+0000 7f2c97fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.117+0000 7f2c97fff640 1 -- 10.20.193.16:0/1621340575 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca006e4a0 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.118+0000 7f2c97fff640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ca00c7fe0 0x7f2ca00cb4f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c500 tx=0x7f2c8800c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.118+0000 7f2c957fa640 10 client.?.objecter ms_handle_connect 0x7f2ca00c7fe0 2026-02-17T06:01:57.119 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.118+0000 7f2c957fa640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 1 -- 10.20.193.16:0/1621340575 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88017070 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 10 monclient: dump: 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.120 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 1 -- 10.20.193.16:0/1621340575 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c88005e00 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2ca6f23640 5 monclient: authenticate success, global_id 5097 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 1 -- 10.20.193.16:0/1621340575 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c880072b0 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c96ffd640 4 set_mon_vals no callback set 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2c957fa640 10 monclient: dump: 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2ca6f23640 10 monclient: _renew_subs 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2ca6f23640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2ca6f23640 1 -- 10.20.193.16:0/1621340575 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca006e780 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2ca6f23640 10 client.5097.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2ca6f23640 10 monclient: _renew_subs 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2ca6f23640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2ca6f23640 1 -- 10.20.193.16:0/1621340575 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca006ed40 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.121 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.119+0000 7f2ca6f23640 1 librados: init done 2026-02-17T06:01:57.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.120+0000 7f2ca6f23640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.120+0000 7f2ca6f23640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.120+0000 7f2ca6f23640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:57.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.120+0000 7f2ca6f23640 1 -- 10.20.193.16:0/1621340575 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca00704d0 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.131+0000 7f2c957fa640 1 -- 10.20.193.16:0/1621340575 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f2c88007510 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.132+0000 7f2c957fa640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c70079140 0x7f2c7007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.132+0000 7f2ca4c98640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c70079140 0x7f2c7007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.132+0000 7f2ca4c98640 10 monclient: get_auth_request con 0x7f2c70079140 auth_method 0 2026-02-17T06:01:57.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.132+0000 7f2ca4c98640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c70079140 0x7f2c7007b600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2c90039000 tx=0x7f2c90033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.132+0000 7f2c957fa640 10 client.5097.objecter ms_handle_connect 0x7f2c70079140 2026-02-17T06:01:57.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.133+0000 7f2c957fa640 1 -- 10.20.193.16:0/1621340575 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f2c8805d680 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.133+0000 7f2c957fa640 10 client.5097.objecter ms_dispatch2 0x7f2ca0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:57.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.133+0000 7f2c957fa640 3 client.5097.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:57.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.133+0000 7f2c957fa640 3 client.5097.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:57.134 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.133+0000 7f2c957fa640 20 client.5097.objecter dump_active .. 0 homeless 2026-02-17T06:01:57.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.234+0000 7f2c957fa640 1 -- 10.20.193.16:0/1621340575 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2c88021020 con 0x7f2ca00c7fe0 2026-02-17T06:01:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.234+0000 7f2c957fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.234+0000 7f2c957fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:57.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.317+0000 7f2ca6f23640 1 -- 10.20.193.16:0/1621340575 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f2ca0146840 con 0x7f2c70079140 2026-02-17T06:01:57.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.317+0000 7f2c957fa640 1 -- 10.20.193.16:0/1621340575 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2ca0146840 con 0x7f2c70079140 2026-02-17T06:01:57.318 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 1 to deep-scrub 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 10 librados: watch_flush enter 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 10 librados: watch_flush exit 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 20 client.5097.objecter shutdown clearing up homeless session... 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 10 client.5097.objecter successfully canceled tick 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 -- 10.20.193.16:0/1621340575 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c70079140 msgr2=0x7f2c7007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c70079140 0x7f2c7007b600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2c90039000 tx=0x7f2c90033000 comp rx=0 tx=0).stop 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 10 monclient: shutdown 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 -- 10.20.193.16:0/1621340575 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ca00c7fe0 msgr2=0x7f2ca00cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ca00c7fe0 0x7f2ca00cb4f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c500 tx=0x7f2c8800c9d0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 -- 10.20.193.16:0/1621340575 shutdown_connections 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2c70079140 0x7f2c7007b600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ca01536f0 0x7f2ca00cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ca00c7fe0 0x7f2ca00cb4f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 --2- 10.20.193.16:0/1621340575 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2ca00700f0 0x7f2ca0139b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 -- 10.20.193.16:0/1621340575 >> 10.20.193.16:0/1621340575 conn(0x7f2ca00cd770 msgr2=0x7f2ca014e320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 -- 10.20.193.16:0/1621340575 shutdown_connections 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 -- 10.20.193.16:0/1621340575 wait complete. 2026-02-17T06:01:57.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.318+0000 7f2ca6f23640 1 librados: shutdown 2026-02-17T06:01:57.327 INFO:tasks.ceph:Scrubbing osd.5 2026-02-17T06:01:57.327 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.396+0000 7fe596f33640 10 monclient: get_monmap_and_config 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.396+0000 7fe596f33640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.396+0000 7fe596f33640 10 monclient: monmap: 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:57.397796+0000 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:57.397796+0000 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:57.397 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.397+0000 7fe596f33640 1 Processor -- start 2026-02-17T06:01:57.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.397+0000 7fe596f33640 1 -- start start 2026-02-17T06:01:57.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.397+0000 7fe596f33640 10 monclient: init 2026-02-17T06:01:57.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 10 monclient: _add_conns ranks=[1,2,4,0,5,3] 2026-02-17T06:01:57.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fe590152550 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:57.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe59015d930 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:57.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fe59006e0a0 0x7fe5900cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.398 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 10 monclient(hunting): picked mon.noname-b con 0x7fe59006e0a0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5900cca70 con 0x7fe590152550 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5900ccd50 con 0x7fe59015d930 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fe5900cc850 con 0x7fe59006e0a0 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe596f33640 10 monclient(hunting): authenticate will time out at 1369.967199s 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe587fff640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7fe59015d930 0x7fe59006d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:51314/0 (socket says 10.20.193.16:51314) 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe587fff640 1 -- 10.20.193.16:0/1715417092 learned_addr learned my addr 10.20.193.16:0/1715417092 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe5954a9640 1 --2- 10.20.193.16:0/1715417092 >> v2:10.20.193.128:3300/0 conn(0x7fe59006e0a0 0x7fe5900cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe5954a9640 10 monclient(hunting): get_auth_request con 0x7fe59006e0a0 auth_method 0 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe5954a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe5954a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.398+0000 7fe5954a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.399+0000 7fe5954a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.399+0000 7fe5954a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.399 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.399+0000 7fe5954a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.399+0000 7fe5877fe640 1 -- 10.20.193.16:0/1715417092 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1866959224 0 0) 0x7fe5900ccd50 con 0x7fe59015d930 2026-02-17T06:01:57.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.399+0000 7fe5877fe640 10 monclient(hunting): _negotiate 2026-02-17T06:01:57.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.399+0000 7fe5877fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.399+0000 7fe5877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.399+0000 7fe5877fe640 10 monclient(hunting): my global_id is 5237 2026-02-17T06:01:57.400 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.399+0000 7fe5877fe640 1 -- 10.20.193.16:0/1715417092 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe57c003620 con 0x7fe59015d930 2026-02-17T06:01:57.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.401+0000 7fe5954a9640 10 monclient(hunting): handle_auth_done global_id 5179 payload 274 2026-02-17T06:01:57.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.401+0000 7fe5954a9640 1 -- 10.20.193.16:0/1715417092 >> v1:10.20.193.148:6789/0 conn(0x7fe59015d930 legacy=0x7fe59006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.401+0000 7fe5954a9640 1 -- 10.20.193.16:0/1715417092 >> v1:10.20.193.128:6789/0 conn(0x7fe590152550 legacy=0x7fe59015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.401+0000 7fe5954a9640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.401+0000 7fe5954a9640 1 monclient: found mon.noname-b 2026-02-17T06:01:57.401 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.401+0000 7fe5954a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.401+0000 7fe5954a9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:57.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.401+0000 7fe5954a9640 1 -- 10.20.193.16:0/1715417092 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5900cd030 con 0x7fe59006e0a0 2026-02-17T06:01:57.402 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.402+0000 7fe5954a9640 1 --2- 10.20.193.16:0/1715417092 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe59006e0a0 0x7fe5900cc310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009d90 tx=0x7fe58c02f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb29f070702cd560 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 1 -- 10.20.193.16:0/1715417092 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c03c070 con 0x7fe59006e0a0 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 10 monclient: dump: 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.405 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 1 -- 10.20.193.16:0/1715417092 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58c037440 con 0x7fe59006e0a0 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe596f33640 5 monclient: authenticate success, global_id 5179 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe596f33640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 1 -- 10.20.193.16:0/1715417092 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c037810 con 0x7fe59006e0a0 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe596f33640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe596f33640 4 set_mon_vals no callback set 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.404+0000 7fe5877fe640 10 monclient: dump: 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 10 monclient: shutdown 2026-02-17T06:01:57.406 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 1 -- 10.20.193.16:0/1715417092 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe59006e0a0 msgr2=0x7fe5900cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 1 --2- 10.20.193.16:0/1715417092 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe59006e0a0 0x7fe5900cc310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009d90 tx=0x7fe58c02f230 comp rx=0 tx=0).stop 2026-02-17T06:01:57.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 1 -- 10.20.193.16:0/1715417092 shutdown_connections 2026-02-17T06:01:57.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 1 --2- 10.20.193.16:0/1715417092 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe59006e0a0 0x7fe5900cc310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 1 -- 10.20.193.16:0/1715417092 >> 10.20.193.16:0/1715417092 conn(0x7fe5900d1c50 msgr2=0x7fe5900d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 1 -- 10.20.193.16:0/1715417092 shutdown_connections 2026-02-17T06:01:57.407 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 1 -- 10.20.193.16:0/1715417092 wait complete. 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 10 monclient: monmap: 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.408 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 1 librados: starting msgr at 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.405+0000 7fe596f33640 1 librados: starting objecter 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 1 Processor -- start 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 1 -- start start 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 1 librados: setting wanted keys 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 1 librados: calling monclient init 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient: init 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe59006e0a0 0x7fe5901f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient(hunting): picked mon.noname-a con 0x7fe59006e0a0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe590152550 0x7fe59015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient(hunting): picked mon.noname-b con 0x7fe590152550 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe59015fa00 0x7fe5901f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient(hunting): picked mon.noname-c con 0x7fe59015fa00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5900cd5f0 con 0x7fe59006e0a0 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fe5900cd960 con 0x7fe590152550 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe590152930 con 0x7fe59015fa00 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe596f33640 10 monclient(hunting): authenticate will time out at 1369.975501s 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe594ca8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe59006e0a0 0x7fe5901f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe587fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe590152550 0x7fe59015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe587fff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe590152550 0x7fe59015f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43454/0 (socket says 10.20.193.16:43454) 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe587fff640 1 -- 10.20.193.16:0/451593701 learned_addr learned my addr 10.20.193.16:0/451593701 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe587fff640 10 monclient(hunting): get_auth_request con 0x7fe590152550 auth_method 0 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe587fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.409 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe587fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe587fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe594ca8640 10 monclient(hunting): get_auth_request con 0x7fe59006e0a0 auth_method 0 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe594ca8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe594ca8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe594ca8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.406+0000 7fe5954a9640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe59015fa00 0x7fe5901f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe587fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe587fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe587fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe5954a9640 10 monclient(hunting): get_auth_request con 0x7fe59015fa00 auth_method 0 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe5954a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe5954a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe5954a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe594ca8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe594ca8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe594ca8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe5954a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe5954a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.410 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.407+0000 7fe5954a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.408+0000 7fe587fff640 10 monclient(hunting): handle_auth_done global_id 5185 payload 824 2026-02-17T06:01:57.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.408+0000 7fe587fff640 1 -- 10.20.193.16:0/451593701 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe59015fa00 msgr2=0x7fe5901f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.408+0000 7fe587fff640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fe59015fa00 0x7fe5901f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.408+0000 7fe587fff640 1 -- 10.20.193.16:0/451593701 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe59006e0a0 msgr2=0x7fe5901f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.408+0000 7fe587fff640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fe59006e0a0 0x7fe5901f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.408+0000 7fe587fff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.408+0000 7fe587fff640 1 monclient: found mon.noname-b 2026-02-17T06:01:57.411 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.408+0000 7fe587fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.408+0000 7fe587fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.408+0000 7fe587fff640 1 -- 10.20.193.16:0/451593701 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5901566c0 con 0x7fe590152550 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.409+0000 7fe587fff640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe590152550 0x7fe59015f4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe57800c500 tx=0x7fe57800c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.409+0000 7fe5857fa640 10 client.?.objecter ms_handle_connect 0x7fe590152550 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.409+0000 7fe5857fa640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe5857fa640 1 -- 10.20.193.16:0/451593701 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578017070 con 0x7fe590152550 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe5857fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe5857fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe5857fa640 10 monclient: dump: 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe5857fa640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe5857fa640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe5857fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe596f33640 5 monclient: authenticate success, global_id 5185 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe596f33640 10 monclient: _renew_subs 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe596f33640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5901f45f0 con 0x7fe590152550 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe596f33640 10 client.5185.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:57.412 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe596f33640 10 monclient: _renew_subs 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe596f33640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe590156c10 con 0x7fe590152550 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.410+0000 7fe596f33640 1 librados: init done 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe5857fa640 1 -- 10.20.193.16:0/451593701 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe578005de0 con 0x7fe590152550 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe5857fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe586ffd640 4 set_mon_vals no callback set 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe5857fa640 1 -- 10.20.193.16:0/451593701 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578021060 con 0x7fe590152550 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe5857fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe5857fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe5857fa640 10 monclient: dump: 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe596f33640 10 client.5185.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe596f33640 15 client.5185.objecter _session_command_op_assign -1 1 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe596f33640 20 client.5185.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe596f33640 10 client.5185.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe596f33640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:57.413 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.411+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe5900cc310 con 0x7fe590152550 2026-02-17T06:01:57.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.423+0000 7fe5857fa640 1 -- 10.20.193.16:0/451593701 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fe578007270 con 0x7fe590152550 2026-02-17T06:01:57.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.423+0000 7fe5857fa640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe560079140 0x7fe56007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.423 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.423+0000 7fe594ca8640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe560079140 0x7fe56007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.423+0000 7fe594ca8640 10 monclient: get_auth_request con 0x7fe560079140 auth_method 0 2026-02-17T06:01:57.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.423+0000 7fe594ca8640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe560079140 0x7fe56007b600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe580002770 tx=0x7fe580007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.424 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.423+0000 7fe5857fa640 10 client.5185.objecter ms_handle_connect 0x7fe560079140 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 1 -- 10.20.193.16:0/451593701 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fe578097850 con 0x7fe590152550 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 10 client.5185.objecter ms_dispatch2 0x7fe590000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 3 client.5185.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 3 client.5185.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 10 client.5185.objecter checking command 1 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] conn(0x7fe560081620 0x7fe560083a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 20 client.5185.objecter _get_session s=0x7fe560080f50 osd=5 3 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 20 client.5185.objecter put_session s=0x7fe560080f50 osd=5 3 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 15 client.5185.objecter _session_command_op_remove -1 1 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 20 client.5185.objecter _get_session s=0x7fe560080f50 osd=5 3 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 20 client.5185.objecter get_session s=0x7fe560080f50 osd=5 3 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 15 client.5185.objecter _session_command_op_assign 5 1 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 20 client.5185.objecter put_session s=0x7fe560080f50 osd=5 4 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 10 client.5185.objecter _send_command 1 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 1 -- 10.20.193.16:0/451593701 --> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe57801fe70 con 0x7fe560081620 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 20 client.5185.objecter dump_active .. 0 homeless 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 1 -- 10.20.193.16:0/451593701 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe57805e0f0 con 0x7fe590152550 2026-02-17T06:01:57.425 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.424+0000 7fe5857fa640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:57.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.425+0000 7fe5954a9640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] conn(0x7fe560081620 0x7fe560083a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.426 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.425+0000 7fe5954a9640 10 monclient: get_auth_request con 0x7fe560081620 auth_method 0 2026-02-17T06:01:57.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.427+0000 7fe5954a9640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] conn(0x7fe560081620 0x7fe560083a40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.427 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.427+0000 7fe5857fa640 10 client.5185.objecter ms_handle_connect 0x7fe560081620 2026-02-17T06:01:57.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.438+0000 7fe5857fa640 1 -- 10.20.193.16:0/451593701 <== osd.5 v2:10.20.193.128:6824/1593443319 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fe57801fe70 con 0x7fe560081620 2026-02-17T06:01:57.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.438+0000 7fe5857fa640 10 client.5185.objecter ms_dispatch2 0x7fe590000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:57.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.438+0000 7fe5857fa640 10 client.5185.objecter _finish_command 1 = system:0 2026-02-17T06:01:57.437 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.438+0000 7fe5857fa640 20 client.5185.objecter put_session s=0x7fe560080f50 osd=5 4 2026-02-17T06:01:57.438 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.438+0000 7fe5857fa640 15 client.5185.objecter _session_command_op_remove 5 1 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 10 client.5185.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 15 client.5185.objecter _session_command_op_assign -1 2 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 20 client.5185.objecter _get_session s=0x7fe560080f50 osd=5 3 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 20 client.5185.objecter put_session s=0x7fe560080f50 osd=5 3 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 20 client.5185.objecter _get_session s=0x7fe560080f50 osd=5 3 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 15 client.5185.objecter _session_command_op_remove -1 2 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 20 client.5185.objecter get_session s=0x7fe560080f50 osd=5 3 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 15 client.5185.objecter _session_command_op_assign 5 2 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 20 client.5185.objecter put_session s=0x7fe560080f50 osd=5 4 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 10 client.5185.objecter _send_command 2 2026-02-17T06:01:57.444 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.444+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 --> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe59014e8f0 con 0x7fe560081620 2026-02-17T06:01:57.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe5857fa640 1 -- 10.20.193.16:0/451593701 <== osd.5 v2:10.20.193.128:6824/1593443319 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe59014e8f0 con 0x7fe560081620 2026-02-17T06:01:57.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe5857fa640 10 client.5185.objecter ms_dispatch2 0x7fe590000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:57.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe5857fa640 10 client.5185.objecter _finish_command 2 = system:0 2026-02-17T06:01:57.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe5857fa640 20 client.5185.objecter put_session s=0x7fe560080f50 osd=5 4 2026-02-17T06:01:57.451 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe5857fa640 15 client.5185.objecter _session_command_op_remove 5 2 2026-02-17T06:01:57.451 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:57.451 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:57.451 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 10 librados: watch_flush enter 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 10 librados: watch_flush exit 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 10 client.5185.objecter close_session for osd.5 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 >> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] conn(0x7fe560081620 msgr2=0x7fe560083a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.128:6824/1593443319,v1:10.20.193.128:6825/1593443319] conn(0x7fe560081620 0x7fe560083a40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 20 client.5185.objecter put_session s=0x7fe560080f50 osd=5 1 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 20 client.5185.objecter shutdown clearing up homeless session... 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 10 client.5185.objecter successfully canceled tick 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe560079140 msgr2=0x7fe56007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fe560079140 0x7fe56007b600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe580002770 tx=0x7fe580007ef0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 10 monclient: shutdown 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe590152550 msgr2=0x7fe59015f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 --2- 10.20.193.16:0/451593701 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fe590152550 0x7fe59015f4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe57800c500 tx=0x7fe57800c9d0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe5954a9640 1 -- 10.20.193.16:0/451593701 reap_dead start 2026-02-17T06:01:57.452 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 shutdown_connections 2026-02-17T06:01:57.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 >> 10.20.193.16:0/451593701 conn(0x7fe5900d1c50 msgr2=0x7fe59014ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 shutdown_connections 2026-02-17T06:01:57.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 -- 10.20.193.16:0/451593701 wait complete. 2026-02-17T06:01:57.453 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.451+0000 7fe596f33640 1 librados: shutdown 2026-02-17T06:01:57.458 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.527+0000 7f30e9961640 10 monclient: get_monmap_and_config 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.527+0000 7f30e9961640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.527+0000 7f30e9961640 10 monclient: monmap: 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:57.528877+0000 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:57.528877+0000 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:57.528 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 1 Processor -- start 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 1 -- start start 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 10 monclient: init 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 10 monclient: _add_conns ranks=[5,1,3,2,0,4] 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f30e40700f0 0x7f30e40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 10 monclient(hunting): picked mon.noname-c con 0x7f30e40700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f30e40c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f30e41536f0 0x7f30e4153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 10 monclient(hunting): picked mon.noname-a con 0x7f30e41536f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30e40c4820 con 0x7f30e40c7fe0 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f30e40c4b00 con 0x7f30e41536f0 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f30e40c4de0 con 0x7f30e40700f0 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.528+0000 7f30e9961640 10 monclient(hunting): authenticate will time out at 1370.097914s 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e37fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f30e41536f0 0x7f30e4153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e27fc640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f30e40c7fe0 0x7f30e40c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:51288/0 (socket says 10.20.193.16:51288) 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e27fc640 1 -- 10.20.193.16:0/3767061522 learned_addr learned my addr 10.20.193.16:0/3767061522 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e2ffd640 1 --2- 10.20.193.16:0/3767061522 >> v2:10.20.193.148:3300/0 conn(0x7f30e40700f0 0x7f30e40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e37fe640 10 monclient(hunting): get_auth_request con 0x7f30e41536f0 auth_method 0 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e2ffd640 10 monclient(hunting): get_auth_request con 0x7f30e40700f0 auth_method 0 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.529+0000 7f30e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.530+0000 7f30e37fe640 10 monclient(hunting): handle_auth_done global_id 5103 payload 274 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.530+0000 7f30e37fe640 1 -- 10.20.193.16:0/3767061522 >> v2:10.20.193.148:3300/0 conn(0x7f30e40700f0 msgr2=0x7f30e40704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.530+0000 7f30e37fe640 1 --2- 10.20.193.16:0/3767061522 >> v2:10.20.193.148:3300/0 conn(0x7f30e40700f0 0x7f30e40704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.530+0000 7f30e37fe640 1 -- 10.20.193.16:0/3767061522 >> v1:10.20.193.128:6789/0 conn(0x7f30e40c7fe0 legacy=0x7f30e40c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.530+0000 7f30e37fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.530+0000 7f30e37fe640 1 monclient: found mon.noname-a 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.530+0000 7f30e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.530+0000 7f30e37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.530+0000 7f30e37fe640 1 -- 10.20.193.16:0/3767061522 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e40c50c0 con 0x7f30e41536f0 2026-02-17T06:01:57.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.531+0000 7f30e37fe640 1 --2- 10.20.193.16:0/3767061522 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e41536f0 0x7f30e4153b20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f30e4070b90 tx=0x7f30d402f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d129cac4d9fd1e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 1 -- 10.20.193.16:0/3767061522 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d403c070 con 0x7f30e41536f0 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 10 monclient: dump: 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 1 -- 10.20.193.16:0/3767061522 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d402fe40 con 0x7f30e41536f0 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 5 monclient: authenticate success, global_id 5103 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 1 -- 10.20.193.16:0/3767061522 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4037660 con 0x7f30e41536f0 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e1ffb640 10 monclient: dump: 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.534 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 4 set_mon_vals no callback set 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 10 monclient: shutdown 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 1 -- 10.20.193.16:0/3767061522 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e41536f0 msgr2=0x7f30e4153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 1 --2- 10.20.193.16:0/3767061522 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e41536f0 0x7f30e4153b20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f30e4070b90 tx=0x7f30d402f130 comp rx=0 tx=0).stop 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 1 -- 10.20.193.16:0/3767061522 shutdown_connections 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 1 --2- 10.20.193.16:0/3767061522 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e41536f0 0x7f30e4153b20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 1 --2- 10.20.193.16:0/3767061522 >> v2:10.20.193.148:3300/0 conn(0x7f30e40700f0 0x7f30e40704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 1 -- 10.20.193.16:0/3767061522 >> 10.20.193.16:0/3767061522 conn(0x7f30e40cd770 msgr2=0x7f30e40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 1 -- 10.20.193.16:0/3767061522 shutdown_connections 2026-02-17T06:01:57.535 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.533+0000 7f30e9961640 1 -- 10.20.193.16:0/3767061522 wait complete. 2026-02-17T06:01:57.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.536 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient: monmap: 2026-02-17T06:01:57.536 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 librados: starting msgr at 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 librados: starting objecter 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 Processor -- start 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 -- start start 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 librados: setting wanted keys 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 librados: calling monclient init 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient: init 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30e40700f0 0x7f30e41f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient(hunting): picked mon.noname-c con 0x7f30e40700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e40c7fe0 0x7f30e4145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient(hunting): picked mon.noname-a con 0x7f30e40c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e41536f0 0x7f30e414b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient(hunting): picked mon.noname-b con 0x7f30e41536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4154060 con 0x7f30e40c7fe0 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f30e40c5680 con 0x7f30e41536f0 2026-02-17T06:01:57.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4074c00 con 0x7f30e40700f0 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.534+0000 7f30e9961640 10 monclient(hunting): authenticate will time out at 1370.103944s 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e27fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e40c7fe0 0x7f30e4145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e37fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e41536f0 0x7f30e414b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e27fc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e40c7fe0 0x7f30e4145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33822/0 (socket says 10.20.193.16:33822) 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e27fc640 1 -- 10.20.193.16:0/2178578516 learned_addr learned my addr 10.20.193.16:0/2178578516 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e27fc640 10 monclient(hunting): get_auth_request con 0x7f30e40c7fe0 auth_method 0 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e37fe640 10 monclient(hunting): get_auth_request con 0x7f30e41536f0 auth_method 0 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e2ffd640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30e40700f0 0x7f30e41f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e2ffd640 10 monclient(hunting): get_auth_request con 0x7f30e40700f0 auth_method 0 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.538 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.535+0000 7f30e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.536+0000 7f30e27fc640 10 monclient(hunting): handle_auth_done global_id 5106 payload 824 2026-02-17T06:01:57.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.536+0000 7f30e27fc640 1 -- 10.20.193.16:0/2178578516 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30e40700f0 msgr2=0x7f30e41f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.539 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.536+0000 7f30e27fc640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30e40700f0 0x7f30e41f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.536+0000 7f30e27fc640 1 -- 10.20.193.16:0/2178578516 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e41536f0 msgr2=0x7f30e414b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.536+0000 7f30e27fc640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e41536f0 0x7f30e414b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.536+0000 7f30e27fc640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.536+0000 7f30e27fc640 1 monclient: found mon.noname-a 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.536+0000 7f30e27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.536+0000 7f30e27fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.536+0000 7f30e27fc640 1 -- 10.20.193.16:0/2178578516 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e414ec60 con 0x7f30e40c7fe0 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.537+0000 7f30e27fc640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e40c7fe0 0x7f30e4145d10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00c4d0 tx=0x7f30cc00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.537+0000 7f30bbfff640 10 client.?.objecter ms_handle_connect 0x7f30e40c7fe0 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.537+0000 7f30bbfff640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 1 -- 10.20.193.16:0/2178578516 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc017070 con 0x7f30e40c7fe0 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 10 monclient: dump: 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 5 monclient: authenticate success, global_id 5106 2026-02-17T06:01:57.540 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 1 -- 10.20.193.16:0/2178578516 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30cc005de0 con 0x7f30e40c7fe0 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 10 monclient: _renew_subs 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 1 -- 10.20.193.16:0/2178578516 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc007290 con 0x7f30e40c7fe0 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 1 -- 10.20.193.16:0/2178578516 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e4154f20 con 0x7f30e40c7fe0 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e17fa640 4 set_mon_vals no callback set 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30bbfff640 10 monclient: dump: 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 10 client.5106.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 10 monclient: _renew_subs 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 1 -- 10.20.193.16:0/2178578516 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e41554e0 con 0x7f30e40c7fe0 2026-02-17T06:01:57.541 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 1 librados: init done 2026-02-17T06:01:57.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:57.542 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.539+0000 7f30e9961640 1 -- 10.20.193.16:0/2178578516 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e40704d0 con 0x7f30e40c7fe0 2026-02-17T06:01:57.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.552+0000 7f30bbfff640 1 -- 10.20.193.16:0/2178578516 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f30cc007430 con 0x7f30e40c7fe0 2026-02-17T06:01:57.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.552+0000 7f30bbfff640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b40791c0 0x7f30b407b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.552+0000 7f30e2ffd640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b40791c0 0x7f30b407b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.552+0000 7f30e2ffd640 10 monclient: get_auth_request con 0x7f30b40791c0 auth_method 0 2026-02-17T06:01:57.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.552+0000 7f30bbfff640 10 client.5106.objecter ms_handle_connect 0x7f30b40791c0 2026-02-17T06:01:57.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.552+0000 7f30e2ffd640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b40791c0 0x7f30b407b680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009ba0 tx=0x7f30d8009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.553+0000 7f30bbfff640 1 -- 10.20.193.16:0/2178578516 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f30cc007750 con 0x7f30e40c7fe0 2026-02-17T06:01:57.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.553+0000 7f30bbfff640 10 client.5106.objecter ms_dispatch2 0x7f30e4000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:57.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.553+0000 7f30bbfff640 3 client.5106.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:57.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.553+0000 7f30bbfff640 3 client.5106.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:57.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.554+0000 7f30bbfff640 20 client.5106.objecter dump_active .. 0 homeless 2026-02-17T06:01:57.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.661+0000 7f30bbfff640 1 -- 10.20.193.16:0/2178578516 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f30cc021020 con 0x7f30e40c7fe0 2026-02-17T06:01:57.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.661+0000 7f30bbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.661 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.661+0000 7f30bbfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:57.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.742+0000 7f30e9961640 1 -- 10.20.193.16:0/2178578516 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f30e40c5890 con 0x7f30b40791c0 2026-02-17T06:01:57.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.743+0000 7f30bbfff640 1 -- 10.20.193.16:0/2178578516 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f30e40c5890 con 0x7f30b40791c0 2026-02-17T06:01:57.744 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 5 to deep-scrub 2026-02-17T06:01:57.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.743+0000 7f30e9961640 10 librados: watch_flush enter 2026-02-17T06:01:57.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.743+0000 7f30e9961640 10 librados: watch_flush exit 2026-02-17T06:01:57.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.743+0000 7f30e9961640 20 client.5106.objecter shutdown clearing up homeless session... 2026-02-17T06:01:57.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.743+0000 7f30e9961640 10 client.5106.objecter successfully canceled tick 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 -- 10.20.193.16:0/2178578516 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b40791c0 msgr2=0x7f30b407b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b40791c0 0x7f30b407b680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009ba0 tx=0x7f30d8009290 comp rx=0 tx=0).stop 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 10 monclient: shutdown 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 -- 10.20.193.16:0/2178578516 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e40c7fe0 msgr2=0x7f30e4145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e40c7fe0 0x7f30e4145d10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00c4d0 tx=0x7f30cc00c9a0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 -- 10.20.193.16:0/2178578516 shutdown_connections 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f30b40791c0 0x7f30b407b680 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f30e41536f0 0x7f30e414b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f30e40c7fe0 0x7f30e4145d10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.746 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 --2- 10.20.193.16:0/2178578516 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f30e40700f0 0x7f30e41f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 -- 10.20.193.16:0/2178578516 >> 10.20.193.16:0/2178578516 conn(0x7f30e40cd770 msgr2=0x7f30e4152970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 -- 10.20.193.16:0/2178578516 shutdown_connections 2026-02-17T06:01:57.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 -- 10.20.193.16:0/2178578516 wait complete. 2026-02-17T06:01:57.747 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.744+0000 7f30e9961640 1 librados: shutdown 2026-02-17T06:01:57.752 INFO:tasks.ceph:Scrubbing osd.9 2026-02-17T06:01:57.752 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:57.821 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.821+0000 7fabc6b79640 10 monclient: get_monmap_and_config 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.821+0000 7fabc6b79640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.821+0000 7fabc6b79640 10 monclient: monmap: 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:57.822611+0000 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:57.822611+0000 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 1 Processor -- start 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 1 -- start start 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 10 monclient: init 2026-02-17T06:01:57.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 10 monclient: _add_conns ranks=[0,3,4,5,2,1] 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fabc00700f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fabc00c86f0 0x7fabc00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 10 monclient(hunting): picked mon.noname-a con 0x7fabc00c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fabc01536f0 0x7fabc0153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 10 monclient(hunting): picked mon.noname-b con 0x7fabc01536f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc00c4820 con 0x7fabc00700f0 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fabc00c4b00 con 0x7fabc00c86f0 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fabc00c4de0 con 0x7fabc01536f0 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.822+0000 7fabc6b79640 10 monclient(hunting): authenticate will time out at 1370.391658s 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabc50ef640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fabc01536f0 0x7fabc0153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabb7fff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fabc00c86f0 0x7fabc00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabc50ef640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fabc01536f0 0x7fabc0153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43462/0 (socket says 10.20.193.16:43462) 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabc50ef640 1 -- 10.20.193.16:0/388955438 learned_addr learned my addr 10.20.193.16:0/388955438 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabc50ef640 10 monclient(hunting): get_auth_request con 0x7fabc01536f0 auth_method 0 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabc50ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabc50ef640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabc50ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabb7fff640 10 monclient(hunting): get_auth_request con 0x7fabc00c86f0 auth_method 0 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabb7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabb7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabc50ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabc50ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.823+0000 7fabc50ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabc50ef640 10 monclient(hunting): handle_auth_done global_id 5194 payload 274 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabc50ef640 1 -- 10.20.193.16:0/388955438 >> v2:10.20.193.16:3300/0 conn(0x7fabc00c86f0 msgr2=0x7fabc00704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabc50ef640 1 --2- 10.20.193.16:0/388955438 >> v2:10.20.193.16:3300/0 conn(0x7fabc00c86f0 0x7fabc00704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabc50ef640 1 -- 10.20.193.16:0/388955438 >> v1:10.20.193.16:6789/0 conn(0x7fabc00700f0 legacy=0x7fabc00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabc50ef640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabc50ef640 1 monclient: found mon.noname-b 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabc50ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabc50ef640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:57.824 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.824+0000 7fabc50ef640 1 -- 10.20.193.16:0/388955438 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc00c50c0 con 0x7fabc01536f0 2026-02-17T06:01:57.825 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.825+0000 7fabc50ef640 1 --2- 10.20.193.16:0/388955438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fabc01536f0 0x7fabc0153b20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fabc00c9540 tx=0x7fabbc02f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1282f515dd456f00 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 1 -- 10.20.193.16:0/388955438 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc02fce0 con 0x7fabc01536f0 2026-02-17T06:01:57.826 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 10 monclient: dump: 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabc6b79640 5 monclient: authenticate success, global_id 5194 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabc6b79640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 1 -- 10.20.193.16:0/388955438 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabbc0373f0 con 0x7fabc01536f0 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 1 -- 10.20.193.16:0/388955438 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0377c0 con 0x7fabc01536f0 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabb77fe640 10 monclient: dump: 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.827 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabc6b79640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.826+0000 7fabc6b79640 4 set_mon_vals no callback set 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 10 monclient: shutdown 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 -- 10.20.193.16:0/388955438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fabc01536f0 msgr2=0x7fabc0153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 --2- 10.20.193.16:0/388955438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fabc01536f0 0x7fabc0153b20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fabc00c9540 tx=0x7fabbc02f1e0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 -- 10.20.193.16:0/388955438 shutdown_connections 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 --2- 10.20.193.16:0/388955438 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fabc01536f0 0x7fabc0153b20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 --2- 10.20.193.16:0/388955438 >> v2:10.20.193.16:3300/0 conn(0x7fabc00c86f0 0x7fabc00704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 -- 10.20.193.16:0/388955438 >> 10.20.193.16:0/388955438 conn(0x7fabc00cd770 msgr2=0x7fabc00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 -- 10.20.193.16:0/388955438 shutdown_connections 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 -- 10.20.193.16:0/388955438 wait complete. 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 10 monclient: monmap: 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:57.828 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 librados: starting msgr at 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 librados: starting objecter 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.827+0000 7fabc6b79640 1 Processor -- start 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 1 -- start start 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 1 librados: setting wanted keys 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 1 librados: calling monclient init 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient: init 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fabc00700f0 0x7fabc0155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient(hunting): picked mon.noname-b con 0x7fabc00700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fabc00c86f0 0x7fabc01f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient(hunting): picked mon.noname-c con 0x7fabc00c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fabc01536f0 0x7fabc00cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient(hunting): picked mon.noname-a con 0x7fabc01536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0154060 con 0x7fabc01536f0 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fabc00c5680 con 0x7fabc00700f0 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0074c00 con 0x7fabc00c86f0 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc6b79640 10 monclient(hunting): authenticate will time out at 1370.397433s 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc50ef640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fabc01536f0 0x7fabc00cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.829 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc48ee640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fabc00700f0 0x7fabc0155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabb7fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fabc00c86f0 0x7fabc01f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc50ef640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fabc01536f0 0x7fabc00cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33846/0 (socket says 10.20.193.16:33846) 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc50ef640 1 -- 10.20.193.16:0/2730468906 learned_addr learned my addr 10.20.193.16:0/2730468906 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc48ee640 10 monclient(hunting): get_auth_request con 0x7fabc00700f0 auth_method 0 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc48ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc48ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc48ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc50ef640 10 monclient(hunting): get_auth_request con 0x7fabc01536f0 auth_method 0 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc50ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc50ef640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.828+0000 7fabc50ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabb7fff640 10 monclient(hunting): get_auth_request con 0x7fabc00c86f0 auth_method 0 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabb7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabb7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabc50ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabc50ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabc50ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabc48ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabc48ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabc48ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.829+0000 7fabb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc48ee640 10 monclient(hunting): handle_auth_done global_id 5197 payload 824 2026-02-17T06:01:57.830 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc48ee640 1 -- 10.20.193.16:0/2730468906 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fabc00c86f0 msgr2=0x7fabc01f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc48ee640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fabc00c86f0 0x7fabc01f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc48ee640 1 -- 10.20.193.16:0/2730468906 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fabc01536f0 msgr2=0x7fabc00cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc48ee640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fabc01536f0 0x7fabc00cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc48ee640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc48ee640 1 monclient: found mon.noname-b 2026-02-17T06:01:57.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc48ee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc48ee640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:57.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc48ee640 1 -- 10.20.193.16:0/2730468906 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc0155fb0 con 0x7fabc00700f0 2026-02-17T06:01:57.831 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.830+0000 7fabc50ef640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fabc01536f0 0x7fabc00cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:57.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.831+0000 7fabc48ee640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fabc00700f0 0x7fabc0155870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fabb000c480 tx=0x7fabb000c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.831+0000 7fabb57fa640 10 client.?.objecter ms_handle_connect 0x7fabc00700f0 2026-02-17T06:01:57.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.831+0000 7fabb57fa640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 1 -- 10.20.193.16:0/2730468906 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0017070 con 0x7fabc00700f0 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 10 monclient: dump: 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.834 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 1 -- 10.20.193.16:0/2730468906 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb00052d0 con 0x7fabc00700f0 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 1 -- 10.20.193.16:0/2730468906 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00036f0 con 0x7fabc00700f0 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb6ffd640 4 set_mon_vals no callback set 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabb57fa640 10 monclient: dump: 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.835 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabc6b79640 5 monclient: authenticate success, global_id 5197 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabc6b79640 10 monclient: _renew_subs 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabc6b79640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.832+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc0156290 con 0x7fabc00700f0 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 10 client.5197.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 10 monclient: _renew_subs 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc0156850 con 0x7fabc00700f0 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 1 librados: init done 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 10 client.5197.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 15 client.5197.objecter _session_command_op_assign -1 1 2026-02-17T06:01:57.836 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 20 client.5197.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:57.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 10 client.5197.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:57.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:57.837 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.833+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabc00704d0 con 0x7fabc00700f0 2026-02-17T06:01:57.845 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.844+0000 7fabb57fa640 1 -- 10.20.193.16:0/2730468906 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fabb0003890 con 0x7fabc00700f0 2026-02-17T06:01:57.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fab90079190 0x7fab9007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb7fff640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fab90079190 0x7fab9007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb7fff640 10 monclient: get_auth_request con 0x7fab90079190 auth_method 0 2026-02-17T06:01:57.846 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 10 client.5197.objecter ms_handle_connect 0x7fab90079190 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb7fff640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fab90079190 0x7fab9007b650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fabc015aae0 tx=0x7faba8009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 1 -- 10.20.193.16:0/2730468906 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fabb005d850 con 0x7fabc00700f0 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 10 client.5197.objecter ms_dispatch2 0x7fabc0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 3 client.5197.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 3 client.5197.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 10 client.5197.objecter checking command 1 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] conn(0x7fab90081670 0x7fab90083a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 20 client.5197.objecter _get_session s=0x7fab90080fa0 osd=9 3 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 20 client.5197.objecter put_session s=0x7fab90080fa0 osd=9 3 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 15 client.5197.objecter _session_command_op_remove -1 1 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 20 client.5197.objecter _get_session s=0x7fab90080fa0 osd=9 3 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 20 client.5197.objecter get_session s=0x7fab90080fa0 osd=9 3 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 15 client.5197.objecter _session_command_op_assign 9 1 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 20 client.5197.objecter put_session s=0x7fab90080fa0 osd=9 4 2026-02-17T06:01:57.847 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 10 client.5197.objecter _send_command 1 2026-02-17T06:01:57.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 1 -- 10.20.193.16:0/2730468906 --> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab90084160 con 0x7fab90081670 2026-02-17T06:01:57.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 20 client.5197.objecter dump_active .. 0 homeless 2026-02-17T06:01:57.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 1 -- 10.20.193.16:0/2730468906 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fabb005dc10 con 0x7fabc00700f0 2026-02-17T06:01:57.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.845+0000 7fabb57fa640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:57.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.846+0000 7fabc50ef640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] conn(0x7fab90081670 0x7fab90083a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.846+0000 7fabc50ef640 10 monclient: get_auth_request con 0x7fab90081670 auth_method 0 2026-02-17T06:01:57.848 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.847+0000 7fabc50ef640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] conn(0x7fab90081670 0x7fab90083a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.849 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.847+0000 7fabb57fa640 10 client.5197.objecter ms_handle_connect 0x7fab90081670 2026-02-17T06:01:57.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.857+0000 7fabb57fa640 1 -- 10.20.193.16:0/2730468906 <== osd.9 v2:10.20.193.128:6800/1813385411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fab90084160 con 0x7fab90081670 2026-02-17T06:01:57.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.857+0000 7fabb57fa640 10 client.5197.objecter ms_dispatch2 0x7fabc0000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:57.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.857+0000 7fabb57fa640 10 client.5197.objecter _finish_command 1 = system:0 2026-02-17T06:01:57.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.857+0000 7fabb57fa640 20 client.5197.objecter put_session s=0x7fab90080fa0 osd=9 4 2026-02-17T06:01:57.858 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.857+0000 7fabb57fa640 15 client.5197.objecter _session_command_op_remove 9 1 2026-02-17T06:01:57.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 10 client.5197.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:57.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 15 client.5197.objecter _session_command_op_assign -1 2 2026-02-17T06:01:57.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 20 client.5197.objecter _get_session s=0x7fab90080fa0 osd=9 3 2026-02-17T06:01:57.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 20 client.5197.objecter put_session s=0x7fab90080fa0 osd=9 3 2026-02-17T06:01:57.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 20 client.5197.objecter _get_session s=0x7fab90080fa0 osd=9 3 2026-02-17T06:01:57.864 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 15 client.5197.objecter _session_command_op_remove -1 2 2026-02-17T06:01:57.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 20 client.5197.objecter get_session s=0x7fab90080fa0 osd=9 3 2026-02-17T06:01:57.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 15 client.5197.objecter _session_command_op_assign 9 2 2026-02-17T06:01:57.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 20 client.5197.objecter put_session s=0x7fab90080fa0 osd=9 4 2026-02-17T06:01:57.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 10 client.5197.objecter _send_command 2 2026-02-17T06:01:57.865 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.863+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 --> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fabc01573f0 con 0x7fab90081670 2026-02-17T06:01:57.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabb57fa640 1 -- 10.20.193.16:0/2730468906 <== osd.9 v2:10.20.193.128:6800/1813385411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fabc01573f0 con 0x7fab90081670 2026-02-17T06:01:57.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabb57fa640 10 client.5197.objecter ms_dispatch2 0x7fabc0000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:57.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabb57fa640 10 client.5197.objecter _finish_command 2 = system:0 2026-02-17T06:01:57.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabb57fa640 20 client.5197.objecter put_session s=0x7fab90080fa0 osd=9 4 2026-02-17T06:01:57.871 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabb57fa640 15 client.5197.objecter _session_command_op_remove 9 2 2026-02-17T06:01:57.871 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:57.872 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:57.872 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:57.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 10 librados: watch_flush enter 2026-02-17T06:01:57.873 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 10 librados: watch_flush exit 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 10 client.5197.objecter close_session for osd.9 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 >> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] conn(0x7fab90081670 msgr2=0x7fab90083a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.128:6800/1813385411,v1:10.20.193.128:6801/1813385411] conn(0x7fab90081670 0x7fab90083a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 20 client.5197.objecter put_session s=0x7fab90080fa0 osd=9 1 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 20 client.5197.objecter shutdown clearing up homeless session... 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 10 client.5197.objecter successfully canceled tick 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fab90079190 msgr2=0x7fab9007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fab90079190 0x7fab9007b650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fabc015aae0 tx=0x7faba8009290 comp rx=0 tx=0).stop 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 10 monclient: shutdown 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fabc00700f0 msgr2=0x7fabc0155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.874 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 1 --2- 10.20.193.16:0/2730468906 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fabc00700f0 0x7fabc0155870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fabb000c480 tx=0x7fabb000c950 comp rx=0 tx=0).stop 2026-02-17T06:01:57.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc50ef640 1 -- 10.20.193.16:0/2730468906 reap_dead start 2026-02-17T06:01:57.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 shutdown_connections 2026-02-17T06:01:57.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 >> 10.20.193.16:0/2730468906 conn(0x7fabc00cd770 msgr2=0x7fabc00c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 shutdown_connections 2026-02-17T06:01:57.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.870+0000 7fabc6b79640 1 -- 10.20.193.16:0/2730468906 wait complete. 2026-02-17T06:01:57.875 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.871+0000 7fabc6b79640 1 librados: shutdown 2026-02-17T06:01:57.880 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2026-02-17T06:01:57.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.948+0000 7f111f5de640 10 monclient: get_monmap_and_config 2026-02-17T06:01:57.949 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.948+0000 7f111f5de640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.948+0000 7f111f5de640 10 monclient: monmap: 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:57.949670+0000 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:57.949670+0000 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:57.950 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:57.951 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:57.951 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:57.951 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 1 Processor -- start 2026-02-17T06:01:57.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 1 -- start start 2026-02-17T06:01:57.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 10 monclient: init 2026-02-17T06:01:57.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 10 monclient: _add_conns ranks=[5,1,2,3,0,4] 2026-02-17T06:01:57.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f11180700f0 0x7f11180704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 10 monclient(hunting): picked mon.noname-c con 0x7f11180700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:57.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f11180c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:57.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f11181536f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:57.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11180c4820 con 0x7f11180c7fe0 2026-02-17T06:01:57.954 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11180c4b00 con 0x7f11181536f0 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f11180c4de0 con 0x7f11180700f0 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.949+0000 7f111f5de640 10 monclient(hunting): authenticate will time out at 1370.518638s 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.950+0000 7f111cb52640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f11180c7fe0 0x7f11180c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:51298/0 (socket says 10.20.193.16:51298) 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.950+0000 7f111cb52640 1 -- 10.20.193.16:0/1611401702 learned_addr learned my addr 10.20.193.16:0/1611401702 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f111d353640 1 --2- 10.20.193.16:0/1611401702 >> v2:10.20.193.148:3300/0 conn(0x7f11180700f0 0x7f11180704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 1 -- 10.20.193.16:0/1611401702 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1432467994 0 0) 0x7f11180c4820 con 0x7f11180c7fe0 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 10 monclient(hunting): _negotiate 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.955 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 10 monclient(hunting): my global_id is 5200 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 1 -- 10.20.193.16:0/1611401702 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1100003620 con 0x7f11180c7fe0 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f111d353640 10 monclient(hunting): get_auth_request con 0x7f11180700f0 auth_method 0 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f111d353640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f111d353640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f111d353640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 1 -- 10.20.193.16:0/1611401702 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3998512460 0 0) 0x7f11180c4b00 con 0x7f11181536f0 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 10 monclient(hunting): _negotiate 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 10 monclient(hunting): my global_id is 5252 2026-02-17T06:01:57.956 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.951+0000 7f1107fff640 1 -- 10.20.193.16:0/1611401702 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11180c4820 con 0x7f11181536f0 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 10 monclient(hunting): handle_auth_done global_id 5255 payload 274 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 1 -- 10.20.193.16:0/1611401702 >> v1:10.20.193.148:6789/0 conn(0x7f11181536f0 legacy=0x7f1118153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 1 -- 10.20.193.16:0/1611401702 >> v1:10.20.193.128:6789/0 conn(0x7f11180c7fe0 legacy=0x7f11180c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 1 monclient: found mon.noname-c 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:57.957 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.952+0000 7f111d353640 1 -- 10.20.193.16:0/1611401702 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11180c50c0 con 0x7f11180700f0 2026-02-17T06:01:57.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.954+0000 7f111d353640 1 --2- 10.20.193.16:0/1611401702 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11180700f0 0x7f11180704d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f11080098a0 tx=0x7f110802edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da9cee0c7f30a5d8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.955+0000 7f1107fff640 1 -- 10.20.193.16:0/1611401702 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110803c070 con 0x7f11180700f0 2026-02-17T06:01:57.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.955+0000 7f1107fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.955+0000 7f1107fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:57.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.955+0000 7f1107fff640 10 monclient: dump: 2026-02-17T06:01:57.965 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.955+0000 7f1107fff640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.955+0000 7f1107fff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.955+0000 7f1107fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.955+0000 7f111f5de640 5 monclient: authenticate success, global_id 5255 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.955+0000 7f111f5de640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f1107fff640 1 -- 10.20.193.16:0/1611401702 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110802fae0 con 0x7f11180700f0 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f1107fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f1107fff640 1 -- 10.20.193.16:0/1611401702 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108037440 con 0x7f11180700f0 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 4 set_mon_vals no callback set 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f1107fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f1107fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f1107fff640 10 monclient: dump: 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.967 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 10 monclient: shutdown 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:57.968 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 1 -- 10.20.193.16:0/1611401702 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11180700f0 msgr2=0x7f11180704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 1 --2- 10.20.193.16:0/1611401702 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11180700f0 0x7f11180704d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f11080098a0 tx=0x7f110802edd0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 1 -- 10.20.193.16:0/1611401702 shutdown_connections 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 1 --2- 10.20.193.16:0/1611401702 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11180700f0 0x7f11180704d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 1 -- 10.20.193.16:0/1611401702 >> 10.20.193.16:0/1611401702 conn(0x7f11180cd770 msgr2=0x7f11180cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 1 -- 10.20.193.16:0/1611401702 shutdown_connections 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.956+0000 7f111f5de640 1 -- 10.20.193.16:0/1611401702 wait complete. 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient: build_initial_monmap 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient: monmap: 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:57.969 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 librados: starting msgr at 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 librados: starting objecter 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 Processor -- start 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 -- start start 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 librados: setting wanted keys 2026-02-17T06:01:57.970 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 librados: calling monclient init 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient: init 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f11180700f0 0x7f1118151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient(hunting): picked mon.noname-a con 0x7f11180700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11180c7fe0 0x7f111813f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient(hunting): picked mon.noname-b con 0x7f11180c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11181536f0 0x7f11181407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient(hunting): picked mon.noname-c con 0x7f11181536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1118074c00 con 0x7f11180700f0 2026-02-17T06:01:57.971 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f11180c5680 con 0x7f11180c7fe0 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1118074d80 con 0x7f11181536f0 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.957+0000 7f111f5de640 10 monclient(hunting): authenticate will time out at 1370.526721s 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111d353640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f11180700f0 0x7f1118151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111cb52640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11180c7fe0 0x7f111813f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111db54640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11181536f0 0x7f11181407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111db54640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11181536f0 0x7f11181407d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:53318/0 (socket says 10.20.193.16:53318) 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111db54640 1 -- 10.20.193.16:0/1389122153 learned_addr learned my addr 10.20.193.16:0/1389122153 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111db54640 10 monclient(hunting): get_auth_request con 0x7f11181536f0 auth_method 0 2026-02-17T06:01:57.972 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111db54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111db54640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111db54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111cb52640 10 monclient(hunting): get_auth_request con 0x7f11180c7fe0 auth_method 0 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111cb52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111cb52640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111cb52640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111d353640 10 monclient(hunting): get_auth_request con 0x7f11180700f0 auth_method 0 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111d353640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111d353640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111d353640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111db54640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111db54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.973 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111db54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111d353640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111d353640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111d353640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111cb52640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111cb52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.958+0000 7f111cb52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.959+0000 7f111db54640 10 monclient(hunting): handle_auth_done global_id 5258 payload 824 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.959+0000 7f111db54640 1 -- 10.20.193.16:0/1389122153 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11180c7fe0 msgr2=0x7f111813f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.959+0000 7f111db54640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11180c7fe0 0x7f111813f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.959+0000 7f111db54640 1 -- 10.20.193.16:0/1389122153 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f11180700f0 msgr2=0x7f1118151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.959+0000 7f111db54640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f11180700f0 0x7f1118151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:57.974 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.959+0000 7f111db54640 10 monclient: _finish_hunting 0 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.959+0000 7f111db54640 1 monclient: found mon.noname-c 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.959+0000 7f111db54640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.959+0000 7f111db54640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.959+0000 7f111db54640 1 -- 10.20.193.16:0/1389122153 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1118151bc0 con 0x7f11181536f0 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.960+0000 7f111db54640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11181536f0 0x7f11181407d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f111400cb90 tx=0x7f111400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.960+0000 7f1105ffb640 10 client.?.objecter ms_handle_connect 0x7f11181536f0 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.960+0000 7f1105ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f1105ffb640 1 -- 10.20.193.16:0/1389122153 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111400ad50 con 0x7f11181536f0 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f1105ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f1105ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f1105ffb640 10 monclient: dump: 2026-02-17T06:01:57.975 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f1105ffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f1105ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f1105ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:57.976 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f111f5de640 5 monclient: authenticate success, global_id 5258 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f111f5de640 10 monclient: _renew_subs 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f111f5de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f111f5de640 1 -- 10.20.193.16:0/1389122153 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1118151ea0 con 0x7f11181536f0 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f111f5de640 10 client.5258.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f111f5de640 10 monclient: _renew_subs 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f111f5de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f111f5de640 1 -- 10.20.193.16:0/1389122153 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111806dd60 con 0x7f11181536f0 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f111f5de640 1 librados: init done 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f1105ffb640 1 -- 10.20.193.16:0/1389122153 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1114003a70 con 0x7f11181536f0 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f1105ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.961+0000 7f11077fe640 4 set_mon_vals no callback set 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.962+0000 7f1105ffb640 1 -- 10.20.193.16:0/1389122153 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114003e20 con 0x7f11181536f0 2026-02-17T06:01:57.977 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.962+0000 7f1105ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.962+0000 7f1105ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.962+0000 7f1105ffb640 10 monclient: dump: 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:57.978 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:57.979 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:57.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.962+0000 7f111f5de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.962+0000 7f111f5de640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:57.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.962+0000 7f111f5de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:57.979 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.962+0000 7f111f5de640 1 -- 10.20.193.16:0/1389122153 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111806df60 con 0x7f11181536f0 2026-02-17T06:01:57.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.974+0000 7f1105ffb640 1 -- 10.20.193.16:0/1389122153 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f11140050d0 con 0x7f11181536f0 2026-02-17T06:01:57.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.974+0000 7f1105ffb640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f10e8079140 0x7f10e807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:57.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.975+0000 7f111d353640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f10e8079140 0x7f10e807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:57.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.975+0000 7f111d353640 10 monclient: get_auth_request con 0x7f10e8079140 auth_method 0 2026-02-17T06:01:57.980 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.975+0000 7f1105ffb640 10 client.5258.objecter ms_handle_connect 0x7f10e8079140 2026-02-17T06:01:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.975+0000 7f111d353640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f10e8079140 0x7f10e807b600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1108039000 tx=0x7f1108033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.976+0000 7f1105ffb640 1 -- 10.20.193.16:0/1389122153 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1114098220 con 0x7f11181536f0 2026-02-17T06:01:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.976+0000 7f1105ffb640 10 client.5258.objecter ms_dispatch2 0x7f1118000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.976+0000 7f1105ffb640 3 client.5258.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.976+0000 7f1105ffb640 3 client.5258.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:57.981 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:57.976+0000 7f1105ffb640 20 client.5258.objecter dump_active .. 0 homeless 2026-02-17T06:01:58.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.078+0000 7f1105ffb640 1 -- 10.20.193.16:0/1389122153 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f111806df60 con 0x7f11181536f0 2026-02-17T06:01:58.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.078+0000 7f1105ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.078+0000 7f1105ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:58.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.161+0000 7f111f5de640 1 -- 10.20.193.16:0/1389122153 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "9", "target": ["mon-mgr", ""]}) -- 0x7f11180704d0 con 0x7f10e8079140 2026-02-17T06:01:58.162 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.161+0000 7f1105ffb640 1 -- 10.20.193.16:0/1389122153 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 9 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f11180704d0 con 0x7f10e8079140 2026-02-17T06:01:58.162 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 9 to deep-scrub 2026-02-17T06:01:58.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 10 librados: watch_flush enter 2026-02-17T06:01:58.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 10 librados: watch_flush exit 2026-02-17T06:01:58.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 20 client.5258.objecter shutdown clearing up homeless session... 2026-02-17T06:01:58.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 10 client.5258.objecter successfully canceled tick 2026-02-17T06:01:58.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 -- 10.20.193.16:0/1389122153 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f10e8079140 msgr2=0x7f10e807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f10e8079140 0x7f10e807b600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1108039000 tx=0x7f1108033000 comp rx=0 tx=0).stop 2026-02-17T06:01:58.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 10 monclient: shutdown 2026-02-17T06:01:58.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.164 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 -- 10.20.193.16:0/1389122153 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11181536f0 msgr2=0x7f11181407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11181536f0 0x7f11181407d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f111400cb90 tx=0x7f111400a430 comp rx=0 tx=0).stop 2026-02-17T06:01:58.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 -- 10.20.193.16:0/1389122153 shutdown_connections 2026-02-17T06:01:58.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f10e8079140 0x7f10e807b600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11180c7fe0 0x7f111813f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11181536f0 0x7f11181407d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 --2- 10.20.193.16:0/1389122153 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f11180700f0 0x7f1118151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 -- 10.20.193.16:0/1389122153 >> 10.20.193.16:0/1389122153 conn(0x7f11180cd770 msgr2=0x7f111814e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 -- 10.20.193.16:0/1389122153 shutdown_connections 2026-02-17T06:01:58.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 -- 10.20.193.16:0/1389122153 wait complete. 2026-02-17T06:01:58.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.162+0000 7f111f5de640 1 librados: shutdown 2026-02-17T06:01:58.171 INFO:tasks.ceph:Scrubbing osd.13 2026-02-17T06:01:58.171 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:58.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.239+0000 7efc4f9c1640 10 monclient: get_monmap_and_config 2026-02-17T06:01:58.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.239+0000 7efc4f9c1640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.240 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.239+0000 7efc4f9c1640 10 monclient: monmap: 2026-02-17T06:01:58.240 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:58.240 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.240 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:58.240740+0000 2026-02-17T06:01:58.241 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:58.240740+0000 2026-02-17T06:01:58.241 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.241 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.241 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:58.241 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:58.241 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:58.241 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:58.241 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:58.241 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:58.241 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 1 Processor -- start 2026-02-17T06:01:58.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 1 -- start start 2026-02-17T06:01:58.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 10 monclient: init 2026-02-17T06:01:58.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.244 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 10 monclient: _add_conns ranks=[4,1,5,2,0,3] 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7efc48155ac0 0x7efc48073890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 10 monclient(hunting): picked mon.noname-b con 0x7efc48155ac0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7efc48155ea0 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7efc48070590 0x7efc4815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 10 monclient(hunting): picked mon.noname-c con 0x7efc48070590 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc48072510 con 0x7efc48155ea0 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7efc480c8d10 con 0x7efc48155ac0 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7efc480727f0 con 0x7efc48070590 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.245 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.240+0000 7efc4f9c1640 10 monclient(hunting): authenticate will time out at 1370.809834s 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4d736640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7efc48155ac0 0x7efc48073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4df37640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7efc48070590 0x7efc4815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4cf35640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7efc48155ea0 0x7efc4806fe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:51308/0 (socket says 10.20.193.16:51308) 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4cf35640 1 -- 10.20.193.16:0/757525796 learned_addr learned my addr 10.20.193.16:0/757525796 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4df37640 10 monclient(hunting): get_auth_request con 0x7efc48070590 auth_method 0 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4df37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4df37640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4df37640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4d736640 10 monclient(hunting): get_auth_request con 0x7efc48155ac0 auth_method 0 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4d736640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4d736640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.246 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4d736640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4df37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4df37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4df37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4d736640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4d736640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.241+0000 7efc4d736640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.242+0000 7efc4d736640 10 monclient(hunting): handle_auth_done global_id 5206 payload 274 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.242+0000 7efc4d736640 1 -- 10.20.193.16:0/757525796 >> v2:10.20.193.148:3300/0 conn(0x7efc48070590 msgr2=0x7efc4815d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.242+0000 7efc4d736640 1 --2- 10.20.193.16:0/757525796 >> v2:10.20.193.148:3300/0 conn(0x7efc48070590 0x7efc4815d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.242+0000 7efc4d736640 1 -- 10.20.193.16:0/757525796 >> v1:10.20.193.128:6789/0 conn(0x7efc48155ea0 legacy=0x7efc4806fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.242+0000 7efc4d736640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.242+0000 7efc4d736640 1 monclient: found mon.noname-b 2026-02-17T06:01:58.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.242+0000 7efc4d736640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.242+0000 7efc4d736640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:58.248 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.242+0000 7efc4d736640 1 -- 10.20.193.16:0/757525796 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc48072ad0 con 0x7efc48155ac0 2026-02-17T06:01:58.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.244+0000 7efc4d736640 1 --2- 10.20.193.16:0/757525796 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc48155ac0 0x7efc48073890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc4806f470 tx=0x7efc3802ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fef39f6f5d05fc62 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.253 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 1 -- 10.20.193.16:0/757525796 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3803c070 con 0x7efc48155ac0 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 10 monclient: dump: 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.254 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 1 -- 10.20.193.16:0/757525796 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc3802faf0 con 0x7efc48155ac0 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 5 monclient: authenticate success, global_id 5206 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 1 -- 10.20.193.16:0/757525796 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38037440 con 0x7efc48155ac0 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:58.255 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 4 set_mon_vals no callback set 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc37fff640 10 monclient: dump: 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.256 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 10 monclient: shutdown 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 1 -- 10.20.193.16:0/757525796 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc48155ac0 msgr2=0x7efc48073890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 1 --2- 10.20.193.16:0/757525796 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc48155ac0 0x7efc48073890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc4806f470 tx=0x7efc3802ede0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 1 -- 10.20.193.16:0/757525796 shutdown_connections 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 1 --2- 10.20.193.16:0/757525796 >> v2:10.20.193.148:3300/0 conn(0x7efc48070590 0x7efc4815d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 1 --2- 10.20.193.16:0/757525796 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc48155ac0 0x7efc48073890 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 1 -- 10.20.193.16:0/757525796 >> 10.20.193.16:0/757525796 conn(0x7efc480cd770 msgr2=0x7efc480cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 1 -- 10.20.193.16:0/757525796 shutdown_connections 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.246+0000 7efc4f9c1640 1 -- 10.20.193.16:0/757525796 wait complete. 2026-02-17T06:01:58.257 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.247+0000 7efc4f9c1640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.247+0000 7efc4f9c1640 10 monclient: monmap: 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.258 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.247+0000 7efc4f9c1640 1 librados: starting msgr at 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.247+0000 7efc4f9c1640 1 librados: starting objecter 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.247+0000 7efc4f9c1640 1 Processor -- start 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.247+0000 7efc4f9c1640 1 -- start start 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.247+0000 7efc4f9c1640 1 librados: setting wanted keys 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.247+0000 7efc4f9c1640 1 librados: calling monclient init 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.247+0000 7efc4f9c1640 10 monclient: init 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc48070590 0x7efc48159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 10 monclient(hunting): picked mon.noname-b con 0x7efc48070590 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efc48155ac0 0x7efc481e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.259 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 10 monclient(hunting): picked mon.noname-c con 0x7efc48155ac0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc48155ea0 0x7efc4814ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 10 monclient(hunting): picked mon.noname-a con 0x7efc48155ea0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc48073370 con 0x7efc48155ea0 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7efc48073090 con 0x7efc48070590 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7efc4815cb30 con 0x7efc48155ac0 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4f9c1640 10 monclient(hunting): authenticate will time out at 1370.817289s 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4d736640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc48070590 0x7efc48159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.260 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4df37640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc48155ea0 0x7efc4814ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4d736640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc48070590 0x7efc48159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43502/0 (socket says 10.20.193.16:43502) 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4d736640 1 -- 10.20.193.16:0/3535656538 learned_addr learned my addr 10.20.193.16:0/3535656538 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4cf35640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efc48155ac0 0x7efc481e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4d736640 10 monclient(hunting): get_auth_request con 0x7efc48070590 auth_method 0 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4d736640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4d736640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4d736640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4cf35640 10 monclient(hunting): get_auth_request con 0x7efc48155ac0 auth_method 0 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4cf35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4cf35640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4cf35640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4df37640 10 monclient(hunting): get_auth_request con 0x7efc48155ea0 auth_method 0 2026-02-17T06:01:58.261 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4df37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4df37640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4df37640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4d736640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4d736640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.248+0000 7efc4d736640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.249+0000 7efc4df37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.249+0000 7efc4df37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.249+0000 7efc4df37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.249+0000 7efc4cf35640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.249+0000 7efc4cf35640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.249+0000 7efc4cf35640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.262 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.250+0000 7efc4d736640 10 monclient(hunting): handle_auth_done global_id 5212 payload 824 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.250+0000 7efc4d736640 1 -- 10.20.193.16:0/3535656538 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efc48155ac0 msgr2=0x7efc481e79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.250+0000 7efc4d736640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7efc48155ac0 0x7efc481e79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.250+0000 7efc4d736640 1 -- 10.20.193.16:0/3535656538 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc48155ea0 msgr2=0x7efc4814ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.250+0000 7efc4d736640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7efc48155ea0 0x7efc4814ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.250+0000 7efc4d736640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.250+0000 7efc4d736640 1 monclient: found mon.noname-b 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.250+0000 7efc4d736640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.250+0000 7efc4d736640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.250+0000 7efc4d736640 1 -- 10.20.193.16:0/3535656538 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4806d730 con 0x7efc48070590 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.251+0000 7efc4d736640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc48070590 0x7efc48159f80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efc38009130 tx=0x7efc380047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.251+0000 7efc35ffb640 10 client.?.objecter ms_handle_connect 0x7efc48070590 2026-02-17T06:01:58.263 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.251+0000 7efc35ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 1 -- 10.20.193.16:0/3535656538 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3803c040 con 0x7efc48070590 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 10 monclient: dump: 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.264 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 5 monclient: authenticate success, global_id 5212 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 10 monclient: _renew_subs 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4806da10 con 0x7efc48070590 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 10 client.5212.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 10 monclient: _renew_subs 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:58.265 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4806dfd0 con 0x7efc48070590 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 1 librados: init done 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 1 -- 10.20.193.16:0/3535656538 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc3802fa70 con 0x7efc48070590 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 1 -- 10.20.193.16:0/3535656538 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38042810 con 0x7efc48070590 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 10 client.5212.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 15 client.5212.objecter _session_command_op_assign -1 1 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc377fe640 4 set_mon_vals no callback set 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc35ffb640 10 monclient: dump: 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.266 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 20 client.5212.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 10 client.5212.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:58.267 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.252+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efc48073890 con 0x7efc48070590 2026-02-17T06:01:58.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.264+0000 7efc35ffb640 1 -- 10.20.193.16:0/3535656538 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7efc380429b0 con 0x7efc48070590 2026-02-17T06:01:58.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.264+0000 7efc35ffb640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc18079140 0x7efc1807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.264+0000 7efc4cf35640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc18079140 0x7efc1807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.264+0000 7efc4cf35640 10 monclient: get_auth_request con 0x7efc18079140 auth_method 0 2026-02-17T06:01:58.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 10 client.5212.objecter ms_handle_connect 0x7efc18079140 2026-02-17T06:01:58.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc4cf35640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc18079140 0x7efc1807b600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efc48159d70 tx=0x7efc3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 1 -- 10.20.193.16:0/3535656538 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7efc380bc720 con 0x7efc48070590 2026-02-17T06:01:58.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 10 client.5212.objecter ms_dispatch2 0x7efc48000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:58.270 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 3 client.5212.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 3 client.5212.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 10 client.5212.objecter checking command 1 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] conn(0x7efc18081620 0x7efc18083a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 20 client.5212.objecter _get_session s=0x7efc18080f50 osd=13 3 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 20 client.5212.objecter put_session s=0x7efc18080f50 osd=13 3 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 15 client.5212.objecter _session_command_op_remove -1 1 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 20 client.5212.objecter _get_session s=0x7efc18080f50 osd=13 3 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 20 client.5212.objecter get_session s=0x7efc18080f50 osd=13 3 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 15 client.5212.objecter _session_command_op_assign 13 1 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 20 client.5212.objecter put_session s=0x7efc18080f50 osd=13 4 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 10 client.5212.objecter _send_command 1 2026-02-17T06:01:58.271 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 1 -- 10.20.193.16:0/3535656538 --> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc18084110 con 0x7efc18081620 2026-02-17T06:01:58.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 20 client.5212.objecter dump_active .. 0 homeless 2026-02-17T06:01:58.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 1 -- 10.20.193.16:0/3535656538 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efc38083260 con 0x7efc48070590 2026-02-17T06:01:58.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.265+0000 7efc35ffb640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:58.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.266+0000 7efc4df37640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] conn(0x7efc18081620 0x7efc18083a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.266+0000 7efc4df37640 10 monclient: get_auth_request con 0x7efc18081620 auth_method 0 2026-02-17T06:01:58.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.267+0000 7efc4df37640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] conn(0x7efc18081620 0x7efc18083a40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.13 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.272 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.267+0000 7efc35ffb640 10 client.5212.objecter ms_handle_connect 0x7efc18081620 2026-02-17T06:01:58.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.278+0000 7efc35ffb640 1 -- 10.20.193.16:0/3535656538 <== osd.13 v2:10.20.193.128:6816/1434515412 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7efc18084110 con 0x7efc18081620 2026-02-17T06:01:58.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.278+0000 7efc35ffb640 10 client.5212.objecter ms_dispatch2 0x7efc48000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:58.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.278+0000 7efc35ffb640 10 client.5212.objecter _finish_command 1 = system:0 2026-02-17T06:01:58.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.278+0000 7efc35ffb640 20 client.5212.objecter put_session s=0x7efc18080f50 osd=13 4 2026-02-17T06:01:58.278 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.278+0000 7efc35ffb640 15 client.5212.objecter _session_command_op_remove 13 1 2026-02-17T06:01:58.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 10 client.5212.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:58.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 15 client.5212.objecter _session_command_op_assign -1 2 2026-02-17T06:01:58.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 20 client.5212.objecter _get_session s=0x7efc18080f50 osd=13 3 2026-02-17T06:01:58.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 20 client.5212.objecter put_session s=0x7efc18080f50 osd=13 3 2026-02-17T06:01:58.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 20 client.5212.objecter _get_session s=0x7efc18080f50 osd=13 3 2026-02-17T06:01:58.284 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 15 client.5212.objecter _session_command_op_remove -1 2 2026-02-17T06:01:58.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 20 client.5212.objecter get_session s=0x7efc18080f50 osd=13 3 2026-02-17T06:01:58.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 15 client.5212.objecter _session_command_op_assign 13 2 2026-02-17T06:01:58.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 20 client.5212.objecter put_session s=0x7efc18080f50 osd=13 4 2026-02-17T06:01:58.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 10 client.5212.objecter _send_command 2 2026-02-17T06:01:58.285 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.284+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 --> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efc4815f030 con 0x7efc18081620 2026-02-17T06:01:58.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc35ffb640 1 -- 10.20.193.16:0/3535656538 <== osd.13 v2:10.20.193.128:6816/1434515412 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efc4815f030 con 0x7efc18081620 2026-02-17T06:01:58.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc35ffb640 10 client.5212.objecter ms_dispatch2 0x7efc48000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:58.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc35ffb640 10 client.5212.objecter _finish_command 2 = system:0 2026-02-17T06:01:58.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc35ffb640 20 client.5212.objecter put_session s=0x7efc18080f50 osd=13 4 2026-02-17T06:01:58.291 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc35ffb640 15 client.5212.objecter _session_command_op_remove 13 2 2026-02-17T06:01:58.291 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:58.291 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:58.291 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:58.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 10 librados: watch_flush enter 2026-02-17T06:01:58.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 10 librados: watch_flush exit 2026-02-17T06:01:58.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 10 client.5212.objecter close_session for osd.13 2026-02-17T06:01:58.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 >> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] conn(0x7efc18081620 msgr2=0x7efc18083a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.128:6816/1434515412,v1:10.20.193.128:6817/1434515412] conn(0x7efc18081620 0x7efc18083a40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 20 client.5212.objecter put_session s=0x7efc18080f50 osd=13 1 2026-02-17T06:01:58.293 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 20 client.5212.objecter shutdown clearing up homeless session... 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 10 client.5212.objecter successfully canceled tick 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc18079140 msgr2=0x7efc1807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7efc18079140 0x7efc1807b600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efc48159d70 tx=0x7efc3c009040 comp rx=0 tx=0).stop 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 10 monclient: shutdown 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc48070590 msgr2=0x7efc48159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 --2- 10.20.193.16:0/3535656538 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7efc48070590 0x7efc48159f80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efc38009130 tx=0x7efc380047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4df37640 1 -- 10.20.193.16:0/3535656538 reap_dead start 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 shutdown_connections 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 >> 10.20.193.16:0/3535656538 conn(0x7efc480cd770 msgr2=0x7efc480cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.294 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 shutdown_connections 2026-02-17T06:01:58.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 -- 10.20.193.16:0/3535656538 wait complete. 2026-02-17T06:01:58.295 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.290+0000 7efc4f9c1640 1 librados: shutdown 2026-02-17T06:01:58.300 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2026-02-17T06:01:58.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.367+0000 7f8404da8640 10 monclient: get_monmap_and_config 2026-02-17T06:01:58.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.367+0000 7f8404da8640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.368 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.367+0000 7f8404da8640 10 monclient: monmap: 2026-02-17T06:01:58.368 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:58.368 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.368 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:58.368720+0000 2026-02-17T06:01:58.369 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:58.368720+0000 2026-02-17T06:01:58.369 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.369 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.369 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:58.369 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:58.369 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:58.369 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:58.369 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:58.369 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:58.369 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.368+0000 7f8404da8640 1 Processor -- start 2026-02-17T06:01:58.372 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.368+0000 7f8404da8640 1 -- start start 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.368+0000 7f8404da8640 10 monclient: init 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.368+0000 7f8404da8640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.368+0000 7f8404da8640 10 monclient: _add_conns ranks=[5,1,0,3,2,4] 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.368+0000 7f8404da8640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f84000700f0 0x7f84000704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.368+0000 7f8404da8640 10 monclient(hunting): picked mon.noname-c con 0x7f84000700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f8404da8640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f84000c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f8404da8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f84001536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f8404da8640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84000c4820 con 0x7f84001536f0 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f8404da8640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84000c4b00 con 0x7f84000c7fe0 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f8404da8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.373 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f8404da8640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f84000c4de0 con 0x7f84000700f0 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f8404da8640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f8404da8640 10 monclient(hunting): authenticate will time out at 1370.938034s 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f83fe575640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f84000700f0 0x7f84000704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f83fed76640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f84001536f0 0x7f8400153b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:41898/0 (socket says 10.20.193.16:41898) 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f83fed76640 1 -- 10.20.193.16:0/14396211 learned_addr learned my addr 10.20.193.16:0/14396211 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f83fe575640 10 monclient(hunting): get_auth_request con 0x7f84000700f0 auth_method 0 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f83fe575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f83fe575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f83fe575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f83fe575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f83fe575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.374 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.369+0000 7f83fe575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 1 -- 10.20.193.16:0/14396211 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 513074253 0 0) 0x7f84000c4b00 con 0x7f84000c7fe0 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 10 monclient(hunting): _negotiate 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 10 monclient(hunting): my global_id is 5215 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 1 -- 10.20.193.16:0/14396211 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83ec003620 con 0x7f84000c7fe0 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 1 -- 10.20.193.16:0/14396211 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1560450283 0 0) 0x7f84000c4820 con 0x7f84001536f0 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 10 monclient(hunting): _negotiate 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 10 monclient(hunting): my global_id is 5124 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fd573640 1 -- 10.20.193.16:0/14396211 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84000c4b00 con 0x7f84001536f0 2026-02-17T06:01:58.375 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fe575640 10 monclient(hunting): handle_auth_done global_id 5267 payload 274 2026-02-17T06:01:58.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fe575640 1 -- 10.20.193.16:0/14396211 >> v1:10.20.193.128:6789/0 conn(0x7f84000c7fe0 legacy=0x7f84000c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fe575640 1 -- 10.20.193.16:0/14396211 >> v1:10.20.193.16:6789/0 conn(0x7f84001536f0 legacy=0x7f8400153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fe575640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fe575640 1 monclient: found mon.noname-c 2026-02-17T06:01:58.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fe575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fe575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:58.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.370+0000 7f83fe575640 1 -- 10.20.193.16:0/14396211 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84000c50c0 con 0x7f84000700f0 2026-02-17T06:01:58.376 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.371+0000 7f83fe575640 1 --2- 10.20.193.16:0/14396211 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f84000700f0 0x7f84000704d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f83f40098a0 tx=0x7f83f402edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40710b9f79a101a8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.382 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.372+0000 7f83fd573640 1 -- 10.20.193.16:0/14396211 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f403c070 con 0x7f84000700f0 2026-02-17T06:01:58.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.372+0000 7f83fd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.372+0000 7f83fd573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:58.383 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.372+0000 7f83fd573640 10 monclient: dump: 2026-02-17T06:01:58.383 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.383 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.383 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.383 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.383 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.383 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.383 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.372+0000 7f83fd573640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.372+0000 7f83fd573640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.372+0000 7f83fd573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.372+0000 7f8404da8640 5 monclient: authenticate success, global_id 5267 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.372+0000 7f8404da8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f83fd573640 1 -- 10.20.193.16:0/14396211 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f402fae0 con 0x7f84000700f0 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f83fd573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f83fd573640 1 -- 10.20.193.16:0/14396211 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4037440 con 0x7f84000700f0 2026-02-17T06:01:58.384 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f83fd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f83fd573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f83fd573640 10 monclient: dump: 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.385 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 4 set_mon_vals no callback set 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 10 monclient: shutdown 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 1 -- 10.20.193.16:0/14396211 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f84000700f0 msgr2=0x7f84000704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 1 --2- 10.20.193.16:0/14396211 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f84000700f0 0x7f84000704d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f83f40098a0 tx=0x7f83f402edd0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 1 -- 10.20.193.16:0/14396211 shutdown_connections 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 1 --2- 10.20.193.16:0/14396211 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f84000700f0 0x7f84000704d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 1 -- 10.20.193.16:0/14396211 >> 10.20.193.16:0/14396211 conn(0x7f84000cd770 msgr2=0x7f84000cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 1 -- 10.20.193.16:0/14396211 shutdown_connections 2026-02-17T06:01:58.386 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.373+0000 7f8404da8640 1 -- 10.20.193.16:0/14396211 wait complete. 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.374+0000 7f8404da8640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.374+0000 7f8404da8640 10 monclient: monmap: 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.387 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.374+0000 7f8404da8640 1 librados: starting msgr at 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.374+0000 7f8404da8640 1 librados: starting objecter 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.374+0000 7f8404da8640 1 Processor -- start 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.374+0000 7f8404da8640 1 -- start start 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.374+0000 7f8404da8640 1 librados: setting wanted keys 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.374+0000 7f8404da8640 1 librados: calling monclient init 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.374+0000 7f8404da8640 10 monclient: init 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f84000700f0 0x7f840013dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 10 monclient(hunting): picked mon.noname-b con 0x7f84000700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f84000c7fe0 0x7f84001f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.388 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 10 monclient(hunting): picked mon.noname-c con 0x7f84000c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f84001536f0 0x7f84001f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 10 monclient(hunting): picked mon.noname-a con 0x7f84001536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8400074c00 con 0x7f84001536f0 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f84000c5680 con 0x7f84000700f0 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8400074d80 con 0x7f84000c7fe0 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f8404da8640 10 monclient(hunting): authenticate will time out at 1370.944086s 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fed76640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f84001536f0 0x7f84001f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.389 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fe575640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f84000700f0 0x7f840013dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fdd74640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f84000c7fe0 0x7f84001f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fe575640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f84000700f0 0x7f840013dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43514/0 (socket says 10.20.193.16:43514) 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fe575640 1 -- 10.20.193.16:0/252137671 learned_addr learned my addr 10.20.193.16:0/252137671 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fe575640 10 monclient(hunting): get_auth_request con 0x7f84000700f0 auth_method 0 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fe575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fe575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fe575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fdd74640 10 monclient(hunting): get_auth_request con 0x7f84000c7fe0 auth_method 0 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fdd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fdd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fdd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.390 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fed76640 10 monclient(hunting): get_auth_request con 0x7f84001536f0 auth_method 0 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fe575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fe575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fe575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fdd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fdd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fdd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.375+0000 7f83fed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.391 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.377+0000 7f83fdd74640 10 monclient(hunting): handle_auth_done global_id 5270 payload 824 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.377+0000 7f83fdd74640 1 -- 10.20.193.16:0/252137671 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f84000700f0 msgr2=0x7f840013dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.377+0000 7f83fdd74640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f84000700f0 0x7f840013dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.377+0000 7f83fdd74640 1 -- 10.20.193.16:0/252137671 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f84001536f0 msgr2=0x7f84001f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.377+0000 7f83fdd74640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f84001536f0 0x7f84001f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.377+0000 7f83fdd74640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.377+0000 7f83fdd74640 1 monclient: found mon.noname-c 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.377+0000 7f83fdd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.377+0000 7f83fdd74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.377+0000 7f83fdd74640 1 -- 10.20.193.16:0/252137671 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840013e210 con 0x7f84000c7fe0 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.378+0000 7f83fdd74640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f84000c7fe0 0x7f84001f4e80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c460 tx=0x7f83e800c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.392 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.378+0000 7f83e6ffd640 10 client.?.objecter ms_handle_connect 0x7f84000c7fe0 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.378+0000 7f83e6ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f83e6ffd640 1 -- 10.20.193.16:0/252137671 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e80052d0 con 0x7f84000c7fe0 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f83e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f83e6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f83e6ffd640 10 monclient: dump: 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.393 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f83e6ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f83e6ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f83e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 5 monclient: authenticate success, global_id 5270 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 10 monclient: _renew_subs 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 1 -- 10.20.193.16:0/252137671 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840013e4f0 con 0x7f84000c7fe0 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 10 client.5270.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 10 monclient: _renew_subs 2026-02-17T06:01:58.394 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 1 -- 10.20.193.16:0/252137671 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840013eab0 con 0x7f84000c7fe0 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 1 librados: init done 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.379+0000 7f8404da8640 1 -- 10.20.193.16:0/252137671 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84000704d0 con 0x7f84000c7fe0 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.380+0000 7f83e6ffd640 1 -- 10.20.193.16:0/252137671 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e8005470 con 0x7f84000c7fe0 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.380+0000 7f83e6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.380+0000 7f83e6ffd640 1 -- 10.20.193.16:0/252137671 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e80038f0 con 0x7f84000c7fe0 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.380+0000 7f83e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.380+0000 7f83e6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.380+0000 7f83fcd72640 4 set_mon_vals no callback set 2026-02-17T06:01:58.395 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.380+0000 7f83e6ffd640 10 monclient: dump: 2026-02-17T06:01:58.396 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.396 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.396 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.396 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.396 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.396 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.396 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.396 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.396 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.396 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.487+0000 7f83e6ffd640 1 -- 10.20.193.16:0/252137671 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f83e8003a90 con 0x7f84000c7fe0 2026-02-17T06:01:58.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.487+0000 7f83e6ffd640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83cc079140 0x7f83cc07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.487+0000 7f83fe575640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83cc079140 0x7f83cc07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.487+0000 7f83fe575640 10 monclient: get_auth_request con 0x7f83cc079140 auth_method 0 2026-02-17T06:01:58.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.487+0000 7f83fe575640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83cc079140 0x7f83cc07b600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f83f40098a0 tx=0x7f83f4033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.488 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.487+0000 7f83e6ffd640 10 client.5270.objecter ms_handle_connect 0x7f83cc079140 2026-02-17T06:01:58.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.488+0000 7f83e6ffd640 1 -- 10.20.193.16:0/252137671 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f83e8097b60 con 0x7f84000c7fe0 2026-02-17T06:01:58.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.488+0000 7f83e6ffd640 10 client.5270.objecter ms_dispatch2 0x7f8400000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:58.489 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.488+0000 7f83e6ffd640 3 client.5270.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:58.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.488+0000 7f83e6ffd640 3 client.5270.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:58.490 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.488+0000 7f83e6ffd640 20 client.5270.objecter dump_active .. 0 homeless 2026-02-17T06:01:58.495 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.495+0000 7f83e6ffd640 1 -- 10.20.193.16:0/252137671 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f83e8062190 con 0x7f84000c7fe0 2026-02-17T06:01:58.495 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.495+0000 7f83e6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.495 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.495+0000 7f83e6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:58.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.578+0000 7f8404da8640 1 -- 10.20.193.16:0/252137671 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "13", "target": ["mon-mgr", ""]}) -- 0x7f84001573f0 con 0x7f83cc079140 2026-02-17T06:01:58.579 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.578+0000 7f83e6ffd640 1 -- 10.20.193.16:0/252137671 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 13 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7f84001573f0 con 0x7f83cc079140 2026-02-17T06:01:58.579 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 13 to deep-scrub 2026-02-17T06:01:58.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 10 librados: watch_flush enter 2026-02-17T06:01:58.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 10 librados: watch_flush exit 2026-02-17T06:01:58.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 20 client.5270.objecter shutdown clearing up homeless session... 2026-02-17T06:01:58.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 10 client.5270.objecter successfully canceled tick 2026-02-17T06:01:58.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 -- 10.20.193.16:0/252137671 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83cc079140 msgr2=0x7f83cc07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83cc079140 0x7f83cc07b600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f83f40098a0 tx=0x7f83f4033000 comp rx=0 tx=0).stop 2026-02-17T06:01:58.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 10 monclient: shutdown 2026-02-17T06:01:58.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.581 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 -- 10.20.193.16:0/252137671 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f84000c7fe0 msgr2=0x7f84001f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f84000c7fe0 0x7f84001f4e80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c460 tx=0x7f83e800c930 comp rx=0 tx=0).stop 2026-02-17T06:01:58.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 -- 10.20.193.16:0/252137671 shutdown_connections 2026-02-17T06:01:58.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83cc079140 0x7f83cc07b600 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f84001536f0 0x7f84001f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f84000c7fe0 0x7f84001f4e80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 --2- 10.20.193.16:0/252137671 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f84000700f0 0x7f840013dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 -- 10.20.193.16:0/252137671 >> 10.20.193.16:0/252137671 conn(0x7f84000cd770 msgr2=0x7f8400153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 -- 10.20.193.16:0/252137671 shutdown_connections 2026-02-17T06:01:58.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 -- 10.20.193.16:0/252137671 wait complete. 2026-02-17T06:01:58.582 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.579+0000 7f8404da8640 1 librados: shutdown 2026-02-17T06:01:58.588 INFO:tasks.ceph:Scrubbing osd.2 2026-02-17T06:01:58.588 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:58.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.657+0000 7f6921161640 10 monclient: get_monmap_and_config 2026-02-17T06:01:58.658 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.657+0000 7f6921161640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.657+0000 7f6921161640 10 monclient: monmap: 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:58.658706+0000 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:58.658706+0000 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:58.659 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:58.660 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:58.660 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:58.660 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 1 Processor -- start 2026-02-17T06:01:58.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 1 -- start start 2026-02-17T06:01:58.660 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 10 monclient: init 2026-02-17T06:01:58.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 10 monclient: _add_conns ranks=[2,3,0,1,4,5] 2026-02-17T06:01:58.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f691c0700f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:58.662 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f691c0c86f0 0x7f691c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 10 monclient(hunting): picked mon.noname-a con 0x7f691c0c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f691c1536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f691c0c4820 con 0x7f691c1536f0 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f691c0c4b00 con 0x7f691c0700f0 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f691c0c4de0 con 0x7f691c0c86f0 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.658+0000 7f6921161640 10 monclient(hunting): authenticate will time out at 1371.227688s 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.659+0000 7f691a575640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f691c0c86f0 0x7f691c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.659+0000 7f691ad76640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f691c0700f0 0x7f691c0c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:51322/0 (socket says 10.20.193.16:51322) 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.659+0000 7f691ad76640 1 -- 10.20.193.16:0/4212637689 learned_addr learned my addr 10.20.193.16:0/4212637689 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:58.663 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.659+0000 7f691a575640 10 monclient(hunting): get_auth_request con 0x7f691c0c86f0 auth_method 0 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.659+0000 7f691a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.659+0000 7f691a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.659+0000 7f691a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.659+0000 7f691a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.659+0000 7f691a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.659+0000 7f691a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f6919d74640 1 -- 10.20.193.16:0/4212637689 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2329624355 0 0) 0x7f691c0c4b00 con 0x7f691c0700f0 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f6919d74640 10 monclient(hunting): _negotiate 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f6919d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f6919d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f6919d74640 10 monclient(hunting): my global_id is 5273 2026-02-17T06:01:58.664 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f6919d74640 1 -- 10.20.193.16:0/4212637689 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6908003620 con 0x7f691c0700f0 2026-02-17T06:01:58.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f691a575640 10 monclient(hunting): handle_auth_done global_id 5130 payload 274 2026-02-17T06:01:58.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f691a575640 1 -- 10.20.193.16:0/4212637689 >> v1:10.20.193.148:6789/0 conn(0x7f691c0700f0 legacy=0x7f691c0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f691a575640 1 -- 10.20.193.16:0/4212637689 >> v1:10.20.193.16:6789/0 conn(0x7f691c1536f0 legacy=0x7f691c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f691a575640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f691a575640 1 monclient: found mon.noname-a 2026-02-17T06:01:58.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f691a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f691a575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:58.665 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.660+0000 7f691a575640 1 -- 10.20.193.16:0/4212637689 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c0c50c0 con 0x7f691c0c86f0 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.663+0000 7f691a575640 1 --2- 10.20.193.16:0/4212637689 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f691c0c86f0 0x7f691c0704d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69040098a0 tx=0x7f690402edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f67d25134a2e1ed server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 1 -- 10.20.193.16:0/4212637689 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690403c070 con 0x7f691c0c86f0 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 10 monclient: dump: 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.669 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 1 -- 10.20.193.16:0/4212637689 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690402fae0 con 0x7f691c0c86f0 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6921161640 5 monclient: authenticate success, global_id 5130 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 1 -- 10.20.193.16:0/4212637689 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904037440 con 0x7f691c0c86f0 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6921161640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6921161640 4 set_mon_vals no callback set 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.664+0000 7f6919d74640 10 monclient: dump: 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 10 monclient: shutdown 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 -- 10.20.193.16:0/4212637689 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f691c0c86f0 msgr2=0x7f691c0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 --2- 10.20.193.16:0/4212637689 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f691c0c86f0 0x7f691c0704d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69040098a0 tx=0x7f690402edd0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 -- 10.20.193.16:0/4212637689 shutdown_connections 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 --2- 10.20.193.16:0/4212637689 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f691c0c86f0 0x7f691c0704d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 -- 10.20.193.16:0/4212637689 >> 10.20.193.16:0/4212637689 conn(0x7f691c0cd770 msgr2=0x7f691c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 -- 10.20.193.16:0/4212637689 shutdown_connections 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 -- 10.20.193.16:0/4212637689 wait complete. 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 10 monclient: monmap: 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:58.670 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 librados: starting msgr at 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 librados: starting objecter 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 Processor -- start 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 -- start start 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 librados: setting wanted keys 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 1 librados: calling monclient init 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.665+0000 7f6921161640 10 monclient: init 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f691c0700f0 0x7f691c1f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 10 monclient(hunting): picked mon.noname-b con 0x7f691c0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f691c0c86f0 0x7f691c1f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 10 monclient(hunting): picked mon.noname-c con 0x7f691c0c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f691c1536f0 0x7f691c0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 10 monclient(hunting): picked mon.noname-a con 0x7f691c1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f691c074c00 con 0x7f691c1536f0 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f691c0c5680 con 0x7f691c0700f0 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f691c074d80 con 0x7f691c0c86f0 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f6921161640 10 monclient(hunting): authenticate will time out at 1371.235294s 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691b577640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f691c1536f0 0x7f691c0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691ad76640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f691c0700f0 0x7f691c1f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691b577640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f691c1536f0 0x7f691c0cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33884/0 (socket says 10.20.193.16:33884) 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691b577640 1 -- 10.20.193.16:0/2048066642 learned_addr learned my addr 10.20.193.16:0/2048066642 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691b577640 10 monclient(hunting): get_auth_request con 0x7f691c1536f0 auth_method 0 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691b577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691a575640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f691c0c86f0 0x7f691c1f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691ad76640 10 monclient(hunting): get_auth_request con 0x7f691c0700f0 auth_method 0 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.666+0000 7f691ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691b577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691b577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691b577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691a575640 10 monclient(hunting): get_auth_request con 0x7f691c0c86f0 auth_method 0 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.667+0000 7f691a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.668+0000 7f691b577640 10 monclient(hunting): handle_auth_done global_id 5136 payload 824 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.668+0000 7f691b577640 1 -- 10.20.193.16:0/2048066642 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f691c0c86f0 msgr2=0x7f691c1f1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.668+0000 7f691b577640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f691c0c86f0 0x7f691c1f1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.668+0000 7f691b577640 1 -- 10.20.193.16:0/2048066642 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f691c0700f0 msgr2=0x7f691c1f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.668+0000 7f691b577640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f691c0700f0 0x7f691c1f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.668+0000 7f691b577640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.668+0000 7f691b577640 1 monclient: found mon.noname-a 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.668+0000 7f691b577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.668+0000 7f691b577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.668+0000 7f691b577640 1 -- 10.20.193.16:0/2048066642 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c1f1cc0 con 0x7f691c1536f0 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.669+0000 7f691b577640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f691c1536f0 0x7f691c0cb3f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f690c00cba0 tx=0x7f690c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.669+0000 7f68fb7fe640 10 client.?.objecter ms_handle_connect 0x7f691c1536f0 2026-02-17T06:01:58.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.669+0000 7f68fb7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.670+0000 7f68fb7fe640 1 -- 10.20.193.16:0/2048066642 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c00ad50 con 0x7f691c1536f0 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.670+0000 7f68fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.670+0000 7f68fb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.670+0000 7f68fb7fe640 10 monclient: dump: 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.670+0000 7f68fb7fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.670+0000 7f68fb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.670+0000 7f68fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 5 monclient: authenticate success, global_id 5136 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.670+0000 7f68fb7fe640 1 -- 10.20.193.16:0/2048066642 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c003a70 con 0x7f691c1536f0 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 10 monclient: _renew_subs 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c1f1fa0 con 0x7f691c1536f0 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f68fb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f68fb7fe640 1 -- 10.20.193.16:0/2048066642 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c0050d0 con 0x7f691c1536f0 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f68fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f68fb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6919573640 4 set_mon_vals no callback set 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f68fb7fe640 10 monclient: dump: 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 10 client.5136.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:58.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 10 monclient: _renew_subs 2026-02-17T06:01:58.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:58.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c073100 con 0x7f691c1536f0 2026-02-17T06:01:58.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 1 librados: init done 2026-02-17T06:01:58.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 10 client.5136.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 15 client.5136.objecter _session_command_op_assign -1 1 2026-02-17T06:01:58.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 20 client.5136.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:58.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 10 client.5136.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:58.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:01:58.674 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.671+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f691c073430 con 0x7f691c1536f0 2026-02-17T06:01:58.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.683+0000 7f68fb7fe640 1 -- 10.20.193.16:0/2048066642 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f690c005270 con 0x7f691c1536f0 2026-02-17T06:01:58.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.683+0000 7f68fb7fe640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f68e8079140 0x7f68e807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.683+0000 7f691ad76640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f68e8079140 0x7f68e807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.683+0000 7f691ad76640 10 monclient: get_auth_request con 0x7f68e8079140 auth_method 0 2026-02-17T06:01:58.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.683+0000 7f691ad76640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f68e8079140 0x7f68e807b600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6910005980 tx=0x7f6910007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.683 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.683+0000 7f68fb7fe640 10 client.5136.objecter ms_handle_connect 0x7f68e8079140 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.684+0000 7f68fb7fe640 1 -- 10.20.193.16:0/2048066642 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f690c0982a0 con 0x7f691c1536f0 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.684+0000 7f68fb7fe640 10 client.5136.objecter ms_dispatch2 0x7f691c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.684+0000 7f68fb7fe640 3 client.5136.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.684+0000 7f68fb7fe640 3 client.5136.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.684+0000 7f68fb7fe640 10 client.5136.objecter checking command 1 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] conn(0x7f68e8081620 0x7f68e8083a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 20 client.5136.objecter _get_session s=0x7f68e8080f50 osd=2 3 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 20 client.5136.objecter put_session s=0x7f68e8080f50 osd=2 3 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 15 client.5136.objecter _session_command_op_remove -1 1 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 20 client.5136.objecter _get_session s=0x7f68e8080f50 osd=2 3 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 20 client.5136.objecter get_session s=0x7f68e8080f50 osd=2 3 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 15 client.5136.objecter _session_command_op_assign 2 1 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 20 client.5136.objecter put_session s=0x7f68e8080f50 osd=2 4 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 10 client.5136.objecter _send_command 1 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 1 -- 10.20.193.16:0/2048066642 --> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f690c004ec0 con 0x7f68e8081620 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 20 client.5136.objecter dump_active .. 0 homeless 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 1 -- 10.20.193.16:0/2048066642 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f690c05ebc0 con 0x7f691c1536f0 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f68fb7fe640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f691a575640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] conn(0x7f68e8081620 0x7f68e8083a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.685 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.685+0000 7f691a575640 10 monclient: get_auth_request con 0x7f68e8081620 auth_method 0 2026-02-17T06:01:58.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.687+0000 7f68fb7fe640 10 client.5136.objecter ms_handle_connect 0x7f68e8081620 2026-02-17T06:01:58.687 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.687+0000 7f691a575640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] conn(0x7f68e8081620 0x7f68e8083a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.697 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.698+0000 7f68fb7fe640 1 -- 10.20.193.16:0/2048066642 <== osd.2 v2:10.20.193.148:6816/1588035471 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f690c004ec0 con 0x7f68e8081620 2026-02-17T06:01:58.697 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.698+0000 7f68fb7fe640 10 client.5136.objecter ms_dispatch2 0x7f691c000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:58.697 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.698+0000 7f68fb7fe640 10 client.5136.objecter _finish_command 1 = system:0 2026-02-17T06:01:58.697 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.698+0000 7f68fb7fe640 20 client.5136.objecter put_session s=0x7f68e8080f50 osd=2 4 2026-02-17T06:01:58.697 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.698+0000 7f68fb7fe640 15 client.5136.objecter _session_command_op_remove 2 1 2026-02-17T06:01:58.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 10 client.5136.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:58.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 15 client.5136.objecter _session_command_op_assign -1 2 2026-02-17T06:01:58.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 20 client.5136.objecter _get_session s=0x7f68e8080f50 osd=2 3 2026-02-17T06:01:58.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 20 client.5136.objecter put_session s=0x7f68e8080f50 osd=2 3 2026-02-17T06:01:58.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 20 client.5136.objecter _get_session s=0x7f68e8080f50 osd=2 3 2026-02-17T06:01:58.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 15 client.5136.objecter _session_command_op_remove -1 2 2026-02-17T06:01:58.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 20 client.5136.objecter get_session s=0x7f68e8080f50 osd=2 3 2026-02-17T06:01:58.703 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 15 client.5136.objecter _session_command_op_assign 2 2 2026-02-17T06:01:58.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 20 client.5136.objecter put_session s=0x7f68e8080f50 osd=2 4 2026-02-17T06:01:58.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 10 client.5136.objecter _send_command 2 2026-02-17T06:01:58.704 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.703+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 --> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f691c0cbb80 con 0x7f68e8081620 2026-02-17T06:01:58.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f68fb7fe640 1 -- 10.20.193.16:0/2048066642 <== osd.2 v2:10.20.193.148:6816/1588035471 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f691c0cbb80 con 0x7f68e8081620 2026-02-17T06:01:58.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f68fb7fe640 10 client.5136.objecter ms_dispatch2 0x7f691c000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:58.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f68fb7fe640 10 client.5136.objecter _finish_command 2 = system:0 2026-02-17T06:01:58.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f68fb7fe640 20 client.5136.objecter put_session s=0x7f68e8080f50 osd=2 4 2026-02-17T06:01:58.710 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f68fb7fe640 15 client.5136.objecter _session_command_op_remove 2 2 2026-02-17T06:01:58.710 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:58.710 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:58.710 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 10 librados: watch_flush enter 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 10 librados: watch_flush exit 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 10 client.5136.objecter close_session for osd.2 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 >> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] conn(0x7f68e8081620 msgr2=0x7f68e8083a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.148:6816/1588035471,v1:10.20.193.148:6817/1588035471] conn(0x7f68e8081620 0x7f68e8083a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 20 client.5136.objecter put_session s=0x7f68e8080f50 osd=2 1 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 20 client.5136.objecter shutdown clearing up homeless session... 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 10 client.5136.objecter successfully canceled tick 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f68e8079140 msgr2=0x7f68e807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f68e8079140 0x7f68e807b600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6910005980 tx=0x7f6910007dd0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 10 monclient: shutdown 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f691c1536f0 msgr2=0x7f691c0cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 --2- 10.20.193.16:0/2048066642 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f691c1536f0 0x7f691c0cb3f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f690c00cba0 tx=0x7f690c00a430 comp rx=0 tx=0).stop 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f691b577640 1 -- 10.20.193.16:0/2048066642 reap_dead start 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 shutdown_connections 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 >> 10.20.193.16:0/2048066642 conn(0x7f691c0cd770 msgr2=0x7f691c0c6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 shutdown_connections 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 -- 10.20.193.16:0/2048066642 wait complete. 2026-02-17T06:01:58.711 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.711+0000 7f6921161640 1 librados: shutdown 2026-02-17T06:01:58.718 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-17T06:01:58.786 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.786+0000 7fcfba67c640 10 monclient: get_monmap_and_config 2026-02-17T06:01:58.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.786+0000 7fcfba67c640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.786+0000 7fcfba67c640 10 monclient: monmap: 2026-02-17T06:01:58.787 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:58.787 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.787 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:58.787425+0000 2026-02-17T06:01:58.787 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:58.787425+0000 2026-02-17T06:01:58.787 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.788 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.788 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:58.788 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:58.788 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:58.788 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:58.788 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:58.788 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:58.788 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.786+0000 7fcfba67c640 1 Processor -- start 2026-02-17T06:01:58.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 1 -- start start 2026-02-17T06:01:58.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 10 monclient: init 2026-02-17T06:01:58.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 10 monclient: _add_conns ranks=[1,4,5,0,3,2] 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fcfb4153a10 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fcfb4153df0 0x7fcfb415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 10 monclient(hunting): picked mon.noname-b con 0x7fcfb4153df0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fcfb415b3b0 0x7fcfb40c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 10 monclient(hunting): picked mon.noname-c con 0x7fcfb415b3b0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfb40691d0 con 0x7fcfb4153a10 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb415a9c0 con 0x7fcfb4153df0 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb415a530 con 0x7fcfb415b3b0 2026-02-17T06:01:58.792 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.787+0000 7fcfba67c640 10 monclient(hunting): authenticate will time out at 1371.356799s 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb37fe640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fcfb4153df0 0x7fcfb415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb8bf2640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fcfb415b3b0 0x7fcfb40c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb3fff640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fcfb4153a10 0x7fcfb4074b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:51318/0 (socket says 10.20.193.16:51318) 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb3fff640 1 -- 10.20.193.16:0/1414468269 learned_addr learned my addr 10.20.193.16:0/1414468269 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb8bf2640 10 monclient(hunting): get_auth_request con 0x7fcfb415b3b0 auth_method 0 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb8bf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb8bf2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb8bf2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb37fe640 10 monclient(hunting): get_auth_request con 0x7fcfb4153df0 auth_method 0 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.793 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb8bf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb8bf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.788+0000 7fcfb8bf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb8bf2640 10 monclient(hunting): handle_auth_done global_id 5279 payload 274 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb8bf2640 1 -- 10.20.193.16:0/1414468269 >> v2:10.20.193.128:3300/0 conn(0x7fcfb4153df0 msgr2=0x7fcfb415ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb8bf2640 1 --2- 10.20.193.16:0/1414468269 >> v2:10.20.193.128:3300/0 conn(0x7fcfb4153df0 0x7fcfb415ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb8bf2640 1 -- 10.20.193.16:0/1414468269 >> v1:10.20.193.128:6789/0 conn(0x7fcfb4153a10 legacy=0x7fcfb4074b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb8bf2640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.794 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb8bf2640 1 monclient: found mon.noname-c 2026-02-17T06:01:58.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb8bf2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb8bf2640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:58.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.789+0000 7fcfb8bf2640 1 -- 10.20.193.16:0/1414468269 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb4159c00 con 0x7fcfb415b3b0 2026-02-17T06:01:58.795 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.791+0000 7fcfb8bf2640 1 --2- 10.20.193.16:0/1414468269 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 0x7fcfb40c7cd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40c1ec0 tx=0x7fcfa802f250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c082fcf8ad6c7f3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.792+0000 7fcfb2ffd640 1 -- 10.20.193.16:0/1414468269 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa803c070 con 0x7fcfb415b3b0 2026-02-17T06:01:58.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.792+0000 7fcfb2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.792+0000 7fcfb2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:58.801 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.792+0000 7fcfb2ffd640 10 monclient: dump: 2026-02-17T06:01:58.801 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.801 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.801 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.801 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.801 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.801 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.792+0000 7fcfb2ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.792+0000 7fcfb2ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.792+0000 7fcfb2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.792+0000 7fcfba67c640 5 monclient: authenticate success, global_id 5279 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.792+0000 7fcfba67c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfb2ffd640 1 -- 10.20.193.16:0/1414468269 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa8037440 con 0x7fcfb415b3b0 2026-02-17T06:01:58.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfb2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfb2ffd640 1 -- 10.20.193.16:0/1414468269 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8037810 con 0x7fcfb415b3b0 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfb2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfb2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfb2ffd640 10 monclient: dump: 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.803 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 4 set_mon_vals no callback set 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 10 monclient: shutdown 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 1 -- 10.20.193.16:0/1414468269 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 msgr2=0x7fcfb40c7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 1 --2- 10.20.193.16:0/1414468269 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 0x7fcfb40c7cd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40c1ec0 tx=0x7fcfa802f250 comp rx=0 tx=0).stop 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 1 -- 10.20.193.16:0/1414468269 shutdown_connections 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 1 --2- 10.20.193.16:0/1414468269 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 0x7fcfb40c7cd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.804 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 1 --2- 10.20.193.16:0/1414468269 >> v2:10.20.193.128:3300/0 conn(0x7fcfb4153df0 0x7fcfb415ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 1 -- 10.20.193.16:0/1414468269 >> 10.20.193.16:0/1414468269 conn(0x7fcfb4073f40 msgr2=0x7fcfb4074350 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 1 -- 10.20.193.16:0/1414468269 shutdown_connections 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.793+0000 7fcfba67c640 1 -- 10.20.193.16:0/1414468269 wait complete. 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient: monmap: 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:58.805 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 librados: starting msgr at 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 librados: starting objecter 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 Processor -- start 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 -- start start 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 librados: setting wanted keys 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 librados: calling monclient init 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient: init 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:58.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcfb4153a10 0x7fcfb4152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient(hunting): picked mon.noname-a con 0x7fcfb4153a10 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcfb4153df0 0x7fcfb41f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient(hunting): picked mon.noname-b con 0x7fcfb4153df0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 0x7fcfb41f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient(hunting): picked mon.noname-c con 0x7fcfb415b3b0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb40c8210 con 0x7fcfb4153a10 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb415a1c0 con 0x7fcfb4153df0 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb40c8390 con 0x7fcfb415b3b0 2026-02-17T06:01:58.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.794+0000 7fcfba67c640 10 monclient(hunting): authenticate will time out at 1371.363712s 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb37fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcfb4153df0 0x7fcfb41f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb8bf2640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 0x7fcfb41f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb8bf2640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 0x7fcfb41f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.148:3300/0 says I am v2:10.20.193.16:53358/0 (socket says 10.20.193.16:53358) 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb37fe640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcfb4153df0 0x7fcfb41f1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43542/0 (socket says 10.20.193.16:43542) 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb8bf2640 1 -- 10.20.193.16:0/664703627 learned_addr learned my addr 10.20.193.16:0/664703627 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb8bf2640 10 monclient(hunting): get_auth_request con 0x7fcfb415b3b0 auth_method 0 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb8bf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb8bf2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb8bf2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.808 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb37fe640 10 monclient(hunting): get_auth_request con 0x7fcfb4153df0 auth_method 0 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb8bf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb8bf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb8bf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb3fff640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcfb4153a10 0x7fcfb4152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb3fff640 10 monclient(hunting): get_auth_request con 0x7fcfb4153a10 auth_method 0 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.809 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.795+0000 7fcfb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.796+0000 7fcfb8bf2640 10 monclient(hunting): handle_auth_done global_id 5282 payload 824 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.796+0000 7fcfb8bf2640 1 -- 10.20.193.16:0/664703627 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcfb4153df0 msgr2=0x7fcfb41f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.796+0000 7fcfb8bf2640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcfb4153df0 0x7fcfb41f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.796+0000 7fcfb8bf2640 1 -- 10.20.193.16:0/664703627 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcfb4153a10 msgr2=0x7fcfb4152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.796+0000 7fcfb8bf2640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcfb4153a10 0x7fcfb4152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.796+0000 7fcfb8bf2640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.796+0000 7fcfb8bf2640 1 monclient: found mon.noname-c 2026-02-17T06:01:58.810 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.796+0000 7fcfb8bf2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.796+0000 7fcfb8bf2640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.796+0000 7fcfb8bf2640 1 -- 10.20.193.16:0/664703627 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb41f4cf0 con 0x7fcfb415b3b0 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.797+0000 7fcfb8bf2640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 0x7fcfb41f1cd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcfa803a040 tx=0x7fcfa8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.797+0000 7fcfb0ff9640 10 client.?.objecter ms_handle_connect 0x7fcfb415b3b0 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.797+0000 7fcfb0ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 1 -- 10.20.193.16:0/664703627 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa803c050 con 0x7fcfb415b3b0 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 10 monclient: dump: 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.811 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 1 -- 10.20.193.16:0/664703627 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa8048070 con 0x7fcfb415b3b0 2026-02-17T06:01:58.812 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 1 -- 10.20.193.16:0/664703627 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8035a90 con 0x7fcfb415b3b0 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb27fc640 4 set_mon_vals no callback set 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfb0ff9640 10 monclient: dump: 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:58.813 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfba67c640 5 monclient: authenticate success, global_id 5282 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfba67c640 10 monclient: _renew_subs 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfba67c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfba67c640 1 -- 10.20.193.16:0/664703627 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb41f4fd0 con 0x7fcfb415b3b0 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfba67c640 10 client.5282.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfba67c640 10 monclient: _renew_subs 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfba67c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfba67c640 1 -- 10.20.193.16:0/664703627 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41f5590 con 0x7fcfb415b3b0 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfba67c640 1 librados: init done 2026-02-17T06:01:58.814 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.799+0000 7fcfba67c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.800+0000 7fcfba67c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.800+0000 7fcfba67c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:58.815 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.800+0000 7fcfba67c640 1 -- 10.20.193.16:0/664703627 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb41f5970 con 0x7fcfb415b3b0 2026-02-17T06:01:58.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.810+0000 7fcfb0ff9640 1 -- 10.20.193.16:0/664703627 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fcfa8035c30 con 0x7fcfb415b3b0 2026-02-17T06:01:58.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.811+0000 7fcfb0ff9640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcf84079190 0x7fcf8407b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.811+0000 7fcfb3fff640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcf84079190 0x7fcf8407b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.816 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.811+0000 7fcfb3fff640 10 monclient: get_auth_request con 0x7fcf84079190 auth_method 0 2026-02-17T06:01:58.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.811+0000 7fcfb3fff640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcf84079190 0x7fcf8407b650 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0002770 tx=0x7fcfa0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.811+0000 7fcfb0ff9640 10 client.5282.objecter ms_handle_connect 0x7fcf84079190 2026-02-17T06:01:58.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.812+0000 7fcfb0ff9640 1 -- 10.20.193.16:0/664703627 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fcfa80847d0 con 0x7fcfb415b3b0 2026-02-17T06:01:58.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.812+0000 7fcfb0ff9640 10 client.5282.objecter ms_dispatch2 0x7fcfb4000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:58.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.812+0000 7fcfb0ff9640 3 client.5282.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:58.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.812+0000 7fcfb0ff9640 3 client.5282.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:58.817 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.812+0000 7fcfb0ff9640 20 client.5282.objecter dump_active .. 0 homeless 2026-02-17T06:01:58.914 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.914+0000 7fcfb0ff9640 1 -- 10.20.193.16:0/664703627 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcfb41f5970 con 0x7fcfb415b3b0 2026-02-17T06:01:58.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.914+0000 7fcfb0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.915 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.914+0000 7fcfb0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:58.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.997+0000 7fcfba67c640 1 -- 10.20.193.16:0/664703627 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fcfb415ae70 con 0x7fcf84079190 2026-02-17T06:01:58.998 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.997+0000 7fcfb0ff9640 1 -- 10.20.193.16:0/664703627 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcfb415ae70 con 0x7fcf84079190 2026-02-17T06:01:58.998 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 2 to deep-scrub 2026-02-17T06:01:59.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 10 librados: watch_flush enter 2026-02-17T06:01:59.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 10 librados: watch_flush exit 2026-02-17T06:01:59.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 20 client.5282.objecter shutdown clearing up homeless session... 2026-02-17T06:01:59.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 10 client.5282.objecter successfully canceled tick 2026-02-17T06:01:59.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 -- 10.20.193.16:0/664703627 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcf84079190 msgr2=0x7fcf8407b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcf84079190 0x7fcf8407b650 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0002770 tx=0x7fcfa0009040 comp rx=0 tx=0).stop 2026-02-17T06:01:59.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 10 monclient: shutdown 2026-02-17T06:01:59.000 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 -- 10.20.193.16:0/664703627 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 msgr2=0x7fcfb41f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 0x7fcfb41f1cd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcfa803a040 tx=0x7fcfa8004b40 comp rx=0 tx=0).stop 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 -- 10.20.193.16:0/664703627 shutdown_connections 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fcf84079190 0x7fcf8407b650 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fcfb4153df0 0x7fcfb41f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fcfb415b3b0 0x7fcfb41f1cd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 --2- 10.20.193.16:0/664703627 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fcfb4153a10 0x7fcfb4152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 -- 10.20.193.16:0/664703627 >> 10.20.193.16:0/664703627 conn(0x7fcfb4073f40 msgr2=0x7fcfb415c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 -- 10.20.193.16:0/664703627 shutdown_connections 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 -- 10.20.193.16:0/664703627 wait complete. 2026-02-17T06:01:59.001 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:58.998+0000 7fcfba67c640 1 librados: shutdown 2026-02-17T06:01:59.007 INFO:tasks.ceph:Scrubbing osd.6 2026-02-17T06:01:59.007 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:59.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.077+0000 7fbe935c1640 10 monclient: get_monmap_and_config 2026-02-17T06:01:59.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.077+0000 7fbe935c1640 10 monclient: build_initial_monmap 2026-02-17T06:01:59.078 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.077+0000 7fbe935c1640 10 monclient: monmap: 2026-02-17T06:01:59.078 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:59.078 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.078 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:59.078541+0000 2026-02-17T06:01:59.078 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:59.078541+0000 2026-02-17T06:01:59.078 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:59.079 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.079 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:59.079 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:59.079 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:59.079 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:59.079 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:59.079 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:59.079 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 1 Processor -- start 2026-02-17T06:01:59.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 1 -- start start 2026-02-17T06:01:59.082 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 10 monclient: init 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 10 monclient: _add_conns ranks=[5,0,3,4,2,1] 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fbe8c0700f0 0x7fbe8c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 10 monclient(hunting): picked mon.noname-c con 0x7fbe8c0700f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbe8c0c7fe0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fbe8c1536f0 0x7fbe8c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 10 monclient(hunting): picked mon.noname-a con 0x7fbe8c1536f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe8c0c4820 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8c0c4b00 con 0x7fbe8c1536f0 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.083 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8c0c4de0 con 0x7fbe8c0700f0 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.078+0000 7fbe935c1640 10 monclient(hunting): authenticate will time out at 1371.647620s 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91336640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7fbe8c0700f0 0x7fbe8c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91b37640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fbe8c1536f0 0x7fbe8c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe90b35640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7fbe8c0c7fe0 0x7fbe8c0c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:41918/0 (socket says 10.20.193.16:41918) 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe90b35640 1 -- 10.20.193.16:0/2436778600 learned_addr learned my addr 10.20.193.16:0/2436778600 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91336640 10 monclient(hunting): get_auth_request con 0x7fbe8c0700f0 auth_method 0 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91336640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91336640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91336640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91b37640 10 monclient(hunting): get_auth_request con 0x7fbe8c1536f0 auth_method 0 2026-02-17T06:01:59.084 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91b37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91b37640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91b37640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91336640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91336640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.079+0000 7fbe91336640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91b37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91b37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91b37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91336640 10 monclient(hunting): handle_auth_done global_id 5285 payload 274 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91336640 1 -- 10.20.193.16:0/2436778600 >> v2:10.20.193.16:3300/0 conn(0x7fbe8c1536f0 msgr2=0x7fbe8c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91336640 1 --2- 10.20.193.16:0/2436778600 >> v2:10.20.193.16:3300/0 conn(0x7fbe8c1536f0 0x7fbe8c153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91336640 1 -- 10.20.193.16:0/2436778600 >> v1:10.20.193.16:6789/0 conn(0x7fbe8c0c7fe0 legacy=0x7fbe8c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.085 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91336640 10 monclient: _finish_hunting 0 2026-02-17T06:01:59.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91336640 1 monclient: found mon.noname-c 2026-02-17T06:01:59.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91336640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:59.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91336640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.080+0000 7fbe91336640 1 -- 10.20.193.16:0/2436778600 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c0c50c0 con 0x7fbe8c0700f0 2026-02-17T06:01:59.086 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.081+0000 7fbe91336640 1 --2- 10.20.193.16:0/2436778600 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbe8c0700f0 0x7fbe8c0704d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c070b90 tx=0x7fbe7c02ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c236aaca1f128b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.082+0000 7fbe7bfff640 1 -- 10.20.193.16:0/2436778600 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c03c070 con 0x7fbe8c0700f0 2026-02-17T06:01:59.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.082+0000 7fbe7bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.082+0000 7fbe7bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:59.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.082+0000 7fbe7bfff640 10 monclient: dump: 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.082+0000 7fbe7bfff640 10 monclient: _finish_auth 0 2026-02-17T06:01:59.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.082+0000 7fbe7bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.082+0000 7fbe7bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.082+0000 7fbe935c1640 5 monclient: authenticate success, global_id 5285 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.082+0000 7fbe935c1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe7bfff640 1 -- 10.20.193.16:0/2436778600 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7c02fa60 con 0x7fbe8c0700f0 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe7bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe7bfff640 1 -- 10.20.193.16:0/2436778600 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c02fe30 con 0x7fbe8c0700f0 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe7bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe7bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe7bfff640 10 monclient: dump: 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.094 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 4 set_mon_vals no callback set 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 10 monclient: shutdown 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 1 -- 10.20.193.16:0/2436778600 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbe8c0700f0 msgr2=0x7fbe8c0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.095 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 1 --2- 10.20.193.16:0/2436778600 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbe8c0700f0 0x7fbe8c0704d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c070b90 tx=0x7fbe7c02ed50 comp rx=0 tx=0).stop 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 1 -- 10.20.193.16:0/2436778600 shutdown_connections 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 1 --2- 10.20.193.16:0/2436778600 >> v2:10.20.193.16:3300/0 conn(0x7fbe8c1536f0 0x7fbe8c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 1 --2- 10.20.193.16:0/2436778600 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbe8c0700f0 0x7fbe8c0704d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 1 -- 10.20.193.16:0/2436778600 >> 10.20.193.16:0/2436778600 conn(0x7fbe8c0cd770 msgr2=0x7fbe8c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 1 -- 10.20.193.16:0/2436778600 shutdown_connections 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.083+0000 7fbe935c1640 1 -- 10.20.193.16:0/2436778600 wait complete. 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 10 monclient: build_initial_monmap 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 10 monclient: monmap: 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:59.096 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 1 librados: starting msgr at 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 1 librados: starting objecter 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 1 Processor -- start 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 1 -- start start 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 1 librados: setting wanted keys 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 1 librados: calling monclient init 2026-02-17T06:01:59.097 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 10 monclient: init 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbe8c0700f0 0x7fbe8c13b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 10 monclient(hunting): picked mon.noname-b con 0x7fbe8c0700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbe8c0c7fe0 0x7fbe8c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 10 monclient(hunting): picked mon.noname-c con 0x7fbe8c0c7fe0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.084+0000 7fbe935c1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbe8c1536f0 0x7fbe8c1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe935c1640 10 monclient(hunting): picked mon.noname-a con 0x7fbe8c1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe935c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe935c1640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c154060 con 0x7fbe8c1536f0 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe935c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe935c1640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c0c5680 con 0x7fbe8c0700f0 2026-02-17T06:01:59.098 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe935c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe935c1640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c074c00 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe935c1640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe935c1640 10 monclient(hunting): authenticate will time out at 1371.654015s 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91b37640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbe8c1536f0 0x7fbe8c1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe90b35640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbe8c0c7fe0 0x7fbe8c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91b37640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbe8c1536f0 0x7fbe8c1562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33918/0 (socket says 10.20.193.16:33918) 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91b37640 1 -- 10.20.193.16:0/1368655899 learned_addr learned my addr 10.20.193.16:0/1368655899 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91b37640 10 monclient(hunting): get_auth_request con 0x7fbe8c1536f0 auth_method 0 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91b37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91b37640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91b37640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.099 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91336640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbe8c0700f0 0x7fbe8c13b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe90b35640 10 monclient(hunting): get_auth_request con 0x7fbe8c0c7fe0 auth_method 0 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe90b35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe90b35640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe90b35640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91b37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91b37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91b37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91336640 10 monclient(hunting): get_auth_request con 0x7fbe8c0700f0 auth_method 0 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91336640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91336640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe91336640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.100 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe90b35640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe90b35640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.085+0000 7fbe90b35640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.086+0000 7fbe91336640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.086+0000 7fbe91336640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.086+0000 7fbe91336640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe90b35640 10 monclient(hunting): handle_auth_done global_id 5288 payload 824 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe90b35640 1 -- 10.20.193.16:0/1368655899 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbe8c0700f0 msgr2=0x7fbe8c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe90b35640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fbe8c0700f0 0x7fbe8c13b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe90b35640 1 -- 10.20.193.16:0/1368655899 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbe8c1536f0 msgr2=0x7fbe8c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe90b35640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbe8c1536f0 0x7fbe8c1562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe90b35640 10 monclient: _finish_hunting 0 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe90b35640 1 monclient: found mon.noname-c 2026-02-17T06:01:59.101 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe90b35640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe90b35640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe90b35640 1 -- 10.20.193.16:0/1368655899 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c13b660 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.087+0000 7fbe91b37640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fbe8c1536f0 0x7fbe8c1562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.088+0000 7fbe90b35640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbe8c0c7fe0 0x7fbe8c155d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000c410 tx=0x7fbe8000c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.088+0000 7fbe79ffb640 10 client.?.objecter ms_handle_connect 0x7fbe8c0c7fe0 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.088+0000 7fbe79ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 1 -- 10.20.193.16:0/1368655899 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80017070 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 10 monclient: dump: 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.102 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 1 -- 10.20.193.16:0/1368655899 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe80005de0 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.103 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe935c1640 5 monclient: authenticate success, global_id 5288 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe935c1640 10 monclient: _renew_subs 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe935c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c1f5830 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 1 -- 10.20.193.16:0/1368655899 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe800072d0 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe79ffb640 10 monclient: dump: 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.104 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe7b7fe640 4 set_mon_vals no callback set 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe935c1640 10 client.5288.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe935c1640 10 monclient: _renew_subs 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe935c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c1f1810 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.089+0000 7fbe935c1640 1 librados: init done 2026-02-17T06:01:59.105 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.090+0000 7fbe935c1640 10 client.5288.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:59.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.090+0000 7fbe935c1640 15 client.5288.objecter _session_command_op_assign -1 1 2026-02-17T06:01:59.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.090+0000 7fbe935c1640 20 client.5288.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:59.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.090+0000 7fbe935c1640 10 client.5288.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:59.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.090+0000 7fbe935c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.106 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.090+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe8c0704d0 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.101+0000 7fbe79ffb640 1 -- 10.20.193.16:0/1368655899 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fbe80007470 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.102+0000 7fbe79ffb640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbe5c079190 0x7fbe5c07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.102+0000 7fbe91336640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbe5c079190 0x7fbe5c07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.102+0000 7fbe91336640 10 monclient: get_auth_request con 0x7fbe5c079190 auth_method 0 2026-02-17T06:01:59.108 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.102+0000 7fbe79ffb640 10 client.5288.objecter ms_handle_connect 0x7fbe5c079190 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.102+0000 7fbe91336640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbe5c079190 0x7fbe5c07b650 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c039000 tx=0x7fbe7c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 1 -- 10.20.193.16:0/1368655899 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fbe80007790 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 10 client.5288.objecter ms_dispatch2 0x7fbe8c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 3 client.5288.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 3 client.5288.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 10 client.5288.objecter checking command 1 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] conn(0x7fbe5c0816d0 0x7fbe5c083af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 20 client.5288.objecter _get_session s=0x7fbe5c081000 osd=6 3 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 20 client.5288.objecter put_session s=0x7fbe5c081000 osd=6 3 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 15 client.5288.objecter _session_command_op_remove -1 1 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 20 client.5288.objecter _get_session s=0x7fbe5c081000 osd=6 3 2026-02-17T06:01:59.109 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 20 client.5288.objecter get_session s=0x7fbe5c081000 osd=6 3 2026-02-17T06:01:59.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 15 client.5288.objecter _session_command_op_assign 6 1 2026-02-17T06:01:59.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 20 client.5288.objecter put_session s=0x7fbe5c081000 osd=6 4 2026-02-17T06:01:59.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 10 client.5288.objecter _send_command 1 2026-02-17T06:01:59.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 1 -- 10.20.193.16:0/1368655899 --> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe80017e70 con 0x7fbe5c0816d0 2026-02-17T06:01:59.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 20 client.5288.objecter dump_active .. 0 homeless 2026-02-17T06:01:59.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 1 -- 10.20.193.16:0/1368655899 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbe8005dd10 con 0x7fbe8c0c7fe0 2026-02-17T06:01:59.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe79ffb640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:59.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.103+0000 7fbe91b37640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] conn(0x7fbe5c0816d0 0x7fbe5c083af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.104+0000 7fbe91b37640 10 monclient: get_auth_request con 0x7fbe5c0816d0 auth_method 0 2026-02-17T06:01:59.110 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.105+0000 7fbe91b37640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] conn(0x7fbe5c0816d0 0x7fbe5c083af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.111 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.105+0000 7fbe79ffb640 10 client.5288.objecter ms_handle_connect 0x7fbe5c0816d0 2026-02-17T06:01:59.115 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.115+0000 7fbe79ffb640 1 -- 10.20.193.16:0/1368655899 <== osd.6 v2:10.20.193.148:6824/2491860138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fbe80017e70 con 0x7fbe5c0816d0 2026-02-17T06:01:59.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.115+0000 7fbe79ffb640 10 client.5288.objecter ms_dispatch2 0x7fbe8c000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:59.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.115+0000 7fbe79ffb640 10 client.5288.objecter _finish_command 1 = system:0 2026-02-17T06:01:59.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.115+0000 7fbe79ffb640 20 client.5288.objecter put_session s=0x7fbe5c081000 osd=6 4 2026-02-17T06:01:59.116 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.115+0000 7fbe79ffb640 15 client.5288.objecter _session_command_op_remove 6 1 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 10 client.5288.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 15 client.5288.objecter _session_command_op_assign -1 2 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 20 client.5288.objecter _get_session s=0x7fbe5c081000 osd=6 3 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 20 client.5288.objecter put_session s=0x7fbe5c081000 osd=6 3 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 20 client.5288.objecter _get_session s=0x7fbe5c081000 osd=6 3 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 15 client.5288.objecter _session_command_op_remove -1 2 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 20 client.5288.objecter get_session s=0x7fbe5c081000 osd=6 3 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 15 client.5288.objecter _session_command_op_assign 6 2 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 20 client.5288.objecter put_session s=0x7fbe5c081000 osd=6 4 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 10 client.5288.objecter _send_command 2 2026-02-17T06:01:59.122 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.121+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 --> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbe8c1573f0 con 0x7fbe5c0816d0 2026-02-17T06:01:59.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe79ffb640 1 -- 10.20.193.16:0/1368655899 <== osd.6 v2:10.20.193.148:6824/2491860138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbe8c1573f0 con 0x7fbe5c0816d0 2026-02-17T06:01:59.128 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe79ffb640 10 client.5288.objecter ms_dispatch2 0x7fbe8c000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:59.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe79ffb640 10 client.5288.objecter _finish_command 2 = system:0 2026-02-17T06:01:59.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe79ffb640 20 client.5288.objecter put_session s=0x7fbe5c081000 osd=6 4 2026-02-17T06:01:59.129 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe79ffb640 15 client.5288.objecter _session_command_op_remove 6 2 2026-02-17T06:01:59.129 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:59.129 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:59.129 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:59.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 10 librados: watch_flush enter 2026-02-17T06:01:59.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 10 librados: watch_flush exit 2026-02-17T06:01:59.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 10 client.5288.objecter close_session for osd.6 2026-02-17T06:01:59.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 >> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] conn(0x7fbe5c0816d0 msgr2=0x7fbe5c083af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.148:6824/2491860138,v1:10.20.193.148:6825/2491860138] conn(0x7fbe5c0816d0 0x7fbe5c083af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 20 client.5288.objecter put_session s=0x7fbe5c081000 osd=6 1 2026-02-17T06:01:59.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 20 client.5288.objecter shutdown clearing up homeless session... 2026-02-17T06:01:59.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 10 client.5288.objecter successfully canceled tick 2026-02-17T06:01:59.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbe5c079190 msgr2=0x7fbe5c07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fbe5c079190 0x7fbe5c07b650 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c039000 tx=0x7fbe7c033000 comp rx=0 tx=0).stop 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 10 monclient: shutdown 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbe8c0c7fe0 msgr2=0x7fbe8c155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 --2- 10.20.193.16:0/1368655899 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fbe8c0c7fe0 0x7fbe8c155d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000c410 tx=0x7fbe8000c8e0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe91b37640 1 -- 10.20.193.16:0/1368655899 reap_dead start 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 shutdown_connections 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 >> 10.20.193.16:0/1368655899 conn(0x7fbe8c0cd770 msgr2=0x7fbe8c14e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 shutdown_connections 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 -- 10.20.193.16:0/1368655899 wait complete. 2026-02-17T06:01:59.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.128+0000 7fbe935c1640 1 librados: shutdown 2026-02-17T06:01:59.137 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2026-02-17T06:01:59.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.204+0000 7f7603fd6640 10 monclient: get_monmap_and_config 2026-02-17T06:01:59.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.204+0000 7f7603fd6640 10 monclient: build_initial_monmap 2026-02-17T06:01:59.205 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.204+0000 7f7603fd6640 10 monclient: monmap: 2026-02-17T06:01:59.205 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:59.205 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.205 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:59.205617+0000 2026-02-17T06:01:59.205 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:59.205617+0000 2026-02-17T06:01:59.205 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:59.206 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.206 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:59.206 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:59.206 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:59.206 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:59.206 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:59.206 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:59.206 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.209 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 1 Processor -- start 2026-02-17T06:01:59.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 1 -- start start 2026-02-17T06:01:59.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 10 monclient: init 2026-02-17T06:01:59.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:59.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 10 monclient: _add_conns ranks=[1,3,2,5,0,4] 2026-02-17T06:01:59.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f75fc0700f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:59.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f75fc0c86f0 0x7f75fc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 10 monclient(hunting): picked mon.noname-a con 0x7f75fc0c86f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:59.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f75fc1536f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:59.210 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75fc0c4820 con 0x7f75fc0700f0 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75fc0c4b00 con 0x7f75fc1536f0 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f75fc0c4de0 con 0x7f75fc0c86f0 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.205+0000 7f7603fd6640 10 monclient(hunting): authenticate will time out at 1371.774632s 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f7601d4b640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f75fc0700f0 0x7f75fc0c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:51334/0 (socket says 10.20.193.16:51334) 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f7601d4b640 1 -- 10.20.193.16:0/2162880632 learned_addr learned my addr 10.20.193.16:0/2162880632 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f760254c640 1 --1- 10.20.193.16:0/2162880632 >> v1:10.20.193.148:6789/0 conn(0x7f75fc1536f0 0x7f75fc153b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:51336/0 (socket says 10.20.193.16:51336) 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f760154a640 1 --2- 10.20.193.16:0/2162880632 >> v2:10.20.193.16:3300/0 conn(0x7f75fc0c86f0 0x7f75fc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f760154a640 10 monclient(hunting): get_auth_request con 0x7f75fc0c86f0 auth_method 0 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f760154a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f760154a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f760154a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f760154a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f760154a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.206+0000 7f760154a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 1 -- 10.20.193.16:0/2162880632 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601383495 0 0) 0x7f75fc0c4b00 con 0x7f75fc1536f0 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 10 monclient(hunting): _negotiate 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 10 monclient(hunting): my global_id is 5291 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 1 -- 10.20.193.16:0/2162880632 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75e4003620 con 0x7f75fc1536f0 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 1 -- 10.20.193.16:0/2162880632 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317420997 0 0) 0x7f75fc0c4820 con 0x7f75fc0700f0 2026-02-17T06:01:59.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 10 monclient(hunting): _negotiate 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 10 monclient(hunting): my global_id is 5236 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f7600d49640 1 -- 10.20.193.16:0/2162880632 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75fc0c4b00 con 0x7f75fc0700f0 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f760154a640 10 monclient(hunting): handle_auth_done global_id 5151 payload 274 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f760154a640 1 -- 10.20.193.16:0/2162880632 >> v1:10.20.193.148:6789/0 conn(0x7f75fc1536f0 legacy=0x7f75fc153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f760154a640 1 -- 10.20.193.16:0/2162880632 >> v1:10.20.193.128:6789/0 conn(0x7f75fc0700f0 legacy=0x7f75fc0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f760154a640 10 monclient: _finish_hunting 0 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f760154a640 1 monclient: found mon.noname-a 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f760154a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f760154a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.207+0000 7f760154a640 1 -- 10.20.193.16:0/2162880632 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc0c50c0 con 0x7f75fc0c86f0 2026-02-17T06:01:59.213 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.208+0000 7f760154a640 1 --2- 10.20.193.16:0/2162880632 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f75fc0c86f0 0x7f75fc0704d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f75f00098a0 tx=0x7f75f002edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d3f1682fd0b22 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 1 -- 10.20.193.16:0/2162880632 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f003c070 con 0x7f75fc0c86f0 2026-02-17T06:01:59.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 10 monclient: dump: 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.220 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 10 monclient: _finish_auth 0 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 10 monclient: _check_auth_tickets 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 1 -- 10.20.193.16:0/2162880632 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f002fae0 con 0x7f75fc0c86f0 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 1 -- 10.20.193.16:0/2162880632 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0037440 con 0x7f75fc0c86f0 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7600d49640 10 monclient: dump: 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.221 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 5 monclient: authenticate success, global_id 5151 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 4 set_mon_vals no callback set 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 10 monclient: shutdown 2026-02-17T06:01:59.222 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 1 -- 10.20.193.16:0/2162880632 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f75fc0c86f0 msgr2=0x7f75fc0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 1 --2- 10.20.193.16:0/2162880632 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f75fc0c86f0 0x7f75fc0704d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f75f00098a0 tx=0x7f75f002edd0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 1 -- 10.20.193.16:0/2162880632 shutdown_connections 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 1 --2- 10.20.193.16:0/2162880632 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f75fc0c86f0 0x7f75fc0704d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 1 -- 10.20.193.16:0/2162880632 >> 10.20.193.16:0/2162880632 conn(0x7f75fc0cd770 msgr2=0x7f75fc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 1 -- 10.20.193.16:0/2162880632 shutdown_connections 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.210+0000 7f7603fd6640 1 -- 10.20.193.16:0/2162880632 wait complete. 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.211+0000 7f7603fd6640 10 monclient: build_initial_monmap 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.211+0000 7f7603fd6640 10 monclient: monmap: 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:59.223 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.211+0000 7f7603fd6640 1 librados: starting msgr at 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.211+0000 7f7603fd6640 1 librados: starting objecter 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.211+0000 7f7603fd6640 1 Processor -- start 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.211+0000 7f7603fd6640 1 -- start start 2026-02-17T06:01:59.224 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.211+0000 7f7603fd6640 1 librados: setting wanted keys 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.211+0000 7f7603fd6640 1 librados: calling monclient init 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.211+0000 7f7603fd6640 10 monclient: init 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f75fc0700f0 0x7f75fc1f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 10 monclient(hunting): picked mon.noname-c con 0x7f75fc0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f75fc0c86f0 0x7f75fc151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 10 monclient(hunting): picked mon.noname-a con 0x7f75fc0c86f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f75fc1536f0 0x7f75fc1f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 10 monclient(hunting): picked mon.noname-b con 0x7f75fc1536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc074c00 con 0x7f75fc0c86f0 2026-02-17T06:01:59.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0c5680 con 0x7f75fc1536f0 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc074d80 con 0x7f75fc0700f0 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7603fd6640 10 monclient(hunting): authenticate will time out at 1371.781497s 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f760254c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f75fc1536f0 0x7f75fc1f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f760154a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f75fc0c86f0 0x7f75fc151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f7601d4b640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f75fc0700f0 0x7f75fc1f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f760254c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f75fc1536f0 0x7f75fc1f3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43564/0 (socket says 10.20.193.16:43564) 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f760254c640 1 -- 10.20.193.16:0/2648454187 learned_addr learned my addr 10.20.193.16:0/2648454187 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f760254c640 10 monclient(hunting): get_auth_request con 0x7f75fc1536f0 auth_method 0 2026-02-17T06:01:59.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f760254c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f760254c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.212+0000 7f760254c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f760154a640 10 monclient(hunting): get_auth_request con 0x7f75fc0c86f0 auth_method 0 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f760154a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f760154a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f760154a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f7601d4b640 10 monclient(hunting): get_auth_request con 0x7f75fc0700f0 auth_method 0 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f7601d4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f7601d4b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f7601d4b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f760254c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f760254c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f760254c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f760154a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f760154a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f760154a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f7601d4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f7601d4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.213+0000 7f7601d4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f7601d4b640 10 monclient(hunting): handle_auth_done global_id 5294 payload 824 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f7601d4b640 1 -- 10.20.193.16:0/2648454187 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f75fc1536f0 msgr2=0x7f75fc1f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f7601d4b640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f75fc1536f0 0x7f75fc1f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f7601d4b640 1 -- 10.20.193.16:0/2648454187 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f75fc0c86f0 msgr2=0x7f75fc151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f7601d4b640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f75fc0c86f0 0x7f75fc151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f7601d4b640 10 monclient: _finish_hunting 0 2026-02-17T06:01:59.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f7601d4b640 1 monclient: found mon.noname-c 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f7601d4b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f7601d4b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f7601d4b640 1 -- 10.20.193.16:0/2648454187 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc1f7d40 con 0x7f75fc0700f0 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.214+0000 7f760254c640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f75fc1536f0 0x7f75fc1f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.215+0000 7f7601d4b640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f75fc0700f0 0x7f75fc1f7800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00c480 tx=0x7f75ec00c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.215+0000 7f75ea7fc640 10 client.?.objecter ms_handle_connect 0x7f75fc0700f0 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.215+0000 7f75ea7fc640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 1 -- 10.20.193.16:0/2648454187 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec017070 con 0x7f75fc0700f0 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 10 monclient: dump: 2026-02-17T06:01:59.229 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:59.230 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 1 -- 10.20.193.16:0/2648454187 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec005de0 con 0x7f75fc0700f0 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f7603fd6640 5 monclient: authenticate success, global_id 5294 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f7603fd6640 10 monclient: _renew_subs 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f7603fd6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f7603fd6640 1 -- 10.20.193.16:0/2648454187 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc151320 con 0x7f75fc0700f0 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 1 -- 10.20.193.16:0/2648454187 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec007290 con 0x7f75fc0700f0 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ebfff640 4 set_mon_vals no callback set 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f75ea7fc640 10 monclient: dump: 2026-02-17T06:01:59.231 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f7603fd6640 10 client.5294.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f7603fd6640 10 monclient: _renew_subs 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f7603fd6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.232 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f7603fd6640 1 -- 10.20.193.16:0/2648454187 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc1f45d0 con 0x7f75fc0700f0 2026-02-17T06:01:59.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.217+0000 7f7603fd6640 1 librados: init done 2026-02-17T06:01:59.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.218+0000 7f7603fd6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:59.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.218+0000 7f7603fd6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:59.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.218+0000 7f7603fd6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.233 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.218+0000 7f7603fd6640 1 -- 10.20.193.16:0/2648454187 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75fc0704d0 con 0x7f75fc0700f0 2026-02-17T06:01:59.234 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.229+0000 7f75ea7fc640 1 -- 10.20.193.16:0/2648454187 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f75ec007430 con 0x7f75fc0700f0 2026-02-17T06:01:59.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.229+0000 7f75ea7fc640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f75cc079190 0x7f75cc07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.229+0000 7f760154a640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f75cc079190 0x7f75cc07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.229+0000 7f760154a640 10 monclient: get_auth_request con 0x7f75cc079190 auth_method 0 2026-02-17T06:01:59.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.229+0000 7f75ea7fc640 10 client.5294.objecter ms_handle_connect 0x7f75cc079190 2026-02-17T06:01:59.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.229+0000 7f760154a640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f75cc079190 0x7f75cc07b650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f75f0039000 tx=0x7f75f0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.230+0000 7f75ea7fc640 1 -- 10.20.193.16:0/2648454187 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f75ec007750 con 0x7f75fc0700f0 2026-02-17T06:01:59.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.230+0000 7f75ea7fc640 10 client.5294.objecter ms_dispatch2 0x7f75fc000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:59.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.230+0000 7f75ea7fc640 3 client.5294.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:59.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.230+0000 7f75ea7fc640 3 client.5294.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:59.235 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.230+0000 7f75ea7fc640 20 client.5294.objecter dump_active .. 0 homeless 2026-02-17T06:01:59.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.336+0000 7f75ea7fc640 1 -- 10.20.193.16:0/2648454187 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f75ec021020 con 0x7f75fc0700f0 2026-02-17T06:01:59.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.336+0000 7f75ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:59.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.336+0000 7f75ea7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:59.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.418+0000 7f7603fd6640 1 -- 10.20.193.16:0/2648454187 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f75fc1573f0 con 0x7f75cc079190 2026-02-17T06:01:59.419 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.418+0000 7f75ea7fc640 1 -- 10.20.193.16:0/2648454187 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f75fc1573f0 con 0x7f75cc079190 2026-02-17T06:01:59.419 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 6 to deep-scrub 2026-02-17T06:01:59.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 10 librados: watch_flush enter 2026-02-17T06:01:59.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 10 librados: watch_flush exit 2026-02-17T06:01:59.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 20 client.5294.objecter shutdown clearing up homeless session... 2026-02-17T06:01:59.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 10 client.5294.objecter successfully canceled tick 2026-02-17T06:01:59.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 -- 10.20.193.16:0/2648454187 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f75cc079190 msgr2=0x7f75cc07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f75cc079190 0x7f75cc07b650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f75f0039000 tx=0x7f75f0033000 comp rx=0 tx=0).stop 2026-02-17T06:01:59.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 10 monclient: shutdown 2026-02-17T06:01:59.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.421 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 -- 10.20.193.16:0/2648454187 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f75fc0700f0 msgr2=0x7f75fc1f7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f75fc0700f0 0x7f75fc1f7800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00c480 tx=0x7f75ec00c950 comp rx=0 tx=0).stop 2026-02-17T06:01:59.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 -- 10.20.193.16:0/2648454187 shutdown_connections 2026-02-17T06:01:59.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f75cc079190 0x7f75cc07b650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f75fc1536f0 0x7f75fc1f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f75fc0c86f0 0x7f75fc151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 --2- 10.20.193.16:0/2648454187 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f75fc0700f0 0x7f75fc1f7800 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 -- 10.20.193.16:0/2648454187 >> 10.20.193.16:0/2648454187 conn(0x7f75fc0cd770 msgr2=0x7f75fc153b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 -- 10.20.193.16:0/2648454187 shutdown_connections 2026-02-17T06:01:59.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 -- 10.20.193.16:0/2648454187 wait complete. 2026-02-17T06:01:59.422 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.419+0000 7f7603fd6640 1 librados: shutdown 2026-02-17T06:01:59.428 INFO:tasks.ceph:Scrubbing osd.10 2026-02-17T06:01:59.428 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:59.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.498+0000 7ffa210c4640 10 monclient: get_monmap_and_config 2026-02-17T06:01:59.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.498+0000 7ffa210c4640 10 monclient: build_initial_monmap 2026-02-17T06:01:59.499 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.498+0000 7ffa210c4640 10 monclient: monmap: 2026-02-17T06:01:59.499 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:59.499880+0000 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:59.499880+0000 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:59.500 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:59.501 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.499+0000 7ffa210c4640 1 Processor -- start 2026-02-17T06:01:59.503 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.499+0000 7ffa210c4640 1 -- start start 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.499+0000 7ffa210c4640 10 monclient: init 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 10 monclient: _add_conns ranks=[1,5,3,0,2,4] 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ffa1c155e00 addr v1:10.20.193.128:6789/0 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7ffa1c1561e0 0x7ffa1c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 10 monclient(hunting): picked mon.noname-c con 0x7ffa1c1561e0 addr v2:10.20.193.148:3300/0 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7ffa1c074dc0 0x7ffa1c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 10 monclient(hunting): picked mon.noname-a con 0x7ffa1c074dc0 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1c0c96d0 con 0x7ffa1c155e00 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.504 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c074860 con 0x7ffa1c074dc0 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c0c9940 con 0x7ffa1c1561e0 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa210c4640 10 monclient(hunting): authenticate will time out at 1372.069249s 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1ad76640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7ffa1c155e00 0x7ffa1c074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:51348/0 (socket says 10.20.193.16:51348) 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1ad76640 1 -- 10.20.193.16:0/4181189008 learned_addr learned my addr 10.20.193.16:0/4181189008 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1a575640 1 --2- 10.20.193.16:0/4181189008 >> v2:10.20.193.148:3300/0 conn(0x7ffa1c1561e0 0x7ffa1c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1b577640 1 --2- 10.20.193.16:0/4181189008 >> v2:10.20.193.16:3300/0 conn(0x7ffa1c074dc0 0x7ffa1c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1a575640 10 monclient(hunting): get_auth_request con 0x7ffa1c1561e0 auth_method 0 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.505 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1b577640 10 monclient(hunting): get_auth_request con 0x7ffa1c074dc0 auth_method 0 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1b577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.500+0000 7ffa1b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.501+0000 7ffa1a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.501+0000 7ffa1a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.501+0000 7ffa1a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.501+0000 7ffa1b577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.501+0000 7ffa1b577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.501+0000 7ffa1b577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa1a575640 10 monclient(hunting): handle_auth_done global_id 5297 payload 274 2026-02-17T06:01:59.506 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa1a575640 1 -- 10.20.193.16:0/4181189008 >> v2:10.20.193.16:3300/0 conn(0x7ffa1c074dc0 msgr2=0x7ffa1c0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:59.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa1a575640 1 --2- 10.20.193.16:0/4181189008 >> v2:10.20.193.16:3300/0 conn(0x7ffa1c074dc0 0x7ffa1c0cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa1a575640 1 -- 10.20.193.16:0/4181189008 >> v1:10.20.193.128:6789/0 conn(0x7ffa1c155e00 legacy=0x7ffa1c074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa1a575640 10 monclient: _finish_hunting 0 2026-02-17T06:01:59.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa1a575640 1 monclient: found mon.noname-c 2026-02-17T06:01:59.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa19d74640 1 -- 10.20.193.16:0/4181189008 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574593711 0 0) 0x7ffa1c0c96d0 con 0x7ffa1c155e00 2026-02-17T06:01:59.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa1a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:59.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa1a575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa1a575640 1 -- 10.20.193.16:0/4181189008 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0c9c20 con 0x7ffa1c1561e0 2026-02-17T06:01:59.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.502+0000 7ffa19d74640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T06:01:59.507 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.503+0000 7ffa1a575640 1 --2- 10.20.193.16:0/4181189008 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa1c1561e0 0x7ffa1c0ca700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0693f0 tx=0x7ffa0402ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a78ef0f814a348c0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 1 -- 10.20.193.16:0/4181189008 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0403c070 con 0x7ffa1c1561e0 2026-02-17T06:01:59.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:59.514 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 10 monclient: dump: 2026-02-17T06:01:59.514 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.514 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.514 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.514 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.514 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 10 monclient: _finish_auth 0 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 10 monclient: _check_auth_tickets 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 1 -- 10.20.193.16:0/4181189008 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0402faf0 con 0x7ffa1c1561e0 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa210c4640 5 monclient: authenticate success, global_id 5297 2026-02-17T06:01:59.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 1 -- 10.20.193.16:0/4181189008 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04037440 con 0x7ffa1c1561e0 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa210c4640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa210c4640 4 set_mon_vals no callback set 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.504+0000 7ffa19d74640 10 monclient: dump: 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.516 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 10 monclient: shutdown 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 1 -- 10.20.193.16:0/4181189008 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa1c1561e0 msgr2=0x7ffa1c0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 1 --2- 10.20.193.16:0/4181189008 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa1c1561e0 0x7ffa1c0ca700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0693f0 tx=0x7ffa0402ede0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 1 -- 10.20.193.16:0/4181189008 shutdown_connections 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 1 --2- 10.20.193.16:0/4181189008 >> v2:10.20.193.16:3300/0 conn(0x7ffa1c074dc0 0x7ffa1c0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 1 --2- 10.20.193.16:0/4181189008 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa1c1561e0 0x7ffa1c0ca700 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 1 -- 10.20.193.16:0/4181189008 >> 10.20.193.16:0/4181189008 conn(0x7ffa1c0ce6c0 msgr2=0x7ffa1c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 1 -- 10.20.193.16:0/4181189008 shutdown_connections 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 1 -- 10.20.193.16:0/4181189008 wait complete. 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 10 monclient: build_initial_monmap 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.505+0000 7ffa210c4640 10 monclient: monmap: 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:59.518 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 librados: starting msgr at 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 librados: starting objecter 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 Processor -- start 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 -- start start 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 librados: setting wanted keys 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 librados: calling monclient init 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient: init 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa1c074dc0 0x7ffa1c14d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient(hunting): picked mon.noname-c con 0x7ffa1c074dc0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffa1c155e00 0x7ffa1c1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.519 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient(hunting): picked mon.noname-b con 0x7ffa1c155e00 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffa1c1561e0 0x7ffa1c1496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient(hunting): picked mon.noname-a con 0x7ffa1c1561e0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c0c9360 con 0x7ffa1c1561e0 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c0ca1e0 con 0x7ffa1c155e00 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c0c8c20 con 0x7ffa1c074dc0 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa210c4640 10 monclient(hunting): authenticate will time out at 1372.075670s 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa1b577640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffa1c1561e0 0x7ffa1c1496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.520 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.506+0000 7ffa1ad76640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa1c074dc0 0x7ffa1c14d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1a575640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffa1c155e00 0x7ffa1c1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1b577640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffa1c1561e0 0x7ffa1c1496e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33938/0 (socket says 10.20.193.16:33938) 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1b577640 1 -- 10.20.193.16:0/2607241714 learned_addr learned my addr 10.20.193.16:0/2607241714 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1b577640 10 monclient(hunting): get_auth_request con 0x7ffa1c1561e0 auth_method 0 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1b577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1ad76640 10 monclient(hunting): get_auth_request con 0x7ffa1c074dc0 auth_method 0 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.521 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1a575640 10 monclient(hunting): get_auth_request con 0x7ffa1c155e00 auth_method 0 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1b577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1b577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1b577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.507+0000 7ffa1a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.522 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.508+0000 7ffa1ad76640 10 monclient(hunting): handle_auth_done global_id 5300 payload 824 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.508+0000 7ffa1ad76640 1 -- 10.20.193.16:0/2607241714 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffa1c155e00 msgr2=0x7ffa1c1491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.508+0000 7ffa1ad76640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7ffa1c155e00 0x7ffa1c1491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.508+0000 7ffa1ad76640 1 -- 10.20.193.16:0/2607241714 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffa1c1561e0 msgr2=0x7ffa1c1496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.508+0000 7ffa1ad76640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7ffa1c1561e0 0x7ffa1c1496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.508+0000 7ffa1ad76640 10 monclient: _finish_hunting 0 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.508+0000 7ffa1ad76640 1 monclient: found mon.noname-c 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.508+0000 7ffa1ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.508+0000 7ffa1ad76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.508+0000 7ffa1ad76640 1 -- 10.20.193.16:0/2607241714 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c1f6eb0 con 0x7ffa1c074dc0 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.509+0000 7ffa1ad76640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa1c074dc0 0x7ffa1c14d390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000c4d0 tx=0x7ffa1000c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.509+0000 7ff9fb7fe640 10 client.?.objecter ms_handle_connect 0x7ffa1c074dc0 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.509+0000 7ff9fb7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:59.523 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 1 -- 10.20.193.16:0/2607241714 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10018070 con 0x7ffa1c074dc0 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 10 monclient: dump: 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.524 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 5 monclient: authenticate success, global_id 5300 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 10 monclient: _renew_subs 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1c1f7190 con 0x7ffa1c074dc0 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 10 client.5300.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 10 monclient: _renew_subs 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1c1f7750 con 0x7ffa1c074dc0 2026-02-17T06:01:59.525 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 1 librados: init done 2026-02-17T06:01:59.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 1 -- 10.20.193.16:0/2607241714 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa10004cf0 con 0x7ffa1c074dc0 2026-02-17T06:01:59.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:59.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 1 -- 10.20.193.16:0/2607241714 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10022060 con 0x7ffa1c074dc0 2026-02-17T06:01:59.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:59.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 10 client.5300.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:59.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 15 client.5300.objecter _session_command_op_assign -1 1 2026-02-17T06:01:59.526 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ff9fb7fe640 10 monclient: dump: 2026-02-17T06:01:59.526 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.526 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 20 client.5300.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 10 client.5300.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa19573640 4 set_mon_vals no callback set 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:01:59.527 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.511+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa1c148cd0 con 0x7ffa1c074dc0 2026-02-17T06:01:59.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.523+0000 7ff9fb7fe640 1 -- 10.20.193.16:0/2607241714 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7ffa100072d0 con 0x7ffa1c074dc0 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.523+0000 7ff9fb7fe640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff9e8079170 0x7ff9e807b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.523+0000 7ffa1a575640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff9e8079170 0x7ff9e807b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.523+0000 7ffa1a575640 10 monclient: get_auth_request con 0x7ff9e8079170 auth_method 0 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.524+0000 7ffa1a575640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff9e8079170 0x7ff9e807b630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa04039000 tx=0x7ffa04033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.524+0000 7ff9fb7fe640 10 client.5300.objecter ms_handle_connect 0x7ff9e8079170 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.524+0000 7ff9fb7fe640 1 -- 10.20.193.16:0/2607241714 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7ffa10097790 con 0x7ffa1c074dc0 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.524+0000 7ff9fb7fe640 10 client.5300.objecter ms_dispatch2 0x7ffa1c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.524+0000 7ff9fb7fe640 3 client.5300.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.524+0000 7ff9fb7fe640 3 client.5300.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 10 client.5300.objecter checking command 1 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] conn(0x7ff9e8081650 0x7ff9e8083a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.530 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 20 client.5300.objecter _get_session s=0x7ff9e8080f80 osd=10 3 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 20 client.5300.objecter put_session s=0x7ff9e8080f80 osd=10 3 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 15 client.5300.objecter _session_command_op_remove -1 1 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 20 client.5300.objecter _get_session s=0x7ff9e8080f80 osd=10 3 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 20 client.5300.objecter get_session s=0x7ff9e8080f80 osd=10 3 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 15 client.5300.objecter _session_command_op_assign 10 1 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 20 client.5300.objecter put_session s=0x7ff9e8080f80 osd=10 4 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 10 client.5300.objecter _send_command 1 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 1 -- 10.20.193.16:0/2607241714 --> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa10017ec0 con 0x7ff9e8081650 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 20 client.5300.objecter dump_active .. 0 homeless 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 1 -- 10.20.193.16:0/2607241714 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffa1005e1c0 con 0x7ffa1c074dc0 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ff9fb7fe640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ffa1b577640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] conn(0x7ff9e8081650 0x7ff9e8083a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.525+0000 7ffa1b577640 10 monclient: get_auth_request con 0x7ff9e8081650 auth_method 0 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.527+0000 7ffa1b577640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] conn(0x7ff9e8081650 0x7ff9e8083a70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.531 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.527+0000 7ff9fb7fe640 10 client.5300.objecter ms_handle_connect 0x7ff9e8081650 2026-02-17T06:01:59.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.537+0000 7ff9fb7fe640 1 -- 10.20.193.16:0/2607241714 <== osd.10 v2:10.20.193.148:6808/2531105113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7ffa10017ec0 con 0x7ff9e8081650 2026-02-17T06:01:59.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.537+0000 7ff9fb7fe640 10 client.5300.objecter ms_dispatch2 0x7ffa1c000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:59.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.537+0000 7ff9fb7fe640 10 client.5300.objecter _finish_command 1 = system:0 2026-02-17T06:01:59.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.537+0000 7ff9fb7fe640 20 client.5300.objecter put_session s=0x7ff9e8080f80 osd=10 4 2026-02-17T06:01:59.537 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.537+0000 7ff9fb7fe640 15 client.5300.objecter _session_command_op_remove 10 1 2026-02-17T06:01:59.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 10 client.5300.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:59.543 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 15 client.5300.objecter _session_command_op_assign -1 2 2026-02-17T06:01:59.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 20 client.5300.objecter _get_session s=0x7ff9e8080f80 osd=10 3 2026-02-17T06:01:59.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 20 client.5300.objecter put_session s=0x7ff9e8080f80 osd=10 3 2026-02-17T06:01:59.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 20 client.5300.objecter _get_session s=0x7ff9e8080f80 osd=10 3 2026-02-17T06:01:59.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 15 client.5300.objecter _session_command_op_remove -1 2 2026-02-17T06:01:59.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 20 client.5300.objecter get_session s=0x7ff9e8080f80 osd=10 3 2026-02-17T06:01:59.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 15 client.5300.objecter _session_command_op_assign 10 2 2026-02-17T06:01:59.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 20 client.5300.objecter put_session s=0x7ff9e8080f80 osd=10 4 2026-02-17T06:01:59.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 10 client.5300.objecter _send_command 2 2026-02-17T06:01:59.544 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.542+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 --> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffa1c0c8fa0 con 0x7ff9e8081650 2026-02-17T06:01:59.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ff9fb7fe640 1 -- 10.20.193.16:0/2607241714 <== osd.10 v2:10.20.193.148:6808/2531105113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffa1c0c8fa0 con 0x7ff9e8081650 2026-02-17T06:01:59.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ff9fb7fe640 10 client.5300.objecter ms_dispatch2 0x7ffa1c000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:59.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ff9fb7fe640 10 client.5300.objecter _finish_command 2 = system:0 2026-02-17T06:01:59.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ff9fb7fe640 20 client.5300.objecter put_session s=0x7ff9e8080f80 osd=10 4 2026-02-17T06:01:59.550 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ff9fb7fe640 15 client.5300.objecter _session_command_op_remove 10 2 2026-02-17T06:01:59.551 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:59.551 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:59.551 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:59.552 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 10 librados: watch_flush enter 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 10 librados: watch_flush exit 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 10 client.5300.objecter close_session for osd.10 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 >> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] conn(0x7ff9e8081650 msgr2=0x7ff9e8083a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.148:6808/2531105113,v1:10.20.193.148:6809/2531105113] conn(0x7ff9e8081650 0x7ff9e8083a70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 20 client.5300.objecter put_session s=0x7ff9e8080f80 osd=10 1 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 20 client.5300.objecter shutdown clearing up homeless session... 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 10 client.5300.objecter successfully canceled tick 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff9e8079170 msgr2=0x7ff9e807b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7ff9e8079170 0x7ff9e807b630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa04039000 tx=0x7ffa04033000 comp rx=0 tx=0).stop 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 10 monclient: shutdown 2026-02-17T06:01:59.553 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa1c074dc0 msgr2=0x7ffa1c14d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 --2- 10.20.193.16:0/2607241714 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7ffa1c074dc0 0x7ffa1c14d390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000c4d0 tx=0x7ffa1000c9a0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa1b577640 1 -- 10.20.193.16:0/2607241714 reap_dead start 2026-02-17T06:01:59.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 shutdown_connections 2026-02-17T06:01:59.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 >> 10.20.193.16:0/2607241714 conn(0x7ffa1c0ce6c0 msgr2=0x7ffa1c14ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 shutdown_connections 2026-02-17T06:01:59.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 -- 10.20.193.16:0/2607241714 wait complete. 2026-02-17T06:01:59.554 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.550+0000 7ffa210c4640 1 librados: shutdown 2026-02-17T06:01:59.558 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2026-02-17T06:01:59.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.627+0000 7fa531ba9640 10 monclient: get_monmap_and_config 2026-02-17T06:01:59.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.627+0000 7fa531ba9640 10 monclient: build_initial_monmap 2026-02-17T06:01:59.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.627+0000 7fa531ba9640 10 monclient: monmap: 2026-02-17T06:01:59.627 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:59.627 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.627 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:59.628427+0000 2026-02-17T06:01:59.627 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:59.628427+0000 2026-02-17T06:01:59.627 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:59.627 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.627 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.627+0000 7fa531ba9640 1 Processor -- start 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.627+0000 7fa531ba9640 1 -- start start 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.627+0000 7fa531ba9640 10 monclient: init 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 10 monclient: _add_conns ranks=[3,2,4,0,1,5] 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fa52c155e00 0x7fa52c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 10 monclient(hunting): picked mon.noname-a con 0x7fa52c155e00 addr v2:10.20.193.16:3300/0 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa52c1561e0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fa52c074dc0 0x7fa52c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 10 monclient(hunting): picked mon.noname-b con 0x7fa52c074dc0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:59.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52c0c96d0 con 0x7fa52c1561e0 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa52c074860 con 0x7fa52c155e00 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fa52c0c9940 con 0x7fa52c074dc0 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa531ba9640 10 monclient(hunting): authenticate will time out at 1372.197496s 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa52b7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fa52c155e00 0x7fa52c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa52bfff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fa52c074dc0 0x7fa52c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.628+0000 7fa52b7fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fa52c155e00 0x7fa52c0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:33942/0 (socket says 10.20.193.16:33942) 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52b7fe640 1 -- 10.20.193.16:0/1527674332 learned_addr learned my addr 10.20.193.16:0/1527674332 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52b7fe640 10 monclient(hunting): get_auth_request con 0x7fa52c155e00 auth_method 0 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52affd640 1 --1- 10.20.193.16:0/1527674332 >> v1:10.20.193.148:6789/0 conn(0x7fa52c1561e0 0x7fa52c074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:51348/0 (socket says 10.20.193.16:51348) 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52bfff640 10 monclient(hunting): get_auth_request con 0x7fa52c074dc0 auth_method 0 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.629+0000 7fa52bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.630+0000 7fa52bfff640 10 monclient(hunting): handle_auth_done global_id 5248 payload 274 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.630+0000 7fa52bfff640 1 -- 10.20.193.16:0/1527674332 >> v2:10.20.193.16:3300/0 conn(0x7fa52c155e00 msgr2=0x7fa52c0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.630+0000 7fa52bfff640 1 --2- 10.20.193.16:0/1527674332 >> v2:10.20.193.16:3300/0 conn(0x7fa52c155e00 0x7fa52c0ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.630+0000 7fa52bfff640 1 -- 10.20.193.16:0/1527674332 >> v1:10.20.193.148:6789/0 conn(0x7fa52c1561e0 legacy=0x7fa52c074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.630+0000 7fa52bfff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.630+0000 7fa52bfff640 1 monclient: found mon.noname-b 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.630+0000 7fa52bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.630+0000 7fa52bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:59.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.630+0000 7fa52bfff640 1 -- 10.20.193.16:0/1527674332 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c0c9c20 con 0x7fa52c074dc0 2026-02-17T06:01:59.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.631+0000 7fa52bfff640 1 --2- 10.20.193.16:0/1527674332 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c074dc0 0x7fa52c0cc1a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa52c06cbb0 tx=0x7fa52002f1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e401670b20262631 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 1 -- 10.20.193.16:0/1527674332 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52003c070 con 0x7fa52c074dc0 2026-02-17T06:01:59.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 10 monclient: dump: 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 1 -- 10.20.193.16:0/1527674332 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa520037440 con 0x7fa52c074dc0 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa531ba9640 5 monclient: authenticate success, global_id 5248 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 1 -- 10.20.193.16:0/1527674332 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520037810 con 0x7fa52c074dc0 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa52a7fc640 10 monclient: dump: 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa531ba9640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:59.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.632+0000 7fa531ba9640 4 set_mon_vals no callback set 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 10 monclient: shutdown 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 -- 10.20.193.16:0/1527674332 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c074dc0 msgr2=0x7fa52c0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 --2- 10.20.193.16:0/1527674332 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c074dc0 0x7fa52c0cc1a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa52c06cbb0 tx=0x7fa52002f1c0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 -- 10.20.193.16:0/1527674332 shutdown_connections 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 --2- 10.20.193.16:0/1527674332 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c074dc0 0x7fa52c0cc1a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 --2- 10.20.193.16:0/1527674332 >> v2:10.20.193.16:3300/0 conn(0x7fa52c155e00 0x7fa52c0ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 -- 10.20.193.16:0/1527674332 >> 10.20.193.16:0/1527674332 conn(0x7fa52c0ce6c0 msgr2=0x7fa52c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 -- 10.20.193.16:0/1527674332 shutdown_connections 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 -- 10.20.193.16:0/1527674332 wait complete. 2026-02-17T06:01:59.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 10 monclient: build_initial_monmap 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 10 monclient: monmap: 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 librados: starting msgr at 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 librados: starting objecter 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 Processor -- start 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 -- start start 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 librados: setting wanted keys 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 1 librados: calling monclient init 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.633+0000 7fa531ba9640 10 monclient: init 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa52c074dc0 0x7fa52c1f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 10 monclient(hunting): picked mon.noname-a con 0x7fa52c074dc0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa52c155e00 0x7fa52c141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 10 monclient(hunting): picked mon.noname-c con 0x7fa52c155e00 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c1561e0 0x7fa52c1f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 10 monclient(hunting): picked mon.noname-b con 0x7fa52c1561e0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c0c9360 con 0x7fa52c074dc0 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c0ca1e0 con 0x7fa52c1561e0 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c0c8c20 con 0x7fa52c155e00 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa531ba9640 10 monclient(hunting): authenticate will time out at 1372.203220s 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52b7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa52c074dc0 0x7fa52c1f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52bfff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c1561e0 0x7fa52c1f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52bfff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c1561e0 0x7fa52c1f3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43580/0 (socket says 10.20.193.16:43580) 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52bfff640 1 -- 10.20.193.16:0/2708981160 learned_addr learned my addr 10.20.193.16:0/2708981160 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52bfff640 10 monclient(hunting): get_auth_request con 0x7fa52c1561e0 auth_method 0 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52b7fe640 10 monclient(hunting): get_auth_request con 0x7fa52c074dc0 auth_method 0 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52affd640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa52c155e00 0x7fa52c141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52affd640 10 monclient(hunting): get_auth_request con 0x7fa52c155e00 auth_method 0 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.634+0000 7fa52affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.635+0000 7fa52affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.635+0000 7fa52affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.635+0000 7fa52affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.635+0000 7fa52bfff640 10 monclient(hunting): handle_auth_done global_id 5251 payload 824 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.635+0000 7fa52bfff640 1 -- 10.20.193.16:0/2708981160 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa52c155e00 msgr2=0x7fa52c141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.635+0000 7fa52bfff640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa52c155e00 0x7fa52c141b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.636+0000 7fa52bfff640 1 -- 10.20.193.16:0/2708981160 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa52c074dc0 msgr2=0x7fa52c1f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.636+0000 7fa52bfff640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa52c074dc0 0x7fa52c1f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.636+0000 7fa52bfff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.636+0000 7fa52bfff640 1 monclient: found mon.noname-b 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.636+0000 7fa52bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.636+0000 7fa52bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.636+0000 7fa52bfff640 1 -- 10.20.193.16:0/2708981160 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c1f4850 con 0x7fa52c1561e0 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.636+0000 7fa52bfff640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c1561e0 0x7fa52c1f3cd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa52003a040 tx=0x7fa520004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.637+0000 7fa507fff640 10 client.?.objecter ms_handle_connect 0x7fa52c1561e0 2026-02-17T06:01:59.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.637+0000 7fa507fff640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:59.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 1 -- 10.20.193.16:0/2708981160 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52003c050 con 0x7fa52c1561e0 2026-02-17T06:01:59.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 10 monclient: dump: 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 10 monclient: _finish_auth 0 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 1 -- 10.20.193.16:0/2708981160 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa520048070 con 0x7fa52c1561e0 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 1 -- 10.20.193.16:0/2708981160 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520035a90 con 0x7fa52c1561e0 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa531ba9640 5 monclient: authenticate success, global_id 5251 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa529ffb640 4 set_mon_vals no callback set 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa507fff640 10 monclient: dump: 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa531ba9640 10 monclient: _renew_subs 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa531ba9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa531ba9640 1 -- 10.20.193.16:0/2708981160 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c1f4b30 con 0x7fa52c1561e0 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa531ba9640 10 client.5251.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa531ba9640 10 monclient: _renew_subs 2026-02-17T06:01:59.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa531ba9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:59.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa531ba9640 1 -- 10.20.193.16:0/2708981160 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c150420 con 0x7fa52c1561e0 2026-02-17T06:01:59.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.638+0000 7fa531ba9640 1 librados: init done 2026-02-17T06:01:59.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.639+0000 7fa531ba9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:59.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.639+0000 7fa531ba9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:59.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.639+0000 7fa531ba9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:59.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.639+0000 7fa531ba9640 1 -- 10.20.193.16:0/2708981160 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c0c8fa0 con 0x7fa52c1561e0 2026-02-17T06:01:59.743 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.743+0000 7fa507fff640 1 -- 10.20.193.16:0/2708981160 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fa520035c30 con 0x7fa52c1561e0 2026-02-17T06:01:59.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.743+0000 7fa507fff640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa4fc079190 0x7fa4fc07b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.743+0000 7fa52b7fe640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa4fc079190 0x7fa4fc07b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.743+0000 7fa52b7fe640 10 monclient: get_auth_request con 0x7fa4fc079190 auth_method 0 2026-02-17T06:01:59.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.744+0000 7fa52b7fe640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa4fc079190 0x7fa4fc07b650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa518006dc0 tx=0x7fa518009340 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.744 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.744+0000 7fa507fff640 10 client.5251.objecter ms_handle_connect 0x7fa4fc079190 2026-02-17T06:01:59.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.745+0000 7fa507fff640 1 -- 10.20.193.16:0/2708981160 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fa520084780 con 0x7fa52c1561e0 2026-02-17T06:01:59.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.745+0000 7fa507fff640 10 client.5251.objecter ms_dispatch2 0x7fa52c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:59.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.745+0000 7fa507fff640 3 client.5251.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:59.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.745+0000 7fa507fff640 3 client.5251.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:59.745 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.745+0000 7fa507fff640 20 client.5251.objecter dump_active .. 0 homeless 2026-02-17T06:01:59.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.751+0000 7fa507fff640 1 -- 10.20.193.16:0/2708981160 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa520088740 con 0x7fa52c1561e0 2026-02-17T06:01:59.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.751+0000 7fa507fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:59.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.751+0000 7fa507fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:59.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.831+0000 7fa531ba9640 1 -- 10.20.193.16:0/2708981160 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "10", "target": ["mon-mgr", ""]}) -- 0x7fa52c0cca70 con 0x7fa4fc079190 2026-02-17T06:01:59.832 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.831+0000 7fa507fff640 1 -- 10.20.193.16:0/2708981160 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 10 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7fa52c0cca70 con 0x7fa4fc079190 2026-02-17T06:01:59.832 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 10 to deep-scrub 2026-02-17T06:01:59.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 10 librados: watch_flush enter 2026-02-17T06:01:59.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 10 librados: watch_flush exit 2026-02-17T06:01:59.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 20 client.5251.objecter shutdown clearing up homeless session... 2026-02-17T06:01:59.833 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 10 client.5251.objecter successfully canceled tick 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 -- 10.20.193.16:0/2708981160 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa4fc079190 msgr2=0x7fa4fc07b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa4fc079190 0x7fa4fc07b650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa518006dc0 tx=0x7fa518009340 comp rx=0 tx=0).stop 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 10 monclient: shutdown 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 -- 10.20.193.16:0/2708981160 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c1561e0 msgr2=0x7fa52c1f3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c1561e0 0x7fa52c1f3cd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa52003a040 tx=0x7fa520004b40 comp rx=0 tx=0).stop 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 -- 10.20.193.16:0/2708981160 shutdown_connections 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa4fc079190 0x7fa4fc07b650 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa52c1561e0 0x7fa52c1f3cd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa52c155e00 0x7fa52c141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 --2- 10.20.193.16:0/2708981160 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa52c074dc0 0x7fa52c1f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 -- 10.20.193.16:0/2708981160 >> 10.20.193.16:0/2708981160 conn(0x7fa52c0ce6c0 msgr2=0x7fa52c153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 -- 10.20.193.16:0/2708981160 shutdown_connections 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 -- 10.20.193.16:0/2708981160 wait complete. 2026-02-17T06:01:59.834 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.832+0000 7fa531ba9640 1 librados: shutdown 2026-02-17T06:01:59.841 INFO:tasks.ceph:Scrubbing osd.14 2026-02-17T06:01:59.841 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:01:59.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.911+0000 7f117bf53640 10 monclient: get_monmap_and_config 2026-02-17T06:01:59.912 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.911+0000 7f117bf53640 10 monclient: build_initial_monmap 2026-02-17T06:01:59.913 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.912+0000 7f117bf53640 10 monclient: monmap: 2026-02-17T06:01:59.913 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:59.913 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.913 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:01:59.913011+0000 2026-02-17T06:01:59.913 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:01:59.913011+0000 2026-02-17T06:01:59.913 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:59.913 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.913 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:01:59.913 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:01:59.913 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:01:59.914 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:01:59.914 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:01:59.914 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:01:59.914 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.912+0000 7f117bf53640 1 Processor -- start 2026-02-17T06:01:59.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.912+0000 7f117bf53640 1 -- start start 2026-02-17T06:01:59.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.912+0000 7f117bf53640 10 monclient: init 2026-02-17T06:01:59.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.912+0000 7f117bf53640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:59.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 10 monclient: _add_conns ranks=[4,2,0,3,1,5] 2026-02-17T06:01:59.917 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f11740700f0 0x7f11740704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 10 monclient(hunting): picked mon.noname-b con 0x7f11740700f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f11740c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f11741536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11740c4820 con 0x7f11741536f0 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11740c4b00 con 0x7f11740c7fe0 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f11740c4de0 con 0x7f11740700f0 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f117bf53640 10 monclient(hunting): authenticate will time out at 1372.482063s 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f1179cc8640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f11740700f0 0x7f11740704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.918 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f11794c7640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f11740c7fe0 0x7f11740c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:51356/0 (socket says 10.20.193.16:51356) 2026-02-17T06:01:59.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f11794c7640 1 -- 10.20.193.16:0/2094059499 learned_addr learned my addr 10.20.193.16:0/2094059499 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:01:59.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f1179cc8640 10 monclient(hunting): get_auth_request con 0x7f11740700f0 auth_method 0 2026-02-17T06:01:59.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f1179cc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f1179cc8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f1179cc8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f1179cc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f1179cc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.913+0000 7f1179cc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.914+0000 7f1179cc8640 10 monclient(hunting): handle_auth_done global_id 5254 payload 274 2026-02-17T06:01:59.919 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.914+0000 7f1179cc8640 1 -- 10.20.193.16:0/2094059499 >> v1:10.20.193.148:6789/0 conn(0x7f11740c7fe0 legacy=0x7f11740c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.914+0000 7f1179cc8640 1 -- 10.20.193.16:0/2094059499 >> v1:10.20.193.16:6789/0 conn(0x7f11741536f0 legacy=0x7f1174153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.914+0000 7f1179cc8640 10 monclient: _finish_hunting 0 2026-02-17T06:01:59.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.914+0000 7f1179cc8640 1 monclient: found mon.noname-b 2026-02-17T06:01:59.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.914+0000 7f1179cc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:59.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.914+0000 7f1178cc6640 1 -- 10.20.193.16:0/2094059499 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3275136806 0 0) 0x7f11740c4b00 con 0x7f11740c7fe0 2026-02-17T06:01:59.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.914+0000 7f1179cc8640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:59.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.914+0000 7f1179cc8640 1 -- 10.20.193.16:0/2094059499 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11740c50c0 con 0x7f11740700f0 2026-02-17T06:01:59.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.914+0000 7f1178cc6640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T06:01:59.920 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.915+0000 7f1179cc8640 1 --2- 10.20.193.16:0/2094059499 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11740700f0 0x7f11740704d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f11640098a0 tx=0x7f116402edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac7fe38f4e7fac8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.917+0000 7f1178cc6640 1 -- 10.20.193.16:0/2094059499 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116403c070 con 0x7f11740700f0 2026-02-17T06:01:59.927 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.917+0000 7f1178cc6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.917+0000 7f1178cc6640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:59.928 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.917+0000 7f1178cc6640 10 monclient: dump: 2026-02-17T06:01:59.928 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.928 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.928 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.928 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.928 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.928 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.928 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.917+0000 7f1178cc6640 10 monclient: _finish_auth 0 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.917+0000 7f1178cc6640 10 monclient: _check_auth_tickets 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.917+0000 7f1178cc6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.917+0000 7f117bf53640 5 monclient: authenticate success, global_id 5254 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.917+0000 7f117bf53640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.918+0000 7f1178cc6640 1 -- 10.20.193.16:0/2094059499 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116402fae0 con 0x7f11740700f0 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.918+0000 7f1178cc6640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:59.929 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.918+0000 7f1178cc6640 1 -- 10.20.193.16:0/2094059499 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164037440 con 0x7f11740700f0 2026-02-17T06:01:59.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.918+0000 7f1178cc6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.918+0000 7f1178cc6640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:59.930 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.918+0000 7f1178cc6640 10 monclient: dump: 2026-02-17T06:01:59.930 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.930 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.930 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.930 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.930 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.930 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.930 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.931 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.931 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.931 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.918+0000 7f117bf53640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:59.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.918+0000 7f117bf53640 4 set_mon_vals no callback set 2026-02-17T06:01:59.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 10 monclient: shutdown 2026-02-17T06:01:59.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 1 -- 10.20.193.16:0/2094059499 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11740700f0 msgr2=0x7f11740704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 1 --2- 10.20.193.16:0/2094059499 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11740700f0 0x7f11740704d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f11640098a0 tx=0x7f116402edd0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.931 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 1 -- 10.20.193.16:0/2094059499 shutdown_connections 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 1 --2- 10.20.193.16:0/2094059499 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11740700f0 0x7f11740704d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 1 -- 10.20.193.16:0/2094059499 >> 10.20.193.16:0/2094059499 conn(0x7f11740cd770 msgr2=0x7f11740cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 1 -- 10.20.193.16:0/2094059499 shutdown_connections 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 1 -- 10.20.193.16:0/2094059499 wait complete. 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 10 monclient: build_initial_monmap 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 10 monclient: monmap: 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:01:59.932 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 1 librados: starting msgr at 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 1 librados: starting objecter 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.919+0000 7f117bf53640 1 Processor -- start 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 1 -- start start 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 1 librados: setting wanted keys 2026-02-17T06:01:59.933 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 1 librados: calling monclient init 2026-02-17T06:01:59.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient: init 2026-02-17T06:01:59.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:59.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:59.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f11740700f0 0x7f117406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient(hunting): picked mon.noname-a con 0x7f11740700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:01:59.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11740c7fe0 0x7f11740cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient(hunting): picked mon.noname-b con 0x7f11740c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:01:59.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11741536f0 0x7f11740cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient(hunting): picked mon.noname-c con 0x7f11741536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:01:59.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1174074c00 con 0x7f11740700f0 2026-02-17T06:01:59.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f11740c5680 con 0x7f11740c7fe0 2026-02-17T06:01:59.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:59.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1174074d80 con 0x7f11741536f0 2026-02-17T06:01:59.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:59.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117bf53640 10 monclient(hunting): authenticate will time out at 1372.489450s 2026-02-17T06:01:59.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f1179cc8640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f11740700f0 0x7f117406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.935 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f11794c7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11740c7fe0 0x7f11740cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117a4c9640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11741536f0 0x7f11740cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f11794c7640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11740c7fe0 0x7f11740cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43606/0 (socket says 10.20.193.16:43606) 2026-02-17T06:01:59.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f11794c7640 1 -- 10.20.193.16:0/2369501309 learned_addr learned my addr 10.20.193.16:0/2369501309 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:01:59.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f11794c7640 10 monclient(hunting): get_auth_request con 0x7f11740c7fe0 auth_method 0 2026-02-17T06:01:59.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f11794c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f11794c7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f11794c7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f1179cc8640 10 monclient(hunting): get_auth_request con 0x7f11740700f0 auth_method 0 2026-02-17T06:01:59.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f1179cc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f1179cc8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f1179cc8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117a4c9640 10 monclient(hunting): get_auth_request con 0x7f11741536f0 auth_method 0 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117a4c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117a4c9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.920+0000 7f117a4c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.921+0000 7f11794c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.921+0000 7f11794c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.921+0000 7f11794c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.921+0000 7f117a4c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.921+0000 7f117a4c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.921+0000 7f117a4c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.921+0000 7f1179cc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:59.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.921+0000 7f1179cc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.921+0000 7f1179cc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.922+0000 7f11794c7640 10 monclient(hunting): handle_auth_done global_id 5257 payload 824 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.922+0000 7f11794c7640 1 -- 10.20.193.16:0/2369501309 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11741536f0 msgr2=0x7f11740cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.922+0000 7f11794c7640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f11741536f0 0x7f11740cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.922+0000 7f11794c7640 1 -- 10.20.193.16:0/2369501309 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f11740700f0 msgr2=0x7f117406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.922+0000 7f11794c7640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f11740700f0 0x7f117406e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.922+0000 7f11794c7640 10 monclient: _finish_hunting 0 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.922+0000 7f11794c7640 1 monclient: found mon.noname-b 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.922+0000 7f11794c7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.922+0000 7f11794c7640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.922+0000 7f11794c7640 1 -- 10.20.193.16:0/2369501309 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1174145b30 con 0x7f11740c7fe0 2026-02-17T06:01:59.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.923+0000 7f11794c7640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11740c7fe0 0x7f11740cbeb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f116800c510 tx=0x7f116800c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.923+0000 7f11627fc640 10 client.?.objecter ms_handle_connect 0x7f11740c7fe0 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.923+0000 7f11627fc640 10 client.?.objecter resend_mon_ops 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 1 -- 10.20.193.16:0/2369501309 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168018070 con 0x7f11740c7fe0 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 10 monclient: dump: 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.939 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 1 -- 10.20.193.16:0/2369501309 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1168004cf0 con 0x7f11740c7fe0 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 1 -- 10.20.193.16:0/2369501309 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168007310 con 0x7f11740c7fe0 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:59.940 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f1163fff640 4 set_mon_vals no callback set 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f11627fc640 10 monclient: dump: 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:01:59.941 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 5 monclient: authenticate success, global_id 5257 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 10 monclient: _renew_subs 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1174145e10 con 0x7f11740c7fe0 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 10 client.5257.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 10 monclient: _renew_subs 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11740cb6b0 con 0x7f11740c7fe0 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 1 librados: init done 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 10 client.5257.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 15 client.5257.objecter _session_command_op_assign -1 1 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 20 client.5257.objecter _get_session osd=-1 returning homeless 2026-02-17T06:01:59.942 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 10 client.5257.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:01:59.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:01:59.943 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.925+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f11740704d0 con 0x7f11740c7fe0 2026-02-17T06:01:59.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.937+0000 7f11627fc640 1 -- 10.20.193.16:0/2369501309 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f11680074b0 con 0x7f11740c7fe0 2026-02-17T06:01:59.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.937+0000 7f11627fc640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1144079190 0x7f114407b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.945 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.937+0000 7f1179cc8640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1144079190 0x7f114407b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.937+0000 7f1179cc8640 10 monclient: get_auth_request con 0x7f1144079190 auth_method 0 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.937+0000 7f1179cc8640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1144079190 0x7f114407b650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1164039000 tx=0x7f1164033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.937+0000 7f11627fc640 10 client.5257.objecter ms_handle_connect 0x7f1144079190 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 1 -- 10.20.193.16:0/2369501309 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f116805d9b0 con 0x7f11740c7fe0 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 10 client.5257.objecter ms_dispatch2 0x7f1174000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 3 client.5257.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 3 client.5257.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 10 client.5257.objecter checking command 1 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] conn(0x7f1144081670 0x7f1144083a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 20 client.5257.objecter _get_session s=0x7f1144080fa0 osd=14 3 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 20 client.5257.objecter put_session s=0x7f1144080fa0 osd=14 3 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 15 client.5257.objecter _session_command_op_remove -1 1 2026-02-17T06:01:59.946 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 20 client.5257.objecter _get_session s=0x7f1144080fa0 osd=14 3 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 20 client.5257.objecter get_session s=0x7f1144080fa0 osd=14 3 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 15 client.5257.objecter _session_command_op_assign 14 1 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 20 client.5257.objecter put_session s=0x7f1144080fa0 osd=14 4 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 10 client.5257.objecter _send_command 1 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 1 -- 10.20.193.16:0/2369501309 --> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1168005e50 con 0x7f1144081670 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 20 client.5257.objecter dump_active .. 0 homeless 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 1 -- 10.20.193.16:0/2369501309 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f116805dd70 con 0x7f11740c7fe0 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f11627fc640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f117a4c9640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] conn(0x7f1144081670 0x7f1144083a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.939+0000 7f117a4c9640 10 monclient: get_auth_request con 0x7f1144081670 auth_method 0 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.941+0000 7f117a4c9640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] conn(0x7f1144081670 0x7f1144083a90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.14 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:59.947 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.941+0000 7f11627fc640 10 client.5257.objecter ms_handle_connect 0x7f1144081670 2026-02-17T06:01:59.951 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.952+0000 7f11627fc640 1 -- 10.20.193.16:0/2369501309 <== osd.14 v2:10.20.193.148:6800/1855767579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f1168005e50 con 0x7f1144081670 2026-02-17T06:01:59.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.952+0000 7f11627fc640 10 client.5257.objecter ms_dispatch2 0x7f1174000b90 command_reply(tid 1: 0 ) 2026-02-17T06:01:59.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.952+0000 7f11627fc640 10 client.5257.objecter _finish_command 1 = system:0 2026-02-17T06:01:59.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.952+0000 7f11627fc640 20 client.5257.objecter put_session s=0x7f1144080fa0 osd=14 4 2026-02-17T06:01:59.952 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.952+0000 7f11627fc640 15 client.5257.objecter _session_command_op_remove 14 1 2026-02-17T06:01:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 10 client.5257.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:01:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 15 client.5257.objecter _session_command_op_assign -1 2 2026-02-17T06:01:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 20 client.5257.objecter _get_session s=0x7f1144080fa0 osd=14 3 2026-02-17T06:01:59.958 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 20 client.5257.objecter put_session s=0x7f1144080fa0 osd=14 3 2026-02-17T06:01:59.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 20 client.5257.objecter _get_session s=0x7f1144080fa0 osd=14 3 2026-02-17T06:01:59.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 15 client.5257.objecter _session_command_op_remove -1 2 2026-02-17T06:01:59.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 20 client.5257.objecter get_session s=0x7f1144080fa0 osd=14 3 2026-02-17T06:01:59.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 15 client.5257.objecter _session_command_op_assign 14 2 2026-02-17T06:01:59.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 20 client.5257.objecter put_session s=0x7f1144080fa0 osd=14 4 2026-02-17T06:01:59.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 10 client.5257.objecter _send_command 2 2026-02-17T06:01:59.959 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.957+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 --> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f11741573f0 con 0x7f1144081670 2026-02-17T06:01:59.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f11627fc640 1 -- 10.20.193.16:0/2369501309 <== osd.14 v2:10.20.193.148:6800/1855767579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f11741573f0 con 0x7f1144081670 2026-02-17T06:01:59.964 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f11627fc640 10 client.5257.objecter ms_dispatch2 0x7f1174000b90 command_reply(tid 2: 0 ) 2026-02-17T06:01:59.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f11627fc640 10 client.5257.objecter _finish_command 2 = system:0 2026-02-17T06:01:59.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f11627fc640 20 client.5257.objecter put_session s=0x7f1144080fa0 osd=14 4 2026-02-17T06:01:59.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f11627fc640 15 client.5257.objecter _session_command_op_remove 14 2 2026-02-17T06:01:59.965 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:01:59.965 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:01:59.965 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:01:59.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 10 librados: watch_flush enter 2026-02-17T06:01:59.965 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 10 librados: watch_flush exit 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 10 client.5257.objecter close_session for osd.14 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 >> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] conn(0x7f1144081670 msgr2=0x7f1144083a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.148:6800/1855767579,v1:10.20.193.148:6801/1855767579] conn(0x7f1144081670 0x7f1144083a90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 20 client.5257.objecter put_session s=0x7f1144080fa0 osd=14 1 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 20 client.5257.objecter shutdown clearing up homeless session... 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 10 client.5257.objecter successfully canceled tick 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1144079190 msgr2=0x7f114407b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1144079190 0x7f114407b650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1164039000 tx=0x7f1164033000 comp rx=0 tx=0).stop 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 10 monclient: shutdown 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11740c7fe0 msgr2=0x7f11740cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 --2- 10.20.193.16:0/2369501309 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f11740c7fe0 0x7f11740cbeb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f116800c510 tx=0x7f116800c9e0 comp rx=0 tx=0).stop 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117a4c9640 1 -- 10.20.193.16:0/2369501309 reap_dead start 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 shutdown_connections 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 >> 10.20.193.16:0/2369501309 conn(0x7f11740cd770 msgr2=0x7f1174148c40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 shutdown_connections 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 -- 10.20.193.16:0/2369501309 wait complete. 2026-02-17T06:01:59.966 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:01:59.964+0000 7f117bf53640 1 librados: shutdown 2026-02-17T06:01:59.972 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2026-02-17T06:02:00.040 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient: get_monmap_and_config 2026-02-17T06:02:00.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient: build_initial_monmap 2026-02-17T06:02:00.041 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient: monmap: 2026-02-17T06:02:00.041 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:00.041 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.041 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:00.041072+0000 2026-02-17T06:02:00.041 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:00.041072+0000 2026-02-17T06:02:00.041 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:00.041 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.042 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:00.042 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:00.042 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:00.042 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:00.042 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:00.042 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:00.042 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 1 Processor -- start 2026-02-17T06:02:00.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 1 -- start start 2026-02-17T06:02:00.042 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient: init 2026-02-17T06:02:00.050 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient: _add_conns ranks=[4,5,0,2,3,1] 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1eb00700f0 0x7f1eb00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient(hunting): picked mon.noname-b con 0x7f1eb00700f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f1eb00c7fe0 0x7f1eb00c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient(hunting): picked mon.noname-c con 0x7f1eb00c7fe0 addr v2:10.20.193.148:3300/0 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1eb01536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb00c8970 con 0x7f1eb01536f0 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb00c4820 con 0x7f1eb00700f0 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.040+0000 7f1eb826c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.051 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb826c640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb00c4b00 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb826c640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb826c640 10 monclient(hunting): authenticate will time out at 1372.610006s 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb5fe1640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1eb00700f0 0x7f1eb00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb67e2640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f1eb01536f0 0x7f1eb0153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:41940/0 (socket says 10.20.193.16:41940) 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb67e2640 1 -- 10.20.193.16:0/812764268 learned_addr learned my addr 10.20.193.16:0/812764268 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb5fe1640 10 monclient(hunting): get_auth_request con 0x7f1eb00700f0 auth_method 0 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb5fe1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb5fe1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb5fe1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb57e0640 1 --2- 10.20.193.16:0/812764268 >> v2:10.20.193.148:3300/0 conn(0x7f1eb00c7fe0 0x7f1eb00c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb57e0640 10 monclient(hunting): get_auth_request con 0x7f1eb00c7fe0 auth_method 0 2026-02-17T06:02:00.052 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb57e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb57e0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb57e0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb5fe1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb5fe1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.041+0000 7f1eb5fe1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb57e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb57e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb57e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb5fe1640 10 monclient(hunting): handle_auth_done global_id 5260 payload 274 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb5fe1640 1 -- 10.20.193.16:0/812764268 >> v2:10.20.193.148:3300/0 conn(0x7f1eb00c7fe0 msgr2=0x7f1eb00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb5fe1640 1 --2- 10.20.193.16:0/812764268 >> v2:10.20.193.148:3300/0 conn(0x7f1eb00c7fe0 0x7f1eb00c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.053 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb5fe1640 1 -- 10.20.193.16:0/812764268 >> v1:10.20.193.16:6789/0 conn(0x7f1eb01536f0 legacy=0x7f1eb0153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb5fe1640 10 monclient: _finish_hunting 0 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb5fe1640 1 monclient: found mon.noname-b 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb5fe1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb5fe1640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.042+0000 7f1eb5fe1640 1 -- 10.20.193.16:0/812764268 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb00c4de0 con 0x7f1eb00700f0 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.043+0000 7f1eb5fe1640 1 --2- 10.20.193.16:0/812764268 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00700f0 0x7f1eb00704d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00c5b80 tx=0x7f1ea002ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c9196bb623a015 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 1 -- 10.20.193.16:0/812764268 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea003c070 con 0x7f1eb00700f0 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 10 monclient: dump: 2026-02-17T06:02:00.054 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 10 monclient: _finish_auth 0 2026-02-17T06:02:00.055 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 10 monclient: _check_auth_tickets 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 5 monclient: authenticate success, global_id 5260 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 1 -- 10.20.193.16:0/812764268 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea002fa60 con 0x7f1eb00700f0 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 1 -- 10.20.193.16:0/812764268 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea002fe30 con 0x7f1eb00700f0 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 4 set_mon_vals no callback set 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb4fdf640 10 monclient: dump: 2026-02-17T06:02:00.056 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 10 monclient: shutdown 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:00.057 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 1 -- 10.20.193.16:0/812764268 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00700f0 msgr2=0x7f1eb00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 1 --2- 10.20.193.16:0/812764268 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00700f0 0x7f1eb00704d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00c5b80 tx=0x7f1ea002ed50 comp rx=0 tx=0).stop 2026-02-17T06:02:00.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 1 -- 10.20.193.16:0/812764268 shutdown_connections 2026-02-17T06:02:00.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 1 --2- 10.20.193.16:0/812764268 >> v2:10.20.193.148:3300/0 conn(0x7f1eb00c7fe0 0x7f1eb00c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 1 --2- 10.20.193.16:0/812764268 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00700f0 0x7f1eb00704d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 1 -- 10.20.193.16:0/812764268 >> 10.20.193.16:0/812764268 conn(0x7f1eb00cd770 msgr2=0x7f1eb00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:00.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 1 -- 10.20.193.16:0/812764268 shutdown_connections 2026-02-17T06:02:00.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.045+0000 7f1eb826c640 1 -- 10.20.193.16:0/812764268 wait complete. 2026-02-17T06:02:00.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient: build_initial_monmap 2026-02-17T06:02:00.058 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient: monmap: 2026-02-17T06:02:00.058 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 librados: starting msgr at 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 librados: starting objecter 2026-02-17T06:02:00.059 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 Processor -- start 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 -- start start 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 librados: setting wanted keys 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 librados: calling monclient init 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient: init 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1eb00700f0 0x7f1eb0072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient(hunting): picked mon.noname-a con 0x7f1eb00700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00c7fe0 0x7f1eb0148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient(hunting): picked mon.noname-b con 0x7f1eb00c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1eb01536f0 0x7f1eb0151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.060 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient(hunting): picked mon.noname-c con 0x7f1eb01536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb00c5680 con 0x7f1eb00700f0 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb00c53a0 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0074c00 con 0x7f1eb01536f0 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.046+0000 7f1eb826c640 10 monclient(hunting): authenticate will time out at 1372.615705s 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb5fe1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1eb00700f0 0x7f1eb0072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb57e0640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00c7fe0 0x7f1eb0148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb67e2640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1eb01536f0 0x7f1eb0151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb57e0640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00c7fe0 0x7f1eb0148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:43628/0 (socket says 10.20.193.16:43628) 2026-02-17T06:02:00.061 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb57e0640 1 -- 10.20.193.16:0/2288890615 learned_addr learned my addr 10.20.193.16:0/2288890615 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb57e0640 10 monclient(hunting): get_auth_request con 0x7f1eb00c7fe0 auth_method 0 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb57e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb57e0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb57e0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb5fe1640 10 monclient(hunting): get_auth_request con 0x7f1eb00700f0 auth_method 0 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb5fe1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb5fe1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb5fe1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb67e2640 10 monclient(hunting): get_auth_request con 0x7f1eb01536f0 auth_method 0 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb67e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb67e2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb67e2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.062 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb57e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb57e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb57e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb5fe1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb5fe1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb5fe1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb67e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb67e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.047+0000 7f1eb67e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.048+0000 7f1eb57e0640 10 monclient(hunting): handle_auth_done global_id 5263 payload 824 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.048+0000 7f1eb57e0640 1 -- 10.20.193.16:0/2288890615 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1eb01536f0 msgr2=0x7f1eb0151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.063 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.048+0000 7f1eb57e0640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1eb01536f0 0x7f1eb0151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.048+0000 7f1eb57e0640 1 -- 10.20.193.16:0/2288890615 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1eb00700f0 msgr2=0x7f1eb0072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.048+0000 7f1eb57e0640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1eb00700f0 0x7f1eb0072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.048+0000 7f1eb57e0640 10 monclient: _finish_hunting 0 2026-02-17T06:02:00.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.048+0000 7f1eb57e0640 1 monclient: found mon.noname-b 2026-02-17T06:02:00.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.048+0000 7f1eb57e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:00.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.048+0000 7f1eb57e0640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.048+0000 7f1eb57e0640 1 -- 10.20.193.16:0/2288890615 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb0072a80 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.049+0000 7f1eb57e0640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00c7fe0 0x7f1eb0148e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400ebe0 tx=0x7f1ea400c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.049+0000 7f1e9e7fc640 10 client.?.objecter ms_handle_connect 0x7f1eb00c7fe0 2026-02-17T06:02:00.064 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.049+0000 7f1e9e7fc640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:00.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1e9e7fc640 1 -- 10.20.193.16:0/2288890615 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4017070 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1e9e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.067 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1e9e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1e9e7fc640 10 monclient: dump: 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.068 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1e9e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1e9e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1e9e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1eb826c640 5 monclient: authenticate success, global_id 5263 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1eb826c640 10 monclient: _renew_subs 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1eb826c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.050+0000 7f1eb826c640 1 -- 10.20.193.16:0/2288890615 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb0072d60 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1eb826c640 10 client.5263.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1eb826c640 10 monclient: _renew_subs 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1eb826c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1eb826c640 1 -- 10.20.193.16:0/2288890615 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb0073320 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1eb826c640 1 librados: init done 2026-02-17T06:02:00.069 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1e9e7fc640 1 -- 10.20.193.16:0/2288890615 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea400cc40 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1e9e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1e9e7fc640 1 -- 10.20.193.16:0/2288890615 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4021060 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1e9e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1e9e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1e9e7fc640 10 monclient: dump: 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.070 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1e9ffff640 4 set_mon_vals no callback set 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1eb826c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1eb826c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1eb826c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.051+0000 7f1eb826c640 1 -- 10.20.193.16:0/2288890615 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb00704d0 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.062+0000 7f1e9e7fc640 1 -- 10.20.193.16:0/2288890615 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1ea4026480 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.062+0000 7f1e9e7fc640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1e80079140 0x7f1e8007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.062+0000 7f1eb5fe1640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1e80079140 0x7f1e8007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.062+0000 7f1eb5fe1640 10 monclient: get_auth_request con 0x7f1e80079140 auth_method 0 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.063+0000 7f1eb5fe1640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1e80079140 0x7f1e8007b600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0039000 tx=0x7f1ea0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.071 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.063+0000 7f1e9e7fc640 10 client.5263.objecter ms_handle_connect 0x7f1e80079140 2026-02-17T06:02:00.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.063+0000 7f1e9e7fc640 1 -- 10.20.193.16:0/2288890615 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1ea405e820 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.063+0000 7f1e9e7fc640 10 client.5263.objecter ms_dispatch2 0x7f1eb0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:00.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.063+0000 7f1e9e7fc640 3 client.5263.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:00.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.063+0000 7f1e9e7fc640 3 client.5263.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:00.072 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.063+0000 7f1e9e7fc640 20 client.5263.objecter dump_active .. 0 homeless 2026-02-17T06:02:00.165 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.165+0000 7f1e9e7fc640 1 -- 10.20.193.16:0/2288890615 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1ea40627e0 con 0x7f1eb00c7fe0 2026-02-17T06:02:00.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.165+0000 7f1e9e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:00.166 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.165+0000 7f1e9e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:00.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.246+0000 7f1eb826c640 1 -- 10.20.193.16:0/2288890615 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "14", "target": ["mon-mgr", ""]}) -- 0x7f1eb014a210 con 0x7f1e80079140 2026-02-17T06:02:00.247 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.246+0000 7f1e9e7fc640 1 -- 10.20.193.16:0/2288890615 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 14 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7f1eb014a210 con 0x7f1e80079140 2026-02-17T06:02:00.247 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 14 to deep-scrub 2026-02-17T06:02:00.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 10 librados: watch_flush enter 2026-02-17T06:02:00.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 10 librados: watch_flush exit 2026-02-17T06:02:00.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 20 client.5263.objecter shutdown clearing up homeless session... 2026-02-17T06:02:00.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 10 client.5263.objecter successfully canceled tick 2026-02-17T06:02:00.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 -- 10.20.193.16:0/2288890615 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1e80079140 msgr2=0x7f1e8007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1e80079140 0x7f1e8007b600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0039000 tx=0x7f1ea0033000 comp rx=0 tx=0).stop 2026-02-17T06:02:00.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 10 monclient: shutdown 2026-02-17T06:02:00.249 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 -- 10.20.193.16:0/2288890615 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00c7fe0 msgr2=0x7f1eb0148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00c7fe0 0x7f1eb0148e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400ebe0 tx=0x7f1ea400c520 comp rx=0 tx=0).stop 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 -- 10.20.193.16:0/2288890615 shutdown_connections 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f1e80079140 0x7f1e8007b600 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1eb00c7fe0 0x7f1eb0148e00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1eb01536f0 0x7f1eb0151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 --2- 10.20.193.16:0/2288890615 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1eb00700f0 0x7f1eb0072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 -- 10.20.193.16:0/2288890615 >> 10.20.193.16:0/2288890615 conn(0x7f1eb00cd770 msgr2=0x7f1eb00cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 -- 10.20.193.16:0/2288890615 shutdown_connections 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 -- 10.20.193.16:0/2288890615 wait complete. 2026-02-17T06:02:00.250 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.247+0000 7f1eb826c640 1 librados: shutdown 2026-02-17T06:02:00.256 INFO:tasks.ceph:Scrubbing osd.3 2026-02-17T06:02:00.257 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.324+0000 7f728a3ce640 10 monclient: get_monmap_and_config 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.324+0000 7f728a3ce640 10 monclient: build_initial_monmap 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.324+0000 7f728a3ce640 10 monclient: monmap: 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:00.325979+0000 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:00.325979+0000 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:00.325 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:00.326 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:00.326 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:00.326 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.325+0000 7f728a3ce640 1 Processor -- start 2026-02-17T06:02:00.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.325+0000 7f728a3ce640 1 -- start start 2026-02-17T06:02:00.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.325+0000 7f728a3ce640 10 monclient: init 2026-02-17T06:02:00.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:00.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 10 monclient: _add_conns ranks=[4,3,1,2,0,5] 2026-02-17T06:02:00.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f728414f020 0x7f728406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 10 monclient(hunting): picked mon.noname-b con 0x7f728414f020 addr v2:10.20.193.128:3300/0 2026-02-17T06:02:00.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f728406e120 0x7f728415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 10 monclient(hunting): picked mon.noname-a con 0x7f728406e120 addr v2:10.20.193.16:3300/0 2026-02-17T06:02:00.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f728414f400 addr v1:10.20.193.128:6789/0 2026-02-17T06:02:00.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72840c5b10 con 0x7f728414f400 2026-02-17T06:02:00.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f728406e620 con 0x7f728406e120 2026-02-17T06:02:00.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f728406e900 con 0x7f728414f020 2026-02-17T06:02:00.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:00.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f728a3ce640 10 monclient(hunting): authenticate will time out at 1372.895156s 2026-02-17T06:02:00.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f72837fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f728406e120 0x7f728415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f7283fff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f728414f020 0x7f728406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f72837fe640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f728406e120 0x7f728415c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:43598/0 (socket says 10.20.193.16:43598) 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f72837fe640 1 -- 10.20.193.16:0/2062214994 learned_addr learned my addr 10.20.193.16:0/2062214994 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f72837fe640 10 monclient(hunting): get_auth_request con 0x7f728406e120 auth_method 0 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f72837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f72837fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.326+0000 7f72837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.327+0000 7f7283fff640 10 monclient(hunting): get_auth_request con 0x7f728414f020 auth_method 0 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.327+0000 7f7283fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.327+0000 7f7283fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.327+0000 7f7283fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.327+0000 7f72837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.327+0000 7f72837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.327+0000 7f72837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.327+0000 7f7283fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.327+0000 7f7283fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.327+0000 7f7283fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.328+0000 7f72837fe640 10 monclient(hunting): handle_auth_done global_id 5181 payload 274 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.328+0000 7f72837fe640 1 -- 10.20.193.16:0/2062214994 >> v2:10.20.193.128:3300/0 conn(0x7f728414f020 msgr2=0x7f728406dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.328+0000 7f72837fe640 1 --2- 10.20.193.16:0/2062214994 >> v2:10.20.193.128:3300/0 conn(0x7f728414f020 0x7f728406dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.328+0000 7f72837fe640 1 -- 10.20.193.16:0/2062214994 >> v1:10.20.193.128:6789/0 conn(0x7f728414f400 legacy=0x7f72840c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.328+0000 7f72837fe640 10 monclient: _finish_hunting 0 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.328+0000 7f72837fe640 1 monclient: found mon.noname-a 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.328+0000 7f72837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.328+0000 7f72837fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.328+0000 7f72837fe640 1 -- 10.20.193.16:0/2062214994 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728406ebe0 con 0x7f728406e120 2026-02-17T06:02:00.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.329+0000 7f72837fe640 1 --2- 10.20.193.16:0/2062214994 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f728406e120 0x7f728415c3e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f72840ce3c0 tx=0x7f727402ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa9ec52e90f95a7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 1 -- 10.20.193.16:0/2062214994 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727403c070 con 0x7f728406e120 2026-02-17T06:02:00.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:00.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 10 monclient: dump: 2026-02-17T06:02:00.340 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.340 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.340 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.340 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.340 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.340 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 10 monclient: _finish_auth 0 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f728a3ce640 5 monclient: authenticate success, global_id 5181 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f728a3ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 1 -- 10.20.193.16:0/2062214994 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f727402fc70 con 0x7f728406e120 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:00.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 1 -- 10.20.193.16:0/2062214994 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274037660 con 0x7f728406e120 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f7282ffd640 10 monclient: dump: 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.342 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f728a3ce640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.330+0000 7f728a3ce640 4 set_mon_vals no callback set 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 10 monclient: shutdown 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 1 -- 10.20.193.16:0/2062214994 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f728406e120 msgr2=0x7f728415c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 1 --2- 10.20.193.16:0/2062214994 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f728406e120 0x7f728415c3e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f72840ce3c0 tx=0x7f727402ef60 comp rx=0 tx=0).stop 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 1 -- 10.20.193.16:0/2062214994 shutdown_connections 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 1 --2- 10.20.193.16:0/2062214994 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f728406e120 0x7f728415c3e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 1 --2- 10.20.193.16:0/2062214994 >> v2:10.20.193.128:3300/0 conn(0x7f728414f020 0x7f728406dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 1 -- 10.20.193.16:0/2062214994 >> 10.20.193.16:0/2062214994 conn(0x7f72840cd770 msgr2=0x7f72840cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:00.343 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 1 -- 10.20.193.16:0/2062214994 shutdown_connections 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 1 -- 10.20.193.16:0/2062214994 wait complete. 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 10 monclient: build_initial_monmap 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.331+0000 7f728a3ce640 10 monclient: monmap: 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:00.344 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 librados: starting msgr at 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 librados: starting objecter 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 Processor -- start 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 -- start start 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 librados: setting wanted keys 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 librados: calling monclient init 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient: init 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f728406e120 0x7f728413e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient(hunting): picked mon.noname-a con 0x7f728406e120 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f728414f020 0x7f7284156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.345 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient(hunting): picked mon.noname-c con 0x7f728414f020 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f728414f400 0x7f7284157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient(hunting): picked mon.noname-b con 0x7f728414f400 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72840c6720 con 0x7f728406e120 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f728406f1a0 con 0x7f728414f400 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f72840c68a0 con 0x7f728414f020 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f728a3ce640 10 monclient(hunting): authenticate will time out at 1372.901627s 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.332+0000 7f7283fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f728406e120 0x7f728413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.346 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f72837fe640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f728414f020 0x7f7284156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7283fff640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f728406e120 0x7f728413e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:43604/0 (socket says 10.20.193.16:43604) 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7283fff640 1 -- 10.20.193.16:0/510668718 learned_addr learned my addr 10.20.193.16:0/510668718 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7283fff640 10 monclient(hunting): get_auth_request con 0x7f728406e120 auth_method 0 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7283fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7283fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7283fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7288944640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f728414f400 0x7f7284157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7288944640 10 monclient(hunting): get_auth_request con 0x7f728414f400 auth_method 0 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7288944640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7288944640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7288944640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7283fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.347 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7283fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7283fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f72837fe640 10 monclient(hunting): get_auth_request con 0x7f728414f020 auth_method 0 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f72837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f72837fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f72837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7288944640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7288944640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f7288944640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f72837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f72837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.333+0000 7f72837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.348 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.334+0000 7f7283fff640 10 monclient(hunting): handle_auth_done global_id 5184 payload 824 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.334+0000 7f7283fff640 1 -- 10.20.193.16:0/510668718 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f728414f020 msgr2=0x7f7284156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.334+0000 7f7283fff640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f728414f020 0x7f7284156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.334+0000 7f7283fff640 1 -- 10.20.193.16:0/510668718 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f728414f400 msgr2=0x7f7284157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.334+0000 7f7283fff640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f728414f400 0x7f7284157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.334+0000 7f7283fff640 10 monclient: _finish_hunting 0 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.334+0000 7f7283fff640 1 monclient: found mon.noname-a 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.334+0000 7f7283fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.334+0000 7f7283fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.334+0000 7f7283fff640 1 -- 10.20.193.16:0/510668718 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72841f4f70 con 0x7f728406e120 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.335+0000 7f7283fff640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f728406e120 0x7f728413e370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f727000e890 tx=0x7f727000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.335+0000 7f7280ff9640 10 client.?.objecter ms_handle_connect 0x7f728406e120 2026-02-17T06:02:00.349 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.335+0000 7f7280ff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 1 -- 10.20.193.16:0/510668718 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270017070 con 0x7f728406e120 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 10 monclient: dump: 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.350 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 10 monclient: _finish_auth 0 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 1 -- 10.20.193.16:0/510668718 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f727000c940 con 0x7f728406e120 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 5 monclient: authenticate success, global_id 5184 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 10 monclient: _renew_subs 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72841f5210 con 0x7f728406e120 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 1 -- 10.20.193.16:0/510668718 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727000cd30 con 0x7f728406e120 2026-02-17T06:02:00.351 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f72827fc640 4 set_mon_vals no callback set 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f7280ff9640 10 monclient: dump: 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.352 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 10 client.5184.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 10 monclient: _renew_subs 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72841f57d0 con 0x7f728406e120 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 1 librados: init done 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 10 client.5184.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 15 client.5184.objecter _session_command_op_assign -1 1 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 20 client.5184.objecter _get_session osd=-1 returning homeless 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 10 client.5184.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.353 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.337+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f728406dbe0 con 0x7f728406e120 2026-02-17T06:02:00.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.349+0000 7f7280ff9640 1 -- 10.20.193.16:0/510668718 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f7270026480 con 0x7f728406e120 2026-02-17T06:02:00.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.349+0000 7f7280ff9640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7254079170 0x7f725407b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.349+0000 7f72837fe640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7254079170 0x7f725407b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.356 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.349+0000 7f72837fe640 10 monclient: get_auth_request con 0x7f7254079170 auth_method 0 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f72837fe640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7254079170 0x7f725407b630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f72840c9d60 tx=0x7f7274039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 10 client.5184.objecter ms_handle_connect 0x7f7254079170 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 1 -- 10.20.193.16:0/510668718 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f7270097f20 con 0x7f728406e120 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 10 client.5184.objecter ms_dispatch2 0x7f7284000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 3 client.5184.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 3 client.5184.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 10 client.5184.objecter checking command 1 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x7f72540816b0 0x7f7254083ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 20 client.5184.objecter _get_session s=0x7f7254080fe0 osd=3 3 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 20 client.5184.objecter put_session s=0x7f7254080fe0 osd=3 3 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 15 client.5184.objecter _session_command_op_remove -1 1 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 20 client.5184.objecter _get_session s=0x7f7254080fe0 osd=3 3 2026-02-17T06:02:00.357 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 20 client.5184.objecter get_session s=0x7f7254080fe0 osd=3 3 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 15 client.5184.objecter _session_command_op_assign 3 1 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 20 client.5184.objecter put_session s=0x7f7254080fe0 osd=3 4 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 10 client.5184.objecter _send_command 1 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 1 -- 10.20.193.16:0/510668718 --> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72540841a0 con 0x7f72540816b0 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.350+0000 7f7280ff9640 20 client.5184.objecter dump_active .. 0 homeless 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.351+0000 7f7288944640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x7f72540816b0 0x7f7254083ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.351+0000 7f7288944640 10 monclient: get_auth_request con 0x7f72540816b0 auth_method 0 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.351+0000 7f7280ff9640 1 -- 10.20.193.16:0/510668718 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f727005e950 con 0x7f728406e120 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.351+0000 7f7280ff9640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.353+0000 7f7280ff9640 10 client.5184.objecter ms_handle_connect 0x7f72540816b0 2026-02-17T06:02:00.358 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.353+0000 7f7288944640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x7f72540816b0 0x7f7254083ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.364+0000 7f7280ff9640 1 -- 10.20.193.16:0/510668718 <== osd.3 v2:10.20.193.164:6824/3772404125 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f72540841a0 con 0x7f72540816b0 2026-02-17T06:02:00.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.364+0000 7f7280ff9640 10 client.5184.objecter ms_dispatch2 0x7f7284000b90 command_reply(tid 1: 0 ) 2026-02-17T06:02:00.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.364+0000 7f7280ff9640 10 client.5184.objecter _finish_command 1 = system:0 2026-02-17T06:02:00.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.364+0000 7f7280ff9640 20 client.5184.objecter put_session s=0x7f7254080fe0 osd=3 4 2026-02-17T06:02:00.364 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.364+0000 7f7280ff9640 15 client.5184.objecter _session_command_op_remove 3 1 2026-02-17T06:02:00.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 10 client.5184.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:02:00.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 15 client.5184.objecter _session_command_op_assign -1 2 2026-02-17T06:02:00.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 20 client.5184.objecter _get_session s=0x7f7254080fe0 osd=3 3 2026-02-17T06:02:00.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 20 client.5184.objecter put_session s=0x7f7254080fe0 osd=3 3 2026-02-17T06:02:00.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 20 client.5184.objecter _get_session s=0x7f7254080fe0 osd=3 3 2026-02-17T06:02:00.370 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 15 client.5184.objecter _session_command_op_remove -1 2 2026-02-17T06:02:00.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 20 client.5184.objecter get_session s=0x7f7254080fe0 osd=3 3 2026-02-17T06:02:00.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 15 client.5184.objecter _session_command_op_assign 3 2 2026-02-17T06:02:00.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 20 client.5184.objecter put_session s=0x7f7254080fe0 osd=3 4 2026-02-17T06:02:00.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 10 client.5184.objecter _send_command 2 2026-02-17T06:02:00.371 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.369+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 --> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7284156710 con 0x7f72540816b0 2026-02-17T06:02:00.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f7280ff9640 1 -- 10.20.193.16:0/510668718 <== osd.3 v2:10.20.193.164:6824/3772404125 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7284156710 con 0x7f72540816b0 2026-02-17T06:02:00.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f7280ff9640 10 client.5184.objecter ms_dispatch2 0x7f7284000b90 command_reply(tid 2: 0 ) 2026-02-17T06:02:00.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f7280ff9640 10 client.5184.objecter _finish_command 2 = system:0 2026-02-17T06:02:00.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f7280ff9640 20 client.5184.objecter put_session s=0x7f7254080fe0 osd=3 4 2026-02-17T06:02:00.377 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f7280ff9640 15 client.5184.objecter _session_command_op_remove 3 2 2026-02-17T06:02:00.378 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:02:00.378 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:02:00.378 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:02:00.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 10 librados: watch_flush enter 2026-02-17T06:02:00.379 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 10 librados: watch_flush exit 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 10 client.5184.objecter close_session for osd.3 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x7f72540816b0 msgr2=0x7f7254083ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.164:6824/3772404125,v1:10.20.193.164:6825/3772404125] conn(0x7f72540816b0 0x7f7254083ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 20 client.5184.objecter put_session s=0x7f7254080fe0 osd=3 1 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 20 client.5184.objecter shutdown clearing up homeless session... 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 10 client.5184.objecter successfully canceled tick 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7254079170 msgr2=0x7f725407b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f7254079170 0x7f725407b630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f72840c9d60 tx=0x7f7274039040 comp rx=0 tx=0).stop 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 10 monclient: shutdown 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:00.380 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f728406e120 msgr2=0x7f728413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 1 --2- 10.20.193.16:0/510668718 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f728406e120 0x7f728413e370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f727000e890 tx=0x7f727000ed60 comp rx=0 tx=0).stop 2026-02-17T06:02:00.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f7288944640 1 -- 10.20.193.16:0/510668718 reap_dead start 2026-02-17T06:02:00.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 shutdown_connections 2026-02-17T06:02:00.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 >> 10.20.193.16:0/510668718 conn(0x7f72840cd770 msgr2=0x7f728414dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:00.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.376+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 shutdown_connections 2026-02-17T06:02:00.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.377+0000 7f728a3ce640 1 -- 10.20.193.16:0/510668718 wait complete. 2026-02-17T06:02:00.381 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.377+0000 7f728a3ce640 1 librados: shutdown 2026-02-17T06:02:00.385 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-17T06:02:00.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.453+0000 7f86bc355640 10 monclient: get_monmap_and_config 2026-02-17T06:02:00.454 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.453+0000 7f86bc355640 10 monclient: build_initial_monmap 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.453+0000 7f86bc355640 10 monclient: monmap: 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:00.454528+0000 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:00.454528+0000 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:00.455 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:00.456 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:00.456 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:00.456 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.453+0000 7f86bc355640 1 Processor -- start 2026-02-17T06:02:00.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.453+0000 7f86bc355640 1 -- start start 2026-02-17T06:02:00.456 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.453+0000 7f86bc355640 10 monclient: init 2026-02-17T06:02:00.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:00.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 10 monclient: _add_conns ranks=[1,4,2,0,3,5] 2026-02-17T06:02:00.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f86b40700f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:02:00.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f86b40c86f0 0x7f86b40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 10 monclient(hunting): picked mon.noname-b con 0x7f86b40c86f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:02:00.459 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f86b41536f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b40c4820 con 0x7f86b40700f0 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b40c4b00 con 0x7f86b41536f0 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f86b40c4de0 con 0x7f86b40c86f0 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86bc355640 10 monclient(hunting): authenticate will time out at 1373.023484s 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.454+0000 7f86ba8cb640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f86b41536f0 0x7f86b4153b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:36234/0 (socket says 10.20.193.16:36234) 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.455+0000 7f86ba8cb640 1 -- 10.20.193.16:0/4135097824 learned_addr learned my addr 10.20.193.16:0/4135097824 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.455+0000 7f86b98c9640 1 --2- 10.20.193.16:0/4135097824 >> v2:10.20.193.128:3300/0 conn(0x7f86b40c86f0 0x7f86b40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.455+0000 7f86b98c9640 10 monclient(hunting): get_auth_request con 0x7f86b40c86f0 auth_method 0 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.455+0000 7f86b98c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.460 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.455+0000 7f86b98c9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.455+0000 7f86b98c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.455+0000 7f86b98c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.455+0000 7f86b98c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.455+0000 7f86b98c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.456+0000 7f86b90c8640 1 -- 10.20.193.16:0/4135097824 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547000886 0 0) 0x7f86b40c4b00 con 0x7f86b41536f0 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.456+0000 7f86b90c8640 10 monclient(hunting): _negotiate 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.456+0000 7f86b90c8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.456+0000 7f86b90c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.456+0000 7f86b90c8640 10 monclient(hunting): my global_id is 5324 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.456+0000 7f86b90c8640 1 -- 10.20.193.16:0/4135097824 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f869c003620 con 0x7f86b41536f0 2026-02-17T06:02:00.461 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.457+0000 7f86b98c9640 10 monclient(hunting): handle_auth_done global_id 5275 payload 274 2026-02-17T06:02:00.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.457+0000 7f86b98c9640 1 -- 10.20.193.16:0/4135097824 >> v1:10.20.193.148:6789/0 conn(0x7f86b41536f0 legacy=0x7f86b4153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.457+0000 7f86b98c9640 1 -- 10.20.193.16:0/4135097824 >> v1:10.20.193.128:6789/0 conn(0x7f86b40700f0 legacy=0x7f86b40c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.457+0000 7f86b98c9640 10 monclient: _finish_hunting 0 2026-02-17T06:02:00.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.457+0000 7f86b98c9640 1 monclient: found mon.noname-b 2026-02-17T06:02:00.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.457+0000 7f86b98c9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:00.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.457+0000 7f86b98c9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.462 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.457+0000 7f86b98c9640 1 -- 10.20.193.16:0/4135097824 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b40c50c0 con 0x7f86b40c86f0 2026-02-17T06:02:00.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.459+0000 7f86b98c9640 1 --2- 10.20.193.16:0/4135097824 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f86b40c86f0 0x7f86b40704d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f86a80098a0 tx=0x7f86a802edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7166e4c46ef7e8c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 1 -- 10.20.193.16:0/4135097824 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a803c070 con 0x7f86b40c86f0 2026-02-17T06:02:00.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:00.468 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 10 monclient: dump: 2026-02-17T06:02:00.468 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.468 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.468 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.468 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 10 monclient: _finish_auth 0 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 10 monclient: _check_auth_tickets 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 1 -- 10.20.193.16:0/4135097824 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a802fae0 con 0x7f86b40c86f0 2026-02-17T06:02:00.469 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86bc355640 5 monclient: authenticate success, global_id 5275 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 1 -- 10.20.193.16:0/4135097824 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8037440 con 0x7f86b40c86f0 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86bc355640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86bc355640 4 set_mon_vals no callback set 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86b90c8640 10 monclient: dump: 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.470 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.471 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.471 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.471 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.471 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.471 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.471 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86bc355640 10 monclient: shutdown 2026-02-17T06:02:00.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86bc355640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:00.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.461+0000 7f86bc355640 1 -- 10.20.193.16:0/4135097824 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f86b40c86f0 msgr2=0x7f86b40704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.471 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 1 --2- 10.20.193.16:0/4135097824 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f86b40c86f0 0x7f86b40704d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f86a80098a0 tx=0x7f86a802edd0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 1 -- 10.20.193.16:0/4135097824 shutdown_connections 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 1 --2- 10.20.193.16:0/4135097824 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f86b40c86f0 0x7f86b40704d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 1 -- 10.20.193.16:0/4135097824 >> 10.20.193.16:0/4135097824 conn(0x7f86b40cd770 msgr2=0x7f86b40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 1 -- 10.20.193.16:0/4135097824 shutdown_connections 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 1 -- 10.20.193.16:0/4135097824 wait complete. 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 10 monclient: build_initial_monmap 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 10 monclient: monmap: 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:00.472 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 1 librados: starting msgr at 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 1 librados: starting objecter 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.462+0000 7f86bc355640 1 Processor -- start 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 1 -- start start 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 1 librados: setting wanted keys 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 1 librados: calling monclient init 2026-02-17T06:02:00.473 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient: init 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f86b40700f0 0x7f86b406ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient(hunting): picked mon.noname-a con 0x7f86b40700f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f86b40c86f0 0x7f86b41e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient(hunting): picked mon.noname-c con 0x7f86b40c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f86b41536f0 0x7f86b40736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient(hunting): picked mon.noname-b con 0x7f86b41536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86b4074c00 con 0x7f86b40700f0 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.474 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f86b40c5680 con 0x7f86b41536f0 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f86b4074d80 con 0x7f86b40c86f0 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86bc355640 10 monclient(hunting): authenticate will time out at 1373.032398s 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba0ca640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f86b40700f0 0x7f86b406ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba8cb640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f86b41536f0 0x7f86b40736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86b98c9640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f86b40c86f0 0x7f86b41e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba8cb640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f86b41536f0 0x7f86b40736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:51456/0 (socket says 10.20.193.16:51456) 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba8cb640 1 -- 10.20.193.16:0/740205052 learned_addr learned my addr 10.20.193.16:0/740205052 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba8cb640 10 monclient(hunting): get_auth_request con 0x7f86b41536f0 auth_method 0 2026-02-17T06:02:00.475 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba8cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba8cb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba8cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba0ca640 10 monclient(hunting): get_auth_request con 0x7f86b40700f0 auth_method 0 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba0ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba0ca640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.463+0000 7f86ba0ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86b98c9640 10 monclient(hunting): get_auth_request con 0x7f86b40c86f0 auth_method 0 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86b98c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86b98c9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86b98c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86ba8cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86ba8cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.476 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86ba8cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86ba0ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86ba0ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86ba0ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86b98c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86b98c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.464+0000 7f86b98c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.465+0000 7f86ba0ca640 10 monclient(hunting): handle_auth_done global_id 5187 payload 824 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.465+0000 7f86ba0ca640 1 -- 10.20.193.16:0/740205052 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f86b40c86f0 msgr2=0x7f86b41e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.465+0000 7f86ba0ca640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f86b40c86f0 0x7f86b41e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.465+0000 7f86ba0ca640 1 -- 10.20.193.16:0/740205052 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f86b41536f0 msgr2=0x7f86b40736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.465+0000 7f86ba0ca640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f86b41536f0 0x7f86b40736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.477 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.465+0000 7f86ba0ca640 10 monclient: _finish_hunting 0 2026-02-17T06:02:00.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.465+0000 7f86ba0ca640 1 monclient: found mon.noname-a 2026-02-17T06:02:00.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.465+0000 7f86ba0ca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:00.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.465+0000 7f86ba0ca640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.465+0000 7f86ba0ca640 1 -- 10.20.193.16:0/740205052 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b41f7eb0 con 0x7f86b40700f0 2026-02-17T06:02:00.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.466+0000 7f86ba0ca640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f86b40700f0 0x7f86b406ebd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f86a400c450 tx=0x7f86a400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.466+0000 7f869affd640 10 client.?.objecter ms_handle_connect 0x7f86b40700f0 2026-02-17T06:02:00.478 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.466+0000 7f869affd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:00.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 1 -- 10.20.193.16:0/740205052 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a40052d0 con 0x7f86b40700f0 2026-02-17T06:02:00.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:00.480 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 10 monclient: dump: 2026-02-17T06:02:00.480 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.480 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.480 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.481 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.481 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.481 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.481 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.481 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.481 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.481 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.481 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 10 monclient: _finish_auth 0 2026-02-17T06:02:00.481 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 10 monclient: _check_auth_tickets 2026-02-17T06:02:00.481 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 1 -- 10.20.193.16:0/740205052 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a4005470 con 0x7f86b40700f0 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 1 -- 10.20.193.16:0/740205052 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a40038f0 con 0x7f86b40700f0 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f869affd640 10 monclient: dump: 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.482 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f86bc355640 5 monclient: authenticate success, global_id 5187 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f86b88c7640 4 set_mon_vals no callback set 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f86bc355640 10 monclient: _renew_subs 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f86bc355640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f86bc355640 1 -- 10.20.193.16:0/740205052 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b41e5940 con 0x7f86b40700f0 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f86bc355640 10 client.5187.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f86bc355640 10 monclient: _renew_subs 2026-02-17T06:02:00.483 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f86bc355640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f86bc355640 1 -- 10.20.193.16:0/740205052 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b41f73d0 con 0x7f86b40700f0 2026-02-17T06:02:00.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.468+0000 7f86bc355640 1 librados: init done 2026-02-17T06:02:00.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.469+0000 7f86bc355640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:00.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.469+0000 7f86bc355640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:00.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.469+0000 7f86bc355640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.484 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.469+0000 7f86bc355640 1 -- 10.20.193.16:0/740205052 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b41f79e0 con 0x7f86b40700f0 2026-02-17T06:02:00.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.480+0000 7f869affd640 1 -- 10.20.193.16:0/740205052 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f86a4003a90 con 0x7f86b40700f0 2026-02-17T06:02:00.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.480+0000 7f869affd640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8684079170 0x7f868407b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.480+0000 7f86b98c9640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8684079170 0x7f868407b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.481+0000 7f86b98c9640 10 monclient: get_auth_request con 0x7f8684079170 auth_method 0 2026-02-17T06:02:00.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.481+0000 7f869affd640 10 client.5187.objecter ms_handle_connect 0x7f8684079170 2026-02-17T06:02:00.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.481+0000 7f86b98c9640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8684079170 0x7f868407b630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f86a8039000 tx=0x7f86a8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.482+0000 7f869affd640 1 -- 10.20.193.16:0/740205052 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f86a405e130 con 0x7f86b40700f0 2026-02-17T06:02:00.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.482+0000 7f869affd640 10 client.5187.objecter ms_dispatch2 0x7f86b4000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:00.486 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.482+0000 7f869affd640 3 client.5187.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:00.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.482+0000 7f869affd640 3 client.5187.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:00.487 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.482+0000 7f869affd640 20 client.5187.objecter dump_active .. 0 homeless 2026-02-17T06:02:00.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.584+0000 7f869affd640 1 -- 10.20.193.16:0/740205052 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f86a40620f0 con 0x7f86b40700f0 2026-02-17T06:02:00.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.584+0000 7f869affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:00.585 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.584+0000 7f869affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:00.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.668+0000 7f86bc355640 1 -- 10.20.193.16:0/740205052 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f86b40704d0 con 0x7f8684079170 2026-02-17T06:02:00.669 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.669+0000 7f869affd640 1 -- 10.20.193.16:0/740205052 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f86b40704d0 con 0x7f8684079170 2026-02-17T06:02:00.670 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 3 to deep-scrub 2026-02-17T06:02:00.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 10 librados: watch_flush enter 2026-02-17T06:02:00.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 10 librados: watch_flush exit 2026-02-17T06:02:00.671 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 20 client.5187.objecter shutdown clearing up homeless session... 2026-02-17T06:02:00.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 10 client.5187.objecter successfully canceled tick 2026-02-17T06:02:00.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 -- 10.20.193.16:0/740205052 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8684079170 msgr2=0x7f868407b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8684079170 0x7f868407b630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f86a8039000 tx=0x7f86a8033000 comp rx=0 tx=0).stop 2026-02-17T06:02:00.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 10 monclient: shutdown 2026-02-17T06:02:00.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:00.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 -- 10.20.193.16:0/740205052 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f86b40700f0 msgr2=0x7f86b406ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f86b40700f0 0x7f86b406ebd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f86a400c450 tx=0x7f86a400c920 comp rx=0 tx=0).stop 2026-02-17T06:02:00.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 -- 10.20.193.16:0/740205052 shutdown_connections 2026-02-17T06:02:00.672 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f8684079170 0x7f868407b630 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f86b41536f0 0x7f86b40736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f86b40c86f0 0x7f86b41e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 --2- 10.20.193.16:0/740205052 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f86b40700f0 0x7f86b406ebd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 -- 10.20.193.16:0/740205052 >> 10.20.193.16:0/740205052 conn(0x7f86b40cd770 msgr2=0x7f86b414f490 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:00.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 -- 10.20.193.16:0/740205052 shutdown_connections 2026-02-17T06:02:00.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 -- 10.20.193.16:0/740205052 wait complete. 2026-02-17T06:02:00.673 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.670+0000 7f86bc355640 1 librados: shutdown 2026-02-17T06:02:00.679 INFO:tasks.ceph:Scrubbing osd.7 2026-02-17T06:02:00.680 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:02:00.751 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.750+0000 7fb44d9bc640 10 monclient: get_monmap_and_config 2026-02-17T06:02:00.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.750+0000 7fb44d9bc640 10 monclient: build_initial_monmap 2026-02-17T06:02:00.752 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.750+0000 7fb44d9bc640 10 monclient: monmap: 2026-02-17T06:02:00.752 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:00.752 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.752 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:00.751753+0000 2026-02-17T06:02:00.752 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:00.751753+0000 2026-02-17T06:02:00.752 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:00.753 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.753 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:00.753 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:00.753 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:00.753 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:00.753 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:00.753 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:00.753 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 1 Processor -- start 2026-02-17T06:02:00.753 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 1 -- start start 2026-02-17T06:02:00.754 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 10 monclient: init 2026-02-17T06:02:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 10 monclient: _add_conns ranks=[3,4,0,1,2,5] 2026-02-17T06:02:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb44814f020 0x7fb44806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 10 monclient(hunting): picked mon.noname-a con 0x7fb44814f020 addr v2:10.20.193.16:3300/0 2026-02-17T06:02:00.764 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fb44806e120 0x7fb44815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 10 monclient(hunting): picked mon.noname-b con 0x7fb44806e120 addr v2:10.20.193.128:3300/0 2026-02-17T06:02:00.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb44814f400 addr v1:10.20.193.16:6789/0 2026-02-17T06:02:00.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4480c5b10 con 0x7fb44814f400 2026-02-17T06:02:00.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb44806e620 con 0x7fb44814f020 2026-02-17T06:02:00.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7fb44806e900 con 0x7fb44806e120 2026-02-17T06:02:00.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:00.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.751+0000 7fb44d9bc640 10 monclient(hunting): authenticate will time out at 1373.320758s 2026-02-17T06:02:00.765 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb4467fc640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fb44806e120 0x7fb44815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb446ffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fb44814f020 0x7fb44806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb4467fc640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7fb44806e120 0x7fb44815c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:51468/0 (socket says 10.20.193.16:51468) 2026-02-17T06:02:00.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb4467fc640 1 -- 10.20.193.16:0/3877196080 learned_addr learned my addr 10.20.193.16:0/3877196080 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:00.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb4467fc640 10 monclient(hunting): get_auth_request con 0x7fb44806e120 auth_method 0 2026-02-17T06:02:00.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb4467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb4467fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb4467fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb446ffd640 10 monclient(hunting): get_auth_request con 0x7fb44814f020 auth_method 0 2026-02-17T06:02:00.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb446ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.766 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb446ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb446ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb4467fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb4467fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb4467fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb446ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb446ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.752+0000 7fb446ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.753+0000 7fb4467fc640 10 monclient(hunting): handle_auth_done global_id 5284 payload 274 2026-02-17T06:02:00.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.753+0000 7fb4467fc640 1 -- 10.20.193.16:0/3877196080 >> v2:10.20.193.16:3300/0 conn(0x7fb44814f020 msgr2=0x7fb44806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:00.767 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.753+0000 7fb4467fc640 1 --2- 10.20.193.16:0/3877196080 >> v2:10.20.193.16:3300/0 conn(0x7fb44814f020 0x7fb44806dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.753+0000 7fb4467fc640 1 -- 10.20.193.16:0/3877196080 >> v1:10.20.193.16:6789/0 conn(0x7fb44814f400 legacy=0x7fb4480c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.753+0000 7fb4467fc640 10 monclient: _finish_hunting 0 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.753+0000 7fb4467fc640 1 monclient: found mon.noname-b 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.753+0000 7fb4467fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.753+0000 7fb4467fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.753+0000 7fb4467fc640 1 -- 10.20.193.16:0/3877196080 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44806ebe0 con 0x7fb44806e120 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.754+0000 7fb4467fc640 1 --2- 10.20.193.16:0/3877196080 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb44806e120 0x7fb44815c3e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb448156400 tx=0x7fb43002ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2ac341f047e8b5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.755+0000 7fb445ffb640 1 -- 10.20.193.16:0/3877196080 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43003c070 con 0x7fb44806e120 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.755+0000 7fb445ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.755+0000 7fb445ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:00.768 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.755+0000 7fb445ffb640 10 monclient: dump: 2026-02-17T06:02:00.769 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.769 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.769 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.769 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.769 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.769 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.769 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.769 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.769 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.770 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.755+0000 7fb445ffb640 10 monclient: _finish_auth 0 2026-02-17T06:02:00.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.755+0000 7fb445ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:02:00.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.755+0000 7fb445ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:00.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.755+0000 7fb44d9bc640 5 monclient: authenticate success, global_id 5284 2026-02-17T06:02:00.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.755+0000 7fb44d9bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:00.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb445ffb640 1 -- 10.20.193.16:0/3877196080 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb43002fc70 con 0x7fb44806e120 2026-02-17T06:02:00.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb445ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:00.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb445ffb640 1 -- 10.20.193.16:0/3877196080 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430037660 con 0x7fb44806e120 2026-02-17T06:02:00.770 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 4 set_mon_vals no callback set 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb445ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb445ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb445ffb640 10 monclient: dump: 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.771 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.772 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.772 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.772 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 10 monclient: shutdown 2026-02-17T06:02:00.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:00.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 1 -- 10.20.193.16:0/3877196080 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb44806e120 msgr2=0x7fb44815c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 1 --2- 10.20.193.16:0/3877196080 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb44806e120 0x7fb44815c3e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb448156400 tx=0x7fb43002ef60 comp rx=0 tx=0).stop 2026-02-17T06:02:00.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 1 -- 10.20.193.16:0/3877196080 shutdown_connections 2026-02-17T06:02:00.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 1 --2- 10.20.193.16:0/3877196080 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb44806e120 0x7fb44815c3e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.772 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 1 --2- 10.20.193.16:0/3877196080 >> v2:10.20.193.16:3300/0 conn(0x7fb44814f020 0x7fb44806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 1 -- 10.20.193.16:0/3877196080 >> 10.20.193.16:0/3877196080 conn(0x7fb4480cd770 msgr2=0x7fb4480cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 1 -- 10.20.193.16:0/3877196080 shutdown_connections 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.756+0000 7fb44d9bc640 1 -- 10.20.193.16:0/3877196080 wait complete. 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.757+0000 7fb44d9bc640 10 monclient: build_initial_monmap 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.757+0000 7fb44d9bc640 10 monclient: monmap: 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:00.773 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.774 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:00.774 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:00.774 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:00.774 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.757+0000 7fb44d9bc640 1 librados: starting msgr at 2026-02-17T06:02:00.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.757+0000 7fb44d9bc640 1 librados: starting objecter 2026-02-17T06:02:00.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.757+0000 7fb44d9bc640 1 Processor -- start 2026-02-17T06:02:00.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.757+0000 7fb44d9bc640 1 -- start start 2026-02-17T06:02:00.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.757+0000 7fb44d9bc640 1 librados: setting wanted keys 2026-02-17T06:02:00.774 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.757+0000 7fb44d9bc640 1 librados: calling monclient init 2026-02-17T06:02:00.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.757+0000 7fb44d9bc640 10 monclient: init 2026-02-17T06:02:00.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:00.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:02:00.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb44806e120 0x7fb448145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 10 monclient(hunting): picked mon.noname-a con 0x7fb44806e120 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:00.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb44814f020 0x7fb4480c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 10 monclient(hunting): picked mon.noname-b con 0x7fb44814f020 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:00.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb44814f400 0x7fb4481446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 10 monclient(hunting): picked mon.noname-c con 0x7fb44814f400 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:00.775 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4480c6720 con 0x7fb44806e120 2026-02-17T06:02:00.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fb44806f1a0 con 0x7fb44814f020 2026-02-17T06:02:00.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4480c68a0 con 0x7fb44814f400 2026-02-17T06:02:00.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:00.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb44d9bc640 10 monclient(hunting): authenticate will time out at 1373.327164s 2026-02-17T06:02:00.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb446ffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb44806e120 0x7fb448145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4467fc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb44814f020 0x7fb4480c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.776 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4467fc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb44814f020 0x7fb4480c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:51480/0 (socket says 10.20.193.16:51480) 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4467fc640 1 -- 10.20.193.16:0/396756093 learned_addr learned my addr 10.20.193.16:0/396756093 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4467fc640 10 monclient(hunting): get_auth_request con 0x7fb44814f020 auth_method 0 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4467fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4467fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4477fe640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb44814f400 0x7fb4481446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb446ffd640 10 monclient(hunting): get_auth_request con 0x7fb44806e120 auth_method 0 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb446ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb446ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb446ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.777 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4467fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4467fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4467fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4477fe640 10 monclient(hunting): get_auth_request con 0x7fb44814f400 auth_method 0 2026-02-17T06:02:00.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4477fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb4477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb446ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb446ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.758+0000 7fb446ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.778 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.759+0000 7fb4477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.759+0000 7fb4477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.759+0000 7fb4477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.760+0000 7fb4467fc640 10 monclient(hunting): handle_auth_done global_id 5287 payload 824 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.760+0000 7fb4467fc640 1 -- 10.20.193.16:0/396756093 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb44814f400 msgr2=0x7fb4481446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.760+0000 7fb4467fc640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fb44814f400 0x7fb4481446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.760+0000 7fb4467fc640 1 -- 10.20.193.16:0/396756093 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb44806e120 msgr2=0x7fb448145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.760+0000 7fb4467fc640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fb44806e120 0x7fb448145f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.760+0000 7fb4467fc640 10 monclient: _finish_hunting 0 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.760+0000 7fb4467fc640 1 monclient: found mon.noname-b 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.760+0000 7fb4467fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:00.779 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.760+0000 7fb4467fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.760+0000 7fb4467fc640 1 -- 10.20.193.16:0/396756093 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb448146460 con 0x7fb44814f020 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.761+0000 7fb4467fc640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb44814f020 0x7fb4480c84c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb430009a70 tx=0x7fb4300047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.761+0000 7fb41ffff640 10 client.?.objecter ms_handle_connect 0x7fb44814f020 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.761+0000 7fb41ffff640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb41ffff640 1 -- 10.20.193.16:0/396756093 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43003c040 con 0x7fb44814f020 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb41ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb41ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb41ffff640 10 monclient: dump: 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.780 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.781 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.781 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.781 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.781 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.781 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.781 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.781 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb41ffff640 10 monclient: _finish_auth 0 2026-02-17T06:02:00.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb41ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:02:00.781 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb41ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 5 monclient: authenticate success, global_id 5287 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 10 monclient: _renew_subs 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4480c7d00 con 0x7fb44814f020 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 10 client.5287.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 10 monclient: _renew_subs 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4481f1d90 con 0x7fb44814f020 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 1 librados: init done 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 10 client.5287.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:00.782 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 15 client.5287.objecter _session_command_op_assign -1 1 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 20 client.5287.objecter _get_session osd=-1 returning homeless 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 10 client.5287.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.762+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb44806dbe0 con 0x7fb44814f020 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.763+0000 7fb41ffff640 1 -- 10.20.193.16:0/396756093 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb44806dbe0 con 0x7fb44814f020 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.763+0000 7fb41ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.763+0000 7fb41ffff640 1 -- 10.20.193.16:0/396756093 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430042630 con 0x7fb44814f020 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.763+0000 7fb41ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.763+0000 7fb41ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.763+0000 7fb41ffff640 10 monclient: dump: 2026-02-17T06:02:00.783 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.784 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.784 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.784 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.784 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.784 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.784 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.784 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.784 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.784 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.784 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.763+0000 7fb4457fa640 4 set_mon_vals no callback set 2026-02-17T06:02:00.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.774+0000 7fb41ffff640 1 -- 10.20.193.16:0/396756093 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fb4300427d0 con 0x7fb44814f020 2026-02-17T06:02:00.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.775+0000 7fb41ffff640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb418079140 0x7fb41807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.775+0000 7fb446ffd640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb418079140 0x7fb41807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.775+0000 7fb446ffd640 10 monclient: get_auth_request con 0x7fb418079140 auth_method 0 2026-02-17T06:02:00.787 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.775+0000 7fb41ffff640 10 client.5287.objecter ms_handle_connect 0x7fb418079140 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.775+0000 7fb446ffd640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb418079140 0x7fb41807b600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb43c009ba0 tx=0x7fb43c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 1 -- 10.20.193.16:0/396756093 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fb4300bd0a0 con 0x7fb44814f020 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 10 client.5287.objecter ms_dispatch2 0x7fb448000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 3 client.5287.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 3 client.5287.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 10 client.5287.objecter checking command 1 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] conn(0x7fb418081680 0x7fb418083aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 20 client.5287.objecter _get_session s=0x7fb418080fb0 osd=7 3 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 20 client.5287.objecter put_session s=0x7fb418080fb0 osd=7 3 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 15 client.5287.objecter _session_command_op_remove -1 1 2026-02-17T06:02:00.788 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 20 client.5287.objecter _get_session s=0x7fb418080fb0 osd=7 3 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 20 client.5287.objecter get_session s=0x7fb418080fb0 osd=7 3 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 15 client.5287.objecter _session_command_op_assign 7 1 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 20 client.5287.objecter put_session s=0x7fb418080fb0 osd=7 4 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 10 client.5287.objecter _send_command 1 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 1 -- 10.20.193.16:0/396756093 --> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb430036ec0 con 0x7fb418081680 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 20 client.5287.objecter dump_active .. 0 homeless 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 1 -- 10.20.193.16:0/396756093 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb430083a50 con 0x7fb44814f020 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb41ffff640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.776+0000 7fb4477fe640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] conn(0x7fb418081680 0x7fb418083aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.777+0000 7fb4477fe640 10 monclient: get_auth_request con 0x7fb418081680 auth_method 0 2026-02-17T06:02:00.789 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.778+0000 7fb4477fe640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] conn(0x7fb418081680 0x7fb418083aa0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.778+0000 7fb41ffff640 10 client.5287.objecter ms_handle_connect 0x7fb418081680 2026-02-17T06:02:00.790 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.790+0000 7fb41ffff640 1 -- 10.20.193.16:0/396756093 <== osd.7 v2:10.20.193.164:6816/3942637745 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fb430036ec0 con 0x7fb418081680 2026-02-17T06:02:00.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.790+0000 7fb41ffff640 10 client.5287.objecter ms_dispatch2 0x7fb448000b90 command_reply(tid 1: 0 ) 2026-02-17T06:02:00.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.790+0000 7fb41ffff640 10 client.5287.objecter _finish_command 1 = system:0 2026-02-17T06:02:00.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.790+0000 7fb41ffff640 20 client.5287.objecter put_session s=0x7fb418080fb0 osd=7 4 2026-02-17T06:02:00.791 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.790+0000 7fb41ffff640 15 client.5287.objecter _session_command_op_remove 7 1 2026-02-17T06:02:00.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 10 client.5287.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:02:00.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 15 client.5287.objecter _session_command_op_assign -1 2 2026-02-17T06:02:00.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 20 client.5287.objecter _get_session s=0x7fb418080fb0 osd=7 3 2026-02-17T06:02:00.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 20 client.5287.objecter put_session s=0x7fb418080fb0 osd=7 3 2026-02-17T06:02:00.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 20 client.5287.objecter _get_session s=0x7fb418080fb0 osd=7 3 2026-02-17T06:02:00.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 15 client.5287.objecter _session_command_op_remove -1 2 2026-02-17T06:02:00.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 20 client.5287.objecter get_session s=0x7fb418080fb0 osd=7 3 2026-02-17T06:02:00.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 15 client.5287.objecter _session_command_op_assign 7 2 2026-02-17T06:02:00.796 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 20 client.5287.objecter put_session s=0x7fb418080fb0 osd=7 4 2026-02-17T06:02:00.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 10 client.5287.objecter _send_command 2 2026-02-17T06:02:00.797 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.795+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 --> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb44815cfa0 con 0x7fb418081680 2026-02-17T06:02:00.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb41ffff640 1 -- 10.20.193.16:0/396756093 <== osd.7 v2:10.20.193.164:6816/3942637745 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb44815cfa0 con 0x7fb418081680 2026-02-17T06:02:00.802 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb41ffff640 10 client.5287.objecter ms_dispatch2 0x7fb448000b90 command_reply(tid 2: 0 ) 2026-02-17T06:02:00.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb41ffff640 10 client.5287.objecter _finish_command 2 = system:0 2026-02-17T06:02:00.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb41ffff640 20 client.5287.objecter put_session s=0x7fb418080fb0 osd=7 4 2026-02-17T06:02:00.803 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb41ffff640 15 client.5287.objecter _session_command_op_remove 7 2 2026-02-17T06:02:00.803 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:02:00.803 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:02:00.803 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:02:00.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 10 librados: watch_flush enter 2026-02-17T06:02:00.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 10 librados: watch_flush exit 2026-02-17T06:02:00.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 10 client.5287.objecter close_session for osd.7 2026-02-17T06:02:00.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 >> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] conn(0x7fb418081680 msgr2=0x7fb418083aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.164:6816/3942637745,v1:10.20.193.164:6817/3942637745] conn(0x7fb418081680 0x7fb418083aa0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 20 client.5287.objecter put_session s=0x7fb418080fb0 osd=7 1 2026-02-17T06:02:00.805 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 20 client.5287.objecter shutdown clearing up homeless session... 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 10 client.5287.objecter successfully canceled tick 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb418079140 msgr2=0x7fb41807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fb418079140 0x7fb41807b600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb43c009ba0 tx=0x7fb43c009290 comp rx=0 tx=0).stop 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 10 monclient: shutdown 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb44814f020 msgr2=0x7fb4480c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 --2- 10.20.193.16:0/396756093 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fb44814f020 0x7fb4480c84c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb430009a70 tx=0x7fb4300047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb4477fe640 1 -- 10.20.193.16:0/396756093 reap_dead start 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 shutdown_connections 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 >> 10.20.193.16:0/396756093 conn(0x7fb4480cd770 msgr2=0x7fb4480c8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:00.806 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 shutdown_connections 2026-02-17T06:02:00.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 -- 10.20.193.16:0/396756093 wait complete. 2026-02-17T06:02:00.807 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.803+0000 7fb44d9bc640 1 librados: shutdown 2026-02-17T06:02:00.811 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2026-02-17T06:02:00.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.879+0000 7f300213e640 10 monclient: get_monmap_and_config 2026-02-17T06:02:00.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.879+0000 7f300213e640 10 monclient: build_initial_monmap 2026-02-17T06:02:00.880 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.879+0000 7f300213e640 10 monclient: monmap: 2026-02-17T06:02:00.880 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:00.880 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.881 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:00.880138+0000 2026-02-17T06:02:00.881 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:00.880138+0000 2026-02-17T06:02:00.881 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:00.881 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.881 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:00.881 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:00.881 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:00.881 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:00.881 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:00.881 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:00.882 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.879+0000 7f300213e640 1 Processor -- start 2026-02-17T06:02:00.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.879+0000 7f300213e640 1 -- start start 2026-02-17T06:02:00.882 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.879+0000 7f300213e640 10 monclient: init 2026-02-17T06:02:00.892 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 10 monclient: _add_conns ranks=[2,3,4,0,5,1] 2026-02-17T06:02:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2ffc14f020 addr v1:10.20.193.148:6789/0 2026-02-17T06:02:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f2ffc14f400 0x7f2ffc15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 10 monclient(hunting): picked mon.noname-a con 0x7f2ffc14f400 addr v2:10.20.193.16:3300/0 2026-02-17T06:02:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f2ffc15c920 0x7f2ffc0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 10 monclient(hunting): picked mon.noname-b con 0x7f2ffc15c920 addr v2:10.20.193.128:3300/0 2026-02-17T06:02:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ffc0c5b10 con 0x7f2ffc14f020 2026-02-17T06:02:00.893 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2ffc15cd20 con 0x7f2ffc14f400 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f2ffc06e620 con 0x7f2ffc15c920 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f300213e640 10 monclient(hunting): authenticate will time out at 1373.449093s 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffaffd640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f2ffc14f400 0x7f2ffc15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffbfff640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f2ffc15c920 0x7f2ffc0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffb7fe640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f2ffc14f020 0x7f2ffc06dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:36240/0 (socket says 10.20.193.16:36240) 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffb7fe640 1 -- 10.20.193.16:0/1868106175 learned_addr learned my addr 10.20.193.16:0/1868106175 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffbfff640 10 monclient(hunting): get_auth_request con 0x7f2ffc15c920 auth_method 0 2026-02-17T06:02:00.894 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffaffd640 10 monclient(hunting): get_auth_request con 0x7f2ffc14f400 auth_method 0 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.880+0000 7f2ffaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.895 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffa7fc640 1 -- 10.20.193.16:0/1868106175 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2097207181 0 0) 0x7f2ffc0c5b10 con 0x7f2ffc14f020 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffa7fc640 10 monclient(hunting): _negotiate 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffa7fc640 10 monclient(hunting): my global_id is 5333 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffa7fc640 1 -- 10.20.193.16:0/1868106175 --> v1:10.20.193.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fe0003620 con 0x7f2ffc14f020 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffaffd640 10 monclient(hunting): handle_auth_done global_id 5199 payload 274 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffaffd640 1 -- 10.20.193.16:0/1868106175 >> v2:10.20.193.128:3300/0 conn(0x7f2ffc15c920 msgr2=0x7f2ffc0c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffaffd640 1 --2- 10.20.193.16:0/1868106175 >> v2:10.20.193.128:3300/0 conn(0x7f2ffc15c920 0x7f2ffc0c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffaffd640 1 -- 10.20.193.16:0/1868106175 >> v1:10.20.193.148:6789/0 conn(0x7f2ffc14f020 legacy=0x7f2ffc06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffaffd640 10 monclient: _finish_hunting 0 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffaffd640 1 monclient: found mon.noname-a 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffaffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.881+0000 7f2ffaffd640 1 -- 10.20.193.16:0/1868106175 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc06e900 con 0x7f2ffc14f400 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.882+0000 7f2ffaffd640 1 --2- 10.20.193.16:0/1868106175 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 0x7f2ffc15c3e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0c9d30 tx=0x7f2fec02eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3666657222edda server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 1 -- 10.20.193.16:0/1868106175 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec03c070 con 0x7f2ffc14f400 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 10 monclient: dump: 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 10 monclient: _finish_auth 0 2026-02-17T06:02:00.896 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 1 -- 10.20.193.16:0/1868106175 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fec02fb80 con 0x7f2ffc14f400 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 1 -- 10.20.193.16:0/1868106175 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec037440 con 0x7f2ffc14f400 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f300213e640 5 monclient: authenticate success, global_id 5199 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f300213e640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f300213e640 4 set_mon_vals no callback set 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.884+0000 7f2ffa7fc640 10 monclient: dump: 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 10 monclient: shutdown 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 -- 10.20.193.16:0/1868106175 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 msgr2=0x7f2ffc15c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 --2- 10.20.193.16:0/1868106175 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 0x7f2ffc15c3e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0c9d30 tx=0x7f2fec02eeb0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 -- 10.20.193.16:0/1868106175 shutdown_connections 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 --2- 10.20.193.16:0/1868106175 >> v2:10.20.193.128:3300/0 conn(0x7f2ffc15c920 0x7f2ffc0c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 --2- 10.20.193.16:0/1868106175 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 0x7f2ffc15c3e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 -- 10.20.193.16:0/1868106175 >> 10.20.193.16:0/1868106175 conn(0x7f2ffc0cd770 msgr2=0x7f2ffc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 -- 10.20.193.16:0/1868106175 shutdown_connections 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 -- 10.20.193.16:0/1868106175 wait complete. 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 10 monclient: build_initial_monmap 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 10 monclient: monmap: 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:00.897 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 librados: starting msgr at 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 librados: starting objecter 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 Processor -- start 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 -- start start 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 librados: setting wanted keys 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 1 librados: calling monclient init 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.885+0000 7f300213e640 10 monclient: init 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ffc14f020 0x7f2ffc1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 10 monclient(hunting): picked mon.noname-c con 0x7f2ffc14f020 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 0x7f2ffc14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 10 monclient(hunting): picked mon.noname-a con 0x7f2ffc14f400 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2ffc15c920 0x7f2ffc145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 10 monclient(hunting): picked mon.noname-b con 0x7f2ffc15c920 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc06f1a0 con 0x7f2ffc14f400 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc06eec0 con 0x7f2ffc15c920 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc06e2f0 con 0x7f2ffc14f020 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f300213e640 10 monclient(hunting): authenticate will time out at 1373.455299s 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffaffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 0x7f2ffc14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffaffd640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 0x7f2ffc14e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:43652/0 (socket says 10.20.193.16:43652) 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffaffd640 1 -- 10.20.193.16:0/1501848931 learned_addr learned my addr 10.20.193.16:0/1501848931 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffaffd640 10 monclient(hunting): get_auth_request con 0x7f2ffc14f400 auth_method 0 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffb7fe640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ffc14f020 0x7f2ffc1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffbfff640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2ffc15c920 0x7f2ffc145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffb7fe640 10 monclient(hunting): get_auth_request con 0x7f2ffc14f020 auth_method 0 2026-02-17T06:02:00.898 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.886+0000 7f2ffb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffbfff640 10 monclient(hunting): get_auth_request con 0x7f2ffc15c920 auth_method 0 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.887+0000 7f2ffbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.888+0000 7f2ffaffd640 10 monclient(hunting): handle_auth_done global_id 5202 payload 824 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.888+0000 7f2ffaffd640 1 -- 10.20.193.16:0/1501848931 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ffc14f020 msgr2=0x7f2ffc1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.888+0000 7f2ffaffd640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ffc14f020 0x7f2ffc1581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.888+0000 7f2ffaffd640 1 -- 10.20.193.16:0/1501848931 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2ffc15c920 msgr2=0x7f2ffc145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.888+0000 7f2ffaffd640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2ffc15c920 0x7f2ffc145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.888+0000 7f2ffaffd640 10 monclient: _finish_hunting 0 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.888+0000 7f2ffaffd640 1 monclient: found mon.noname-a 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.888+0000 7f2ffaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.888+0000 7f2ffaffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.888+0000 7f2ffaffd640 1 -- 10.20.193.16:0/1501848931 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc1567b0 con 0x7f2ffc14f400 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.889+0000 7f2ffaffd640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 0x7f2ffc14e0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2fec03a040 tx=0x7f2fec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.889+0000 7f2fd7fff640 10 client.?.objecter ms_handle_connect 0x7f2ffc14f400 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.889+0000 7f2fd7fff640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 1 -- 10.20.193.16:0/1501848931 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec046020 con 0x7f2ffc14f400 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 10 monclient: dump: 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.899 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 10 monclient: _finish_auth 0 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 10 monclient: _check_auth_tickets 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 1 -- 10.20.193.16:0/1501848931 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fec042630 con 0x7f2ffc14f400 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 1 -- 10.20.193.16:0/1501848931 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec03c040 con 0x7f2ffc14f400 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2ff9ffb640 4 set_mon_vals no callback set 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f2fd7fff640 10 monclient: dump: 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 5 monclient: authenticate success, global_id 5202 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 10 monclient: _renew_subs 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 1 -- 10.20.193.16:0/1501848931 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc156a90 con 0x7f2ffc14f400 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 10 client.5202.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 10 monclient: _renew_subs 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 1 -- 10.20.193.16:0/1501848931 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc157050 con 0x7f2ffc14f400 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 1 librados: init done 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:00.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.891+0000 7f300213e640 1 -- 10.20.193.16:0/1501848931 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc15c3e0 con 0x7f2ffc14f400 2026-02-17T06:02:00.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.903+0000 7f2fd7fff640 1 -- 10.20.193.16:0/1501848931 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f2fec053070 con 0x7f2ffc14f400 2026-02-17T06:02:00.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.904+0000 7f2fd7fff640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2fcc079140 0x7f2fcc07b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:00.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.904+0000 7f2ffb7fe640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2fcc079140 0x7f2fcc07b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:00.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.904+0000 7f2ffb7fe640 10 monclient: get_auth_request con 0x7f2fcc079140 auth_method 0 2026-02-17T06:02:00.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.904+0000 7f2ffb7fe640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2fcc079140 0x7f2fcc07b600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8007b60 tx=0x7f2fe8007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:00.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.904+0000 7f2fd7fff640 10 client.5202.objecter ms_handle_connect 0x7f2fcc079140 2026-02-17T06:02:00.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.906+0000 7f2fd7fff640 1 -- 10.20.193.16:0/1501848931 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f2fec02fa40 con 0x7f2ffc14f400 2026-02-17T06:02:00.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.906+0000 7f2fd7fff640 10 client.5202.objecter ms_dispatch2 0x7f2ffc000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:00.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.906+0000 7f2fd7fff640 3 client.5202.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:00.906 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.906+0000 7f2fd7fff640 3 client.5202.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:00.907 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:00.906+0000 7f2fd7fff640 20 client.5202.objecter dump_active .. 0 homeless 2026-02-17T06:02:01.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.008+0000 7f2fd7fff640 1 -- 10.20.193.16:0/1501848931 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2fec050020 con 0x7f2ffc14f400 2026-02-17T06:02:01.008 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.008+0000 7f2fd7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:01.009 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.008+0000 7f2fd7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:01.090 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.090+0000 7f300213e640 1 -- 10.20.193.16:0/1501848931 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f2ffc0cb300 con 0x7f2fcc079140 2026-02-17T06:02:01.091 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.090+0000 7f2fd7fff640 1 -- 10.20.193.16:0/1501848931 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2ffc0cb300 con 0x7f2fcc079140 2026-02-17T06:02:01.091 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 7 to deep-scrub 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 10 librados: watch_flush enter 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 10 librados: watch_flush exit 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 20 client.5202.objecter shutdown clearing up homeless session... 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 10 client.5202.objecter successfully canceled tick 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 -- 10.20.193.16:0/1501848931 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2fcc079140 msgr2=0x7f2fcc07b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2fcc079140 0x7f2fcc07b600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8007b60 tx=0x7f2fe8007ab0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 10 monclient: shutdown 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 -- 10.20.193.16:0/1501848931 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 msgr2=0x7f2ffc14e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 0x7f2ffc14e0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2fec03a040 tx=0x7f2fec0047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 -- 10.20.193.16:0/1501848931 shutdown_connections 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f2fcc079140 0x7f2fcc07b600 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f2ffc15c920 0x7f2ffc145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f2ffc14f400 0x7f2ffc14e0a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 --2- 10.20.193.16:0/1501848931 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f2ffc14f020 0x7f2ffc1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.092 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 -- 10.20.193.16:0/1501848931 >> 10.20.193.16:0/1501848931 conn(0x7f2ffc0cd770 msgr2=0x7f2ffc0cb820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:01.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 -- 10.20.193.16:0/1501848931 shutdown_connections 2026-02-17T06:02:01.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 -- 10.20.193.16:0/1501848931 wait complete. 2026-02-17T06:02:01.093 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.091+0000 7f300213e640 1 librados: shutdown 2026-02-17T06:02:01.100 INFO:tasks.ceph:Scrubbing osd.11 2026-02-17T06:02:01.100 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:02:01.170 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.170+0000 7f4ac1734640 10 monclient: get_monmap_and_config 2026-02-17T06:02:01.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.170+0000 7f4ac1734640 10 monclient: build_initial_monmap 2026-02-17T06:02:01.171 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.170+0000 7f4ac1734640 10 monclient: monmap: 2026-02-17T06:02:01.171 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:01.171 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.171 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:01.171249+0000 2026-02-17T06:02:01.171 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:01.171249+0000 2026-02-17T06:02:01.171 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:01.171 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.171 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:01.172 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:01.172 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:01.172 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:01.172 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:01.172 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:01.172 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.170+0000 7f4ac1734640 1 Processor -- start 2026-02-17T06:02:01.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.170+0000 7f4ac1734640 1 -- start start 2026-02-17T06:02:01.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.170+0000 7f4ac1734640 10 monclient: init 2026-02-17T06:02:01.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:01.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 10 monclient: _add_conns ranks=[4,1,0,2,5,3] 2026-02-17T06:02:01.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f4abc0700f0 0x7f4abc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 10 monclient(hunting): picked mon.noname-b con 0x7f4abc0700f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:02:01.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f4abc0c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T06:02:01.175 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4abc1536f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:02:01.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4abc0c4820 con 0x7f4abc1536f0 2026-02-17T06:02:01.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4abc0c4b00 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f4abc0c4de0 con 0x7f4abc0700f0 2026-02-17T06:02:01.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:01.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4ac1734640 10 monclient(hunting): authenticate will time out at 1373.740273s 2026-02-17T06:02:01.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4abb7fe640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f4abc1536f0 0x7f4abc153b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:51002/0 (socket says 10.20.193.16:51002) 2026-02-17T06:02:01.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4abb7fe640 1 -- 10.20.193.16:0/3072692771 learned_addr learned my addr 10.20.193.16:0/3072692771 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:02:01.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.171+0000 7f4abaffd640 1 --2- 10.20.193.16:0/3072692771 >> v2:10.20.193.128:3300/0 conn(0x7f4abc0700f0 0x7f4abc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.176 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.172+0000 7f4abaffd640 10 monclient(hunting): get_auth_request con 0x7f4abc0700f0 auth_method 0 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.172+0000 7f4abaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.172+0000 7f4abaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.172+0000 7f4abaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.172+0000 7f4abaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.172+0000 7f4abaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.172+0000 7f4abaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.173+0000 7f4ab9ffb640 1 -- 10.20.193.16:0/3072692771 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1148710963 0 0) 0x7f4abc0c4820 con 0x7f4abc1536f0 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.173+0000 7f4ab9ffb640 10 monclient(hunting): _negotiate 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.173+0000 7f4ab9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.173+0000 7f4ab9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.173+0000 7f4ab9ffb640 10 monclient(hunting): my global_id is 5205 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.173+0000 7f4ab9ffb640 1 -- 10.20.193.16:0/3072692771 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4aa8003620 con 0x7f4abc1536f0 2026-02-17T06:02:01.177 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.174+0000 7f4abaffd640 10 monclient(hunting): handle_auth_done global_id 5296 payload 274 2026-02-17T06:02:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.174+0000 7f4abaffd640 1 -- 10.20.193.16:0/3072692771 >> v1:10.20.193.128:6789/0 conn(0x7f4abc0c7fe0 legacy=0x7f4abc0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.174+0000 7f4abaffd640 1 -- 10.20.193.16:0/3072692771 >> v1:10.20.193.16:6789/0 conn(0x7f4abc1536f0 legacy=0x7f4abc153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.174+0000 7f4abaffd640 10 monclient: _finish_hunting 0 2026-02-17T06:02:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.174+0000 7f4abaffd640 1 monclient: found mon.noname-b 2026-02-17T06:02:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.174+0000 7f4abaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.174+0000 7f4abaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:01.178 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.174+0000 7f4abaffd640 1 -- 10.20.193.16:0/3072692771 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc0c50c0 con 0x7f4abc0700f0 2026-02-17T06:02:01.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.175+0000 7f4abaffd640 1 --2- 10.20.193.16:0/3072692771 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4abc0700f0 0x7f4abc0704d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00098a0 tx=0x7f4ab002edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c09dd84a73575dc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 1 -- 10.20.193.16:0/3072692771 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab003c070 con 0x7f4abc0700f0 2026-02-17T06:02:01.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:01.183 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 10 monclient: dump: 2026-02-17T06:02:01.183 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.183 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.183 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 1 -- 10.20.193.16:0/3072692771 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab002fae0 con 0x7f4abc0700f0 2026-02-17T06:02:01.184 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ac1734640 5 monclient: authenticate success, global_id 5296 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ac1734640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 1 -- 10.20.193.16:0/3072692771 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0037440 con 0x7f4abc0700f0 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ac1734640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ac1734640 4 set_mon_vals no callback set 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.177+0000 7f4ab9ffb640 10 monclient: dump: 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.185 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.186 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.186 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.186 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.186 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.186 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.186 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.186 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 10 monclient: shutdown 2026-02-17T06:02:01.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:01.186 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 -- 10.20.193.16:0/3072692771 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4abc0700f0 msgr2=0x7f4abc0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 --2- 10.20.193.16:0/3072692771 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4abc0700f0 0x7f4abc0704d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00098a0 tx=0x7f4ab002edd0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 -- 10.20.193.16:0/3072692771 shutdown_connections 2026-02-17T06:02:01.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 --2- 10.20.193.16:0/3072692771 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4abc0700f0 0x7f4abc0704d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 -- 10.20.193.16:0/3072692771 >> 10.20.193.16:0/3072692771 conn(0x7f4abc0cd770 msgr2=0x7f4abc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:01.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 -- 10.20.193.16:0/3072692771 shutdown_connections 2026-02-17T06:02:01.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 -- 10.20.193.16:0/3072692771 wait complete. 2026-02-17T06:02:01.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 10 monclient: build_initial_monmap 2026-02-17T06:02:01.187 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 10 monclient: monmap: 2026-02-17T06:02:01.187 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:01.188 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:01.188 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:01.188 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:01.188 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:01.188 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.188 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:01.188 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:01.188 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:01.188 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.188 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 librados: starting msgr at 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 librados: starting objecter 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 Processor -- start 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 -- start start 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 librados: setting wanted keys 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 1 librados: calling monclient init 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.178+0000 7f4ac1734640 10 monclient: init 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4abc0700f0 0x7f4abc13ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 10 monclient(hunting): picked mon.noname-c con 0x7f4abc0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4abc0c7fe0 0x7f4abc072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.189 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 10 monclient(hunting): picked mon.noname-b con 0x7f4abc0c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4abc1536f0 0x7f4abc152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 10 monclient(hunting): picked mon.noname-a con 0x7f4abc1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc074c00 con 0x7f4abc1536f0 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc0c5680 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc074d80 con 0x7f4abc0700f0 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4ac1734640 10 monclient(hunting): authenticate will time out at 1373.748407s 2026-02-17T06:02:01.190 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4aba7fc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4abc0c7fe0 0x7f4abc072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4abb7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4abc1536f0 0x7f4abc152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4aba7fc640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4abc0c7fe0 0x7f4abc072400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:51512/0 (socket says 10.20.193.16:51512) 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4aba7fc640 1 -- 10.20.193.16:0/1415900267 learned_addr learned my addr 10.20.193.16:0/1415900267 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4aba7fc640 10 monclient(hunting): get_auth_request con 0x7f4abc0c7fe0 auth_method 0 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4aba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4aba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4aba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4abaffd640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4abc0700f0 0x7f4abc13ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4abb7fe640 10 monclient(hunting): get_auth_request con 0x7f4abc1536f0 auth_method 0 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4abb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4abb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.191 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.179+0000 7f4abb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4aba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4aba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4aba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4abaffd640 10 monclient(hunting): get_auth_request con 0x7f4abc0700f0 auth_method 0 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4abaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4abaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4abaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4abb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4abb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4abb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4abaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.192 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4abaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.180+0000 7f4abaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.181+0000 7f4aba7fc640 10 monclient(hunting): handle_auth_done global_id 5302 payload 824 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.181+0000 7f4aba7fc640 1 -- 10.20.193.16:0/1415900267 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4abc0700f0 msgr2=0x7f4abc13ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.181+0000 7f4aba7fc640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f4abc0700f0 0x7f4abc13ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.181+0000 7f4aba7fc640 1 -- 10.20.193.16:0/1415900267 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4abc1536f0 msgr2=0x7f4abc152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.181+0000 7f4aba7fc640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f4abc1536f0 0x7f4abc152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.181+0000 7f4aba7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.181+0000 7f4aba7fc640 1 monclient: found mon.noname-b 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.181+0000 7f4aba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.181+0000 7f4aba7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.181+0000 7f4aba7fc640 1 -- 10.20.193.16:0/1415900267 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc152f40 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.193 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.182+0000 7f4aba7fc640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4abc0c7fe0 0x7f4abc072400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c700 tx=0x7f4aa400cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.182+0000 7f4a9b7fe640 10 client.?.objecter ms_handle_connect 0x7f4abc0c7fe0 2026-02-17T06:02:01.194 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.182+0000 7f4a9b7fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:01.195 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 1 -- 10.20.193.16:0/1415900267 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40052d0 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 10 monclient: dump: 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.196 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 1 -- 10.20.193.16:0/1415900267 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa4005470 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 1 -- 10.20.193.16:0/1415900267 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa400a860 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ab97fa640 4 set_mon_vals no callback set 2026-02-17T06:02:01.197 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4a9b7fe640 10 monclient: dump: 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 5 monclient: authenticate success, global_id 5302 2026-02-17T06:02:01.198 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 10 monclient: _renew_subs 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4abc0cb390 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 10 client.5302.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 10 monclient: _renew_subs 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4abc0cb950 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 1 librados: init done 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 10 client.5302.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 15 client.5302.objecter _session_command_op_assign -1 1 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 20 client.5302.objecter _get_session osd=-1 returning homeless 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 10 client.5302.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:01.199 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.184+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4abc0704d0 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.196+0000 7f4a9b7fe640 1 -- 10.20.193.16:0/1415900267 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f4aa400aa00 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.197+0000 7f4a9b7fe640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4a88079140 0x7f4a8807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.201 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.197+0000 7f4abaffd640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4a88079140 0x7f4a8807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.197+0000 7f4abaffd640 10 monclient: get_auth_request con 0x7f4a88079140 auth_method 0 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.197+0000 7f4abaffd640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4a88079140 0x7f4a8807b600 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0039000 tx=0x7f4ab0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.197+0000 7f4a9b7fe640 10 client.5302.objecter ms_handle_connect 0x7f4a88079140 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 1 -- 10.20.193.16:0/1415900267 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f4aa400ad20 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 10 client.5302.objecter ms_dispatch2 0x7f4abc000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 3 client.5302.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 3 client.5302.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 10 client.5302.objecter checking command 1 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] conn(0x7f4a88081620 0x7f4a88083a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 20 client.5302.objecter _get_session s=0x7f4a88080f50 osd=11 3 2026-02-17T06:02:01.202 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 20 client.5302.objecter put_session s=0x7f4a88080f50 osd=11 3 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 15 client.5302.objecter _session_command_op_remove -1 1 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 20 client.5302.objecter _get_session s=0x7f4a88080f50 osd=11 3 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 20 client.5302.objecter get_session s=0x7f4a88080f50 osd=11 3 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 15 client.5302.objecter _session_command_op_assign 11 1 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 20 client.5302.objecter put_session s=0x7f4a88080f50 osd=11 4 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 10 client.5302.objecter _send_command 1 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 1 -- 10.20.193.16:0/1415900267 --> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4aa4004ec0 con 0x7f4a88081620 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 20 client.5302.objecter dump_active .. 0 homeless 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 1 -- 10.20.193.16:0/1415900267 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4aa405e5a0 con 0x7f4abc0c7fe0 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4a9b7fe640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4abb7fe640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] conn(0x7f4a88081620 0x7f4a88083a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.198+0000 7f4abb7fe640 10 monclient: get_auth_request con 0x7f4a88081620 auth_method 0 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.200+0000 7f4abb7fe640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] conn(0x7f4a88081620 0x7f4a88083a40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.203 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.200+0000 7f4a9b7fe640 10 client.5302.objecter ms_handle_connect 0x7f4a88081620 2026-02-17T06:02:01.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.212+0000 7f4a9b7fe640 1 -- 10.20.193.16:0/1415900267 <== osd.11 v2:10.20.193.164:6808/188912883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f4aa4004ec0 con 0x7f4a88081620 2026-02-17T06:02:01.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.212+0000 7f4a9b7fe640 10 client.5302.objecter ms_dispatch2 0x7f4abc000b90 command_reply(tid 1: 0 ) 2026-02-17T06:02:01.211 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.212+0000 7f4a9b7fe640 10 client.5302.objecter _finish_command 1 = system:0 2026-02-17T06:02:01.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.212+0000 7f4a9b7fe640 20 client.5302.objecter put_session s=0x7f4a88080f50 osd=11 4 2026-02-17T06:02:01.212 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.212+0000 7f4a9b7fe640 15 client.5302.objecter _session_command_op_remove 11 1 2026-02-17T06:02:01.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 10 client.5302.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:02:01.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 15 client.5302.objecter _session_command_op_assign -1 2 2026-02-17T06:02:01.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 20 client.5302.objecter _get_session s=0x7f4a88080f50 osd=11 3 2026-02-17T06:02:01.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 20 client.5302.objecter put_session s=0x7f4a88080f50 osd=11 3 2026-02-17T06:02:01.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 20 client.5302.objecter _get_session s=0x7f4a88080f50 osd=11 3 2026-02-17T06:02:01.218 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 15 client.5302.objecter _session_command_op_remove -1 2 2026-02-17T06:02:01.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 20 client.5302.objecter get_session s=0x7f4a88080f50 osd=11 3 2026-02-17T06:02:01.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 15 client.5302.objecter _session_command_op_assign 11 2 2026-02-17T06:02:01.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 20 client.5302.objecter put_session s=0x7f4a88080f50 osd=11 4 2026-02-17T06:02:01.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 10 client.5302.objecter _send_command 2 2026-02-17T06:02:01.219 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.217+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 --> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4abc0749e0 con 0x7f4a88081620 2026-02-17T06:02:01.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4a9b7fe640 1 -- 10.20.193.16:0/1415900267 <== osd.11 v2:10.20.193.164:6808/188912883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4abc0749e0 con 0x7f4a88081620 2026-02-17T06:02:01.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4a9b7fe640 10 client.5302.objecter ms_dispatch2 0x7f4abc000b90 command_reply(tid 2: 0 ) 2026-02-17T06:02:01.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4a9b7fe640 10 client.5302.objecter _finish_command 2 = system:0 2026-02-17T06:02:01.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4a9b7fe640 20 client.5302.objecter put_session s=0x7f4a88080f50 osd=11 4 2026-02-17T06:02:01.225 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4a9b7fe640 15 client.5302.objecter _session_command_op_remove 11 2 2026-02-17T06:02:01.225 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:02:01.226 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:02:01.226 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:02:01.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 10 librados: watch_flush enter 2026-02-17T06:02:01.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 10 librados: watch_flush exit 2026-02-17T06:02:01.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 10 client.5302.objecter close_session for osd.11 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 >> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] conn(0x7f4a88081620 msgr2=0x7f4a88083a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.164:6808/188912883,v1:10.20.193.164:6809/188912883] conn(0x7f4a88081620 0x7f4a88083a40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 20 client.5302.objecter put_session s=0x7f4a88080f50 osd=11 1 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 20 client.5302.objecter shutdown clearing up homeless session... 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 10 client.5302.objecter successfully canceled tick 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4a88079140 msgr2=0x7f4a8807b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f4a88079140 0x7f4a8807b600 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0039000 tx=0x7f4ab0033000 comp rx=0 tx=0).stop 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 10 monclient: shutdown 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4abc0c7fe0 msgr2=0x7f4abc072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 1 --2- 10.20.193.16:0/1415900267 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f4abc0c7fe0 0x7f4abc072400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c700 tx=0x7f4aa400cbd0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4abb7fe640 1 -- 10.20.193.16:0/1415900267 reap_dead start 2026-02-17T06:02:01.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 shutdown_connections 2026-02-17T06:02:01.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 >> 10.20.193.16:0/1415900267 conn(0x7f4abc0cd770 msgr2=0x7f4abc14eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:01.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.224+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 shutdown_connections 2026-02-17T06:02:01.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.225+0000 7f4ac1734640 1 -- 10.20.193.16:0/1415900267 wait complete. 2026-02-17T06:02:01.229 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.225+0000 7f4ac1734640 1 librados: shutdown 2026-02-17T06:02:01.233 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2026-02-17T06:02:01.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.300+0000 7f04f867d640 10 monclient: get_monmap_and_config 2026-02-17T06:02:01.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.300+0000 7f04f867d640 10 monclient: build_initial_monmap 2026-02-17T06:02:01.300 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.300+0000 7f04f867d640 10 monclient: monmap: 2026-02-17T06:02:01.300 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:01.301301+0000 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:01.301301+0000 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.300+0000 7f04f867d640 1 Processor -- start 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.300+0000 7f04f867d640 1 -- start start 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.300+0000 7f04f867d640 10 monclient: init 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:01.301 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 10 monclient: _add_conns ranks=[0,2,5,4,1,3] 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f04f00700f0 addr v1:10.20.193.16:6789/0 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f04f00c86f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f04f0153e00 0x7f04f00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 10 monclient(hunting): picked mon.noname-c con 0x7f04f0153e00 addr v2:10.20.193.148:3300/0 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 1 -- --> v1:10.20.193.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f00c4820 con 0x7f04f00700f0 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f00c4b00 con 0x7f04f00c86f0 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f04f00c4de0 con 0x7f04f0153e00 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.301+0000 7f04f867d640 10 monclient(hunting): authenticate will time out at 1373.870564s 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.302+0000 7f04f63f2640 1 --1- >> v1:10.20.193.16:6789/0 conn(0x7f04f00700f0 0x7f04f00c7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.16:6789/0 says I am v1:10.20.193.16:51004/0 (socket says 10.20.193.16:51004) 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.302+0000 7f04f63f2640 1 -- 10.20.193.16:0/810008049 learned_addr learned my addr 10.20.193.16:0/810008049 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.302+0000 7f04f6bf3640 1 --2- 10.20.193.16:0/810008049 >> v2:10.20.193.148:3300/0 conn(0x7f04f0153e00 0x7f04f00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.302+0000 7f04f6bf3640 10 monclient(hunting): get_auth_request con 0x7f04f0153e00 auth_method 0 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.302+0000 7f04f6bf3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.302+0000 7f04f6bf3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.302 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.302+0000 7f04f6bf3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.302+0000 7f04f6bf3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.302+0000 7f04f6bf3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.302+0000 7f04f6bf3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f53f0640 1 -- 10.20.193.16:0/810008049 <== mon.0 v1:10.20.193.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396983522 0 0) 0x7f04f00c4820 con 0x7f04f00700f0 2026-02-17T06:02:01.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f53f0640 10 monclient(hunting): _negotiate 2026-02-17T06:02:01.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f53f0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f53f0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f53f0640 10 monclient(hunting): my global_id is 5211 2026-02-17T06:02:01.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f53f0640 1 -- 10.20.193.16:0/810008049 --> v1:10.20.193.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04d8003620 con 0x7f04f00700f0 2026-02-17T06:02:01.303 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f6bf3640 10 monclient(hunting): handle_auth_done global_id 5342 payload 274 2026-02-17T06:02:01.304 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f6bf3640 1 -- 10.20.193.16:0/810008049 >> v1:10.20.193.148:6789/0 conn(0x7f04f00c86f0 legacy=0x7f04f01536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:01.304 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f6bf3640 1 -- 10.20.193.16:0/810008049 >> v1:10.20.193.16:6789/0 conn(0x7f04f00700f0 legacy=0x7f04f00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.304 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f6bf3640 10 monclient: _finish_hunting 0 2026-02-17T06:02:01.304 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f6bf3640 1 monclient: found mon.noname-c 2026-02-17T06:02:01.304 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f6bf3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:01.304 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f6bf3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:02:01.304 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.303+0000 7f04f6bf3640 1 -- 10.20.193.16:0/810008049 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f00c50c0 con 0x7f04f0153e00 2026-02-17T06:02:01.305 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.305+0000 7f04f6bf3640 1 --2- 10.20.193.16:0/810008049 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04f0153e00 0x7f04f00704d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009d90 tx=0x7f04ec02f190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6139dc43b671e1fc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 1 -- 10.20.193.16:0/810008049 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec03c070 con 0x7f04f0153e00 2026-02-17T06:02:01.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:01.313 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 10 monclient: dump: 2026-02-17T06:02:01.313 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.313 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.314 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.314 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.314 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.314 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.314 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.314 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.314 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.314 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 10 monclient: _finish_auth 0 2026-02-17T06:02:01.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 10 monclient: _check_auth_tickets 2026-02-17T06:02:01.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:01.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 1 -- 10.20.193.16:0/810008049 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec02fe10 con 0x7f04f0153e00 2026-02-17T06:02:01.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:01.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 1 -- 10.20.193.16:0/810008049 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec037660 con 0x7f04f0153e00 2026-02-17T06:02:01.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:01.315 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f53f0640 10 monclient: dump: 2026-02-17T06:02:01.315 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.315 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.315 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 5 monclient: authenticate success, global_id 5342 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 4 set_mon_vals no callback set 2026-02-17T06:02:01.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 10 monclient: shutdown 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 1 -- 10.20.193.16:0/810008049 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04f0153e00 msgr2=0x7f04f00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 1 --2- 10.20.193.16:0/810008049 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04f0153e00 0x7f04f00704d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009d90 tx=0x7f04ec02f190 comp rx=0 tx=0).stop 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 1 -- 10.20.193.16:0/810008049 shutdown_connections 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 1 --2- 10.20.193.16:0/810008049 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04f0153e00 0x7f04f00704d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 1 -- 10.20.193.16:0/810008049 >> 10.20.193.16:0/810008049 conn(0x7f04f00cd770 msgr2=0x7f04f00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 1 -- 10.20.193.16:0/810008049 shutdown_connections 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.307+0000 7f04f867d640 1 -- 10.20.193.16:0/810008049 wait complete. 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient: build_initial_monmap 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient: monmap: 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:01.317 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:01.318 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:01.318 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:01.318 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:01.318 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.318 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:01.318 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:01.318 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:01.318 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 librados: starting msgr at 2026-02-17T06:02:01.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 librados: starting objecter 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 Processor -- start 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 -- start start 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 librados: setting wanted keys 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 librados: calling monclient init 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient: init 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04f00700f0 0x7f04f0142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient(hunting): picked mon.noname-b con 0x7f04f00700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04f00c86f0 0x7f04f0151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient(hunting): picked mon.noname-c con 0x7f04f00c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:01.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04f0153e00 0x7f04f01523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient(hunting): picked mon.noname-a con 0x7f04f0153e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0074c00 con 0x7f04f0153e00 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f04f00c5680 con 0x7f04f00700f0 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0074d80 con 0x7f04f00c86f0 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.308+0000 7f04f867d640 10 monclient(hunting): authenticate will time out at 1373.877885s 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f63f2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04f00700f0 0x7f04f0142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f6bf3640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04f0153e00 0x7f04f01523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f5bf1640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04f00c86f0 0x7f04f0151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f63f2640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04f00700f0 0x7f04f0142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:51522/0 (socket says 10.20.193.16:51522) 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f63f2640 1 -- 10.20.193.16:0/2720305063 learned_addr learned my addr 10.20.193.16:0/2720305063 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f63f2640 10 monclient(hunting): get_auth_request con 0x7f04f00700f0 auth_method 0 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f63f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f63f2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f63f2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f6bf3640 10 monclient(hunting): get_auth_request con 0x7f04f0153e00 auth_method 0 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f6bf3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f6bf3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f6bf3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f5bf1640 10 monclient(hunting): get_auth_request con 0x7f04f00c86f0 auth_method 0 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f5bf1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f5bf1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f5bf1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f63f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f63f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f63f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f5bf1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f5bf1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f5bf1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f6bf3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f6bf3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.309+0000 7f04f6bf3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.322 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.310+0000 7f04f63f2640 10 monclient(hunting): handle_auth_done global_id 5305 payload 824 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.310+0000 7f04f63f2640 1 -- 10.20.193.16:0/2720305063 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04f00c86f0 msgr2=0x7f04f0151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.310+0000 7f04f63f2640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04f00c86f0 0x7f04f0151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.310+0000 7f04f63f2640 1 -- 10.20.193.16:0/2720305063 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04f0153e00 msgr2=0x7f04f01523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.310+0000 7f04f63f2640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04f0153e00 0x7f04f01523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.310+0000 7f04f63f2640 10 monclient: _finish_hunting 0 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.310+0000 7f04f63f2640 1 monclient: found mon.noname-b 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.310+0000 7f04f63f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.310+0000 7f04f63f2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.310+0000 7f04f63f2640 1 -- 10.20.193.16:0/2720305063 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f01e6170 con 0x7f04f00700f0 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.311+0000 7f04f63f2640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04f00700f0 0x7f04f0142770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c520 tx=0x7f04e000c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.311+0000 7f04d6ffd640 10 client.?.objecter ms_handle_connect 0x7f04f00700f0 2026-02-17T06:02:01.323 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.311+0000 7f04d6ffd640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:01.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 1 -- 10.20.193.16:0/2720305063 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0017070 con 0x7f04f00700f0 2026-02-17T06:02:01.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:01.326 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 10 monclient: dump: 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 10 monclient: _finish_auth 0 2026-02-17T06:02:01.327 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 5 monclient: authenticate success, global_id 5305 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 10 monclient: _renew_subs 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 1 -- 10.20.193.16:0/2720305063 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f01e6450 con 0x7f04f00700f0 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 10 client.5305.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 10 monclient: _renew_subs 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 1 -- 10.20.193.16:0/2720305063 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f01516a0 con 0x7f04f00700f0 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 1 librados: init done 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 1 -- 10.20.193.16:0/2720305063 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e0002a20 con 0x7f04f00700f0 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:01.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 1 -- 10.20.193.16:0/2720305063 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0021060 con 0x7f04f00700f0 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04d6ffd640 10 monclient: dump: 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f4bef640 4 set_mon_vals no callback set 2026-02-17T06:02:01.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:01.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:01.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:01.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.313+0000 7f04f867d640 1 -- 10.20.193.16:0/2720305063 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f00704d0 con 0x7f04f00700f0 2026-02-17T06:02:01.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.324+0000 7f04d6ffd640 1 -- 10.20.193.16:0/2720305063 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f04e0024080 con 0x7f04f00700f0 2026-02-17T06:02:01.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.325+0000 7f04d6ffd640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04c0079140 0x7f04c007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.325+0000 7f04f5bf1640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04c0079140 0x7f04c007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.325+0000 7f04f5bf1640 10 monclient: get_auth_request con 0x7f04c0079140 auth_method 0 2026-02-17T06:02:01.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.325+0000 7f04d6ffd640 10 client.5305.objecter ms_handle_connect 0x7f04c0079140 2026-02-17T06:02:01.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.325+0000 7f04f5bf1640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04c0079140 0x7f04c007b600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f04f01e6c10 tx=0x7f04e4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.326+0000 7f04d6ffd640 1 -- 10.20.193.16:0/2720305063 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f04e005d6b0 con 0x7f04f00700f0 2026-02-17T06:02:01.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.326+0000 7f04d6ffd640 10 client.5305.objecter ms_dispatch2 0x7f04f0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:01.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.326+0000 7f04d6ffd640 3 client.5305.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:01.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.326+0000 7f04d6ffd640 3 client.5305.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:01.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.326+0000 7f04d6ffd640 20 client.5305.objecter dump_active .. 0 homeless 2026-02-17T06:02:01.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.430+0000 7f04d6ffd640 1 -- 10.20.193.16:0/2720305063 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f04e0061670 con 0x7f04f00700f0 2026-02-17T06:02:01.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.430+0000 7f04d6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:01.430 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.430+0000 7f04d6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:01.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.512+0000 7f04f867d640 1 -- 10.20.193.16:0/2720305063 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "11", "target": ["mon-mgr", ""]}) -- 0x7f04f01514e0 con 0x7f04c0079140 2026-02-17T06:02:01.513 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.512+0000 7f04d6ffd640 1 -- 10.20.193.16:0/2720305063 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 11 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7f04e4002c50 con 0x7f04c0079140 2026-02-17T06:02:01.513 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 11 to deep-scrub 2026-02-17T06:02:01.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 10 librados: watch_flush enter 2026-02-17T06:02:01.515 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 10 librados: watch_flush exit 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 20 client.5305.objecter shutdown clearing up homeless session... 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 10 client.5305.objecter successfully canceled tick 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 -- 10.20.193.16:0/2720305063 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04c0079140 msgr2=0x7f04c007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04c0079140 0x7f04c007b600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f04f01e6c10 tx=0x7f04e4009190 comp rx=0 tx=0).stop 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 10 monclient: shutdown 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 -- 10.20.193.16:0/2720305063 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04f00700f0 msgr2=0x7f04f0142770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04f00700f0 0x7f04f0142770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c520 tx=0x7f04e000c9f0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 -- 10.20.193.16:0/2720305063 shutdown_connections 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f04c0079140 0x7f04c007b600 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.516 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f04f0153e00 0x7f04f01523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f04f00c86f0 0x7f04f0151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 --2- 10.20.193.16:0/2720305063 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f04f00700f0 0x7f04f0142770 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 -- 10.20.193.16:0/2720305063 >> 10.20.193.16:0/2720305063 conn(0x7f04f00cd770 msgr2=0x7f04f00cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:01.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 -- 10.20.193.16:0/2720305063 shutdown_connections 2026-02-17T06:02:01.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 -- 10.20.193.16:0/2720305063 wait complete. 2026-02-17T06:02:01.517 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.513+0000 7f04f867d640 1 librados: shutdown 2026-02-17T06:02:01.523 INFO:tasks.ceph:Scrubbing osd.15 2026-02-17T06:02:01.523 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:02:01.592 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.591+0000 7f0c3b65b640 10 monclient: get_monmap_and_config 2026-02-17T06:02:01.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.591+0000 7f0c3b65b640 10 monclient: build_initial_monmap 2026-02-17T06:02:01.593 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 10 monclient: monmap: 2026-02-17T06:02:01.593 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:01.593 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.593 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:01.593008+0000 2026-02-17T06:02:01.593 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:01.593008+0000 2026-02-17T06:02:01.594 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:01.594 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.594 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:01.594 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:01.594 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:01.594 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:01.594 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:01.594 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:01.594 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.594 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 1 Processor -- start 2026-02-17T06:02:01.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 1 -- start start 2026-02-17T06:02:01.595 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 10 monclient: init 2026-02-17T06:02:01.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:01.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 10 monclient: _add_conns ranks=[2,1,5,0,4,3] 2026-02-17T06:02:01.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0c340700f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:02:01.598 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0c340c86f0 addr v1:10.20.193.128:6789/0 2026-02-17T06:02:01.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f0c34153e00 0x7f0c340704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 10 monclient(hunting): picked mon.noname-c con 0x7f0c34153e00 addr v2:10.20.193.148:3300/0 2026-02-17T06:02:01.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c340c4820 con 0x7f0c340c86f0 2026-02-17T06:02:01.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c340c4b00 con 0x7f0c340700f0 2026-02-17T06:02:01.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f0c340c4de0 con 0x7f0c34153e00 2026-02-17T06:02:01.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:01.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.592+0000 7f0c3b65b640 10 monclient(hunting): authenticate will time out at 1374.161990s 2026-02-17T06:02:01.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.593+0000 7f0c38bcf640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f0c340c86f0 0x7f0c341536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:45996/0 (socket says 10.20.193.16:45996) 2026-02-17T06:02:01.599 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.593+0000 7f0c38bcf640 1 -- 10.20.193.16:0/1555851672 learned_addr learned my addr 10.20.193.16:0/1555851672 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:02:01.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.593+0000 7f0c39bd1640 1 --2- 10.20.193.16:0/1555851672 >> v2:10.20.193.148:3300/0 conn(0x7f0c34153e00 0x7f0c340704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.593+0000 7f0c39bd1640 10 monclient(hunting): get_auth_request con 0x7f0c34153e00 auth_method 0 2026-02-17T06:02:01.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.593+0000 7f0c39bd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.593+0000 7f0c39bd1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.593+0000 7f0c39bd1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.594+0000 7f0c39bd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.594+0000 7f0c39bd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.594+0000 7f0c39bd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.594+0000 7f0c23fff640 1 -- 10.20.193.16:0/1555851672 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2200103695 0 0) 0x7f0c340c4820 con 0x7f0c340c86f0 2026-02-17T06:02:01.600 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.594+0000 7f0c23fff640 10 monclient(hunting): _negotiate 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.594+0000 7f0c23fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.594+0000 7f0c23fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.594+0000 7f0c23fff640 10 monclient(hunting): my global_id is 5308 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.594+0000 7f0c23fff640 1 -- 10.20.193.16:0/1555851672 --> v1:10.20.193.128:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c1c003620 con 0x7f0c340c86f0 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.595+0000 7f0c39bd1640 10 monclient(hunting): handle_auth_done global_id 5348 payload 274 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.595+0000 7f0c39bd1640 1 -- 10.20.193.16:0/1555851672 >> v1:10.20.193.148:6789/0 conn(0x7f0c340700f0 legacy=0x7f0c340c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.595+0000 7f0c39bd1640 1 -- 10.20.193.16:0/1555851672 >> v1:10.20.193.128:6789/0 conn(0x7f0c340c86f0 legacy=0x7f0c341536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.595+0000 7f0c39bd1640 10 monclient: _finish_hunting 0 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.595+0000 7f0c39bd1640 1 monclient: found mon.noname-c 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.595+0000 7f0c39bd1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.595+0000 7f0c39bd1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:02:01.601 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.595+0000 7f0c39bd1640 1 -- 10.20.193.16:0/1555851672 --> v2:10.20.193.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c340c50c0 con 0x7f0c34153e00 2026-02-17T06:02:01.602 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.596+0000 7f0c39bd1640 1 --2- 10.20.193.16:0/1555851672 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0c34153e00 0x7f0c340704d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009d90 tx=0x7f0c3002f190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=698ea0c77f8244b0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 1 -- 10.20.193.16:0/1555851672 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3003c070 con 0x7f0c34153e00 2026-02-17T06:02:01.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:01.608 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 10 monclient: dump: 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 10 monclient: _finish_auth 0 2026-02-17T06:02:01.609 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 10 monclient: _check_auth_tickets 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 1 -- 10.20.193.16:0/1555851672 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3002fe10 con 0x7f0c34153e00 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 5 monclient: authenticate success, global_id 5348 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 1 -- 10.20.193.16:0/1555851672 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30037660 con 0x7f0c34153e00 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 4 set_mon_vals no callback set 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c23fff640 10 monclient: dump: 2026-02-17T06:02:01.610 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 10 monclient: shutdown 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:01.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 1 -- 10.20.193.16:0/1555851672 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0c34153e00 msgr2=0x7f0c340704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 1 --2- 10.20.193.16:0/1555851672 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0c34153e00 0x7f0c340704d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009d90 tx=0x7f0c3002f190 comp rx=0 tx=0).stop 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 1 -- 10.20.193.16:0/1555851672 shutdown_connections 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 1 --2- 10.20.193.16:0/1555851672 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0c34153e00 0x7f0c340704d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 1 -- 10.20.193.16:0/1555851672 >> 10.20.193.16:0/1555851672 conn(0x7f0c340cd770 msgr2=0x7f0c340cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 1 -- 10.20.193.16:0/1555851672 shutdown_connections 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.598+0000 7f0c3b65b640 1 -- 10.20.193.16:0/1555851672 wait complete. 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient: build_initial_monmap 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient: monmap: 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:01.612 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 librados: starting msgr at 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 librados: starting objecter 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 Processor -- start 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 -- start start 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 librados: setting wanted keys 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 librados: calling monclient init 2026-02-17T06:02:01.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient: init 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0c340700f0 0x7f0c340744e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient(hunting): picked mon.noname-b con 0x7f0c340700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0c340c86f0 0x7f0c340cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient(hunting): picked mon.noname-c con 0x7f0c340c86f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0c34153e00 0x7f0c3406d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient(hunting): picked mon.noname-a con 0x7f0c34153e00 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34074c00 con 0x7f0c34153e00 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f0c340c5680 con 0x7f0c340700f0 2026-02-17T06:02:01.614 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34074d80 con 0x7f0c340c86f0 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.599+0000 7f0c3b65b640 10 monclient(hunting): authenticate will time out at 1374.168899s 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c393d0640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0c340700f0 0x7f0c340744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c39bd1640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0c34153e00 0x7f0c3406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c38bcf640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0c340c86f0 0x7f0c340cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c393d0640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0c340700f0 0x7f0c340744e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:51530/0 (socket says 10.20.193.16:51530) 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c393d0640 1 -- 10.20.193.16:0/854764026 learned_addr learned my addr 10.20.193.16:0/854764026 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c393d0640 10 monclient(hunting): get_auth_request con 0x7f0c340700f0 auth_method 0 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c393d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c393d0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c393d0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c39bd1640 10 monclient(hunting): get_auth_request con 0x7f0c34153e00 auth_method 0 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c39bd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c39bd1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c39bd1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c38bcf640 10 monclient(hunting): get_auth_request con 0x7f0c340c86f0 auth_method 0 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c38bcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c38bcf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c38bcf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c38bcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c38bcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c38bcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c393d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c393d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c393d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c39bd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c39bd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.600+0000 7f0c39bd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.602+0000 7f0c38bcf640 10 monclient(hunting): handle_auth_done global_id 5351 payload 824 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.602+0000 7f0c38bcf640 1 -- 10.20.193.16:0/854764026 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0c340700f0 msgr2=0x7f0c340744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.602+0000 7f0c38bcf640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f0c340700f0 0x7f0c340744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.602+0000 7f0c38bcf640 1 -- 10.20.193.16:0/854764026 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0c34153e00 msgr2=0x7f0c3406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.602+0000 7f0c38bcf640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f0c34153e00 0x7f0c3406d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.602+0000 7f0c38bcf640 10 monclient: _finish_hunting 0 2026-02-17T06:02:01.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.602+0000 7f0c38bcf640 1 monclient: found mon.noname-c 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.602+0000 7f0c38bcf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.602+0000 7f0c38bcf640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.148:3300/0 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.602+0000 7f0c38bcf640 1 -- 10.20.193.16:0/854764026 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3406e7c0 con 0x7f0c340c86f0 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c38bcf640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0c340c86f0 0x7f0c340cc9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800c480 tx=0x7f0c2800c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c21ffb640 10 client.?.objecter ms_handle_connect 0x7f0c340c86f0 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c21ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c21ffb640 1 -- 10.20.193.16:0/854764026 <== mon.2 v2:10.20.193.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28017070 con 0x7f0c340c86f0 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c21ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c21ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c21ffb640 10 monclient: dump: 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.618 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c21ffb640 10 monclient: _finish_auth 0 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c21ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:02:01.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c21ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c3b65b640 5 monclient: authenticate success, global_id 5351 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c3b65b640 10 monclient: _renew_subs 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c3b65b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c340ccf30 con 0x7f0c340c86f0 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c3b65b640 10 client.5351.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c3b65b640 10 monclient: _renew_subs 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c3b65b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c341f3c90 con 0x7f0c340c86f0 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.603+0000 7f0c3b65b640 1 librados: init done 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c3b65b640 10 client.5351.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c3b65b640 15 client.5351.objecter _session_command_op_assign -1 1 2026-02-17T06:02:01.620 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c3b65b640 20 client.5351.objecter _get_session osd=-1 returning homeless 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c3b65b640 10 client.5351.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c3b65b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.148:3300/0 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c341f48a0 con 0x7f0c340c86f0 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c21ffb640 1 -- 10.20.193.16:0/854764026 <== mon.2 v2:10.20.193.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c341f48a0 con 0x7f0c340c86f0 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c21ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c21ffb640 1 -- 10.20.193.16:0/854764026 <== mon.2 v2:10.20.193.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28021060 con 0x7f0c340c86f0 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c21ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c21ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c21ffb640 10 monclient: dump: 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.621 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.622 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.622 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.622 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.622 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.622 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.622 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.604+0000 7f0c237fe640 4 set_mon_vals no callback set 2026-02-17T06:02:01.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.617+0000 7f0c21ffb640 1 -- 10.20.193.16:0/854764026 <== mon.2 v2:10.20.193.148:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f0c28007070 con 0x7f0c340c86f0 2026-02-17T06:02:01.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.617+0000 7f0c21ffb640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0c04079140 0x7f0c0407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.624 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.617+0000 7f0c393d0640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0c04079140 0x7f0c0407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.617+0000 7f0c393d0640 10 monclient: get_auth_request con 0x7f0c04079140 auth_method 0 2026-02-17T06:02:01.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.618+0000 7f0c393d0640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0c04079140 0x7f0c0407b600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0c34152f90 tx=0x7f0c24009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.618+0000 7f0c21ffb640 10 client.5351.objecter ms_handle_connect 0x7f0c04079140 2026-02-17T06:02:01.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 1 -- 10.20.193.16:0/854764026 <== mon.2 v2:10.20.193.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f0c280975a0 con 0x7f0c340c86f0 2026-02-17T06:02:01.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 10 client.5351.objecter ms_dispatch2 0x7f0c34000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:01.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 3 client.5351.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:01.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 3 client.5351.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:01.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 10 client.5351.objecter checking command 1 2026-02-17T06:02:01.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] conn(0x7f0c04081620 0x7f0c04083a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 20 client.5351.objecter _get_session s=0x7f0c04080f50 osd=15 3 2026-02-17T06:02:01.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 20 client.5351.objecter put_session s=0x7f0c04080f50 osd=15 3 2026-02-17T06:02:01.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 15 client.5351.objecter _session_command_op_remove -1 1 2026-02-17T06:02:01.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 20 client.5351.objecter _get_session s=0x7f0c04080f50 osd=15 3 2026-02-17T06:02:01.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 20 client.5351.objecter get_session s=0x7f0c04080f50 osd=15 3 2026-02-17T06:02:01.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 15 client.5351.objecter _session_command_op_assign 15 1 2026-02-17T06:02:01.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 20 client.5351.objecter put_session s=0x7f0c04080f50 osd=15 4 2026-02-17T06:02:01.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 10 client.5351.objecter _send_command 1 2026-02-17T06:02:01.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 1 -- 10.20.193.16:0/854764026 --> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c04084110 con 0x7f0c04081620 2026-02-17T06:02:01.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 20 client.5351.objecter dump_active .. 0 homeless 2026-02-17T06:02:01.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 1 -- 10.20.193.16:0/854764026 <== mon.2 v2:10.20.193.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c2805def0 con 0x7f0c340c86f0 2026-02-17T06:02:01.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c21ffb640 10 monclient: handle_get_version_reply finishing 1 version 34 2026-02-17T06:02:01.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.619+0000 7f0c39bd1640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] conn(0x7f0c04081620 0x7f0c04083a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.620+0000 7f0c39bd1640 10 monclient: get_auth_request con 0x7f0c04081620 auth_method 0 2026-02-17T06:02:01.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.622+0000 7f0c39bd1640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] conn(0x7f0c04081620 0x7f0c04083a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.15 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.622+0000 7f0c21ffb640 10 client.5351.objecter ms_handle_connect 0x7f0c04081620 2026-02-17T06:02:01.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.633+0000 7f0c21ffb640 1 -- 10.20.193.16:0/854764026 <== osd.15 v2:10.20.193.164:6800/1881720827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f0c04084110 con 0x7f0c04081620 2026-02-17T06:02:01.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.633+0000 7f0c21ffb640 10 client.5351.objecter ms_dispatch2 0x7f0c34000b90 command_reply(tid 1: 0 ) 2026-02-17T06:02:01.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.633+0000 7f0c21ffb640 10 client.5351.objecter _finish_command 1 = system:0 2026-02-17T06:02:01.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.633+0000 7f0c21ffb640 20 client.5351.objecter put_session s=0x7f0c04080f50 osd=15 4 2026-02-17T06:02:01.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.633+0000 7f0c21ffb640 15 client.5351.objecter _session_command_op_remove 15 1 2026-02-17T06:02:01.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 10 client.5351.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-17T06:02:01.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 15 client.5351.objecter _session_command_op_assign -1 2 2026-02-17T06:02:01.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 20 client.5351.objecter _get_session s=0x7f0c04080f50 osd=15 3 2026-02-17T06:02:01.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 20 client.5351.objecter put_session s=0x7f0c04080f50 osd=15 3 2026-02-17T06:02:01.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 20 client.5351.objecter _get_session s=0x7f0c04080f50 osd=15 3 2026-02-17T06:02:01.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 15 client.5351.objecter _session_command_op_remove -1 2 2026-02-17T06:02:01.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 20 client.5351.objecter get_session s=0x7f0c04080f50 osd=15 3 2026-02-17T06:02:01.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 15 client.5351.objecter _session_command_op_assign 15 2 2026-02-17T06:02:01.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 20 client.5351.objecter put_session s=0x7f0c04080f50 osd=15 4 2026-02-17T06:02:01.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 10 client.5351.objecter _send_command 2 2026-02-17T06:02:01.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.638+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 --> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0c34074a20 con 0x7f0c04081620 2026-02-17T06:02:01.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c21ffb640 1 -- 10.20.193.16:0/854764026 <== osd.15 v2:10.20.193.164:6800/1881720827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0c34074a20 con 0x7f0c04081620 2026-02-17T06:02:01.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c21ffb640 10 client.5351.objecter ms_dispatch2 0x7f0c34000b90 command_reply(tid 2: 0 ) 2026-02-17T06:02:01.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c21ffb640 10 client.5351.objecter _finish_command 2 = system:0 2026-02-17T06:02:01.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c21ffb640 20 client.5351.objecter put_session s=0x7f0c04080f50 osd=15 4 2026-02-17T06:02:01.646 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c21ffb640 15 client.5351.objecter _session_command_op_remove 15 2 2026-02-17T06:02:01.646 INFO:teuthology.orchestra.run.trial016.stdout:{ 2026-02-17T06:02:01.646 INFO:teuthology.orchestra.run.trial016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:02:01.647 INFO:teuthology.orchestra.run.trial016.stdout:} 2026-02-17T06:02:01.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 10 librados: watch_flush enter 2026-02-17T06:02:01.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 10 librados: watch_flush exit 2026-02-17T06:02:01.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 10 client.5351.objecter close_session for osd.15 2026-02-17T06:02:01.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 >> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] conn(0x7f0c04081620 msgr2=0x7f0c04083a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.164:6800/1881720827,v1:10.20.193.164:6801/1881720827] conn(0x7f0c04081620 0x7f0c04083a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 20 client.5351.objecter put_session s=0x7f0c04080f50 osd=15 1 2026-02-17T06:02:01.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 20 client.5351.objecter shutdown clearing up homeless session... 2026-02-17T06:02:01.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 10 client.5351.objecter successfully canceled tick 2026-02-17T06:02:01.648 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0c04079140 msgr2=0x7f0c0407b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f0c04079140 0x7f0c0407b600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0c34152f90 tx=0x7f0c24009210 comp rx=0 tx=0).stop 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 10 monclient: shutdown 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0c340c86f0 msgr2=0x7f0c340cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 1 --2- 10.20.193.16:0/854764026 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f0c340c86f0 0x7f0c340cc9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800c480 tx=0x7f0c2800c950 comp rx=0 tx=0).stop 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c39bd1640 1 -- 10.20.193.16:0/854764026 reap_dead start 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 shutdown_connections 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 >> 10.20.193.16:0/854764026 conn(0x7f0c340cd770 msgr2=0x7f0c34145fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 shutdown_connections 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.645+0000 7f0c3b65b640 1 -- 10.20.193.16:0/854764026 wait complete. 2026-02-17T06:02:01.649 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.646+0000 7f0c3b65b640 1 librados: shutdown 2026-02-17T06:02:01.654 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.724+0000 7f83d9038640 10 monclient: get_monmap_and_config 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.724+0000 7f83d9038640 10 monclient: build_initial_monmap 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.724+0000 7f83d9038640 10 monclient: monmap: 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:01.725295+0000 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:01.725295+0000 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:01.724 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.724+0000 7f83d9038640 1 Processor -- start 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.724+0000 7f83d9038640 1 -- start start 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.724+0000 7f83d9038640 10 monclient: init 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 10 monclient: _add_conns ranks=[3,2,4,0,5,1] 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f83d40700f0 0x7f83d40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 10 monclient(hunting): picked mon.noname-a con 0x7f83d40700f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f83d40c7fe0 addr v1:10.20.193.148:6789/0 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f83d41536f0 0x7f83d4153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 10 monclient(hunting): picked mon.noname-b con 0x7f83d41536f0 addr v2:10.20.193.128:3300/0 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d40c4820 con 0x7f83d40c7fe0 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f83d40c4b00 con 0x7f83d40700f0 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f83d40c4de0 con 0x7f83d41536f0 2026-02-17T06:02:01.725 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d9038640 10 monclient(hunting): authenticate will time out at 1374.294319s 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d2d76640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f83d40700f0 0x7f83d40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d3577640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f83d41536f0 0x7f83d4153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d2575640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f83d40c7fe0 0x7f83d40c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:36266/0 (socket says 10.20.193.16:36266) 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d2d76640 10 monclient(hunting): get_auth_request con 0x7f83d40700f0 auth_method 0 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d2575640 1 -- 10.20.193.16:0/43670179 learned_addr learned my addr 10.20.193.16:0/43670179 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d3577640 10 monclient(hunting): get_auth_request con 0x7f83d41536f0 auth_method 0 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d3577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d3577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.725+0000 7f83d3577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.726+0000 7f83d2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.726+0000 7f83d2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.726+0000 7f83d2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.726+0000 7f83d3577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.726+0000 7f83d3577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.726 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.726+0000 7f83d3577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d2d76640 10 monclient(hunting): handle_auth_done global_id 5220 payload 274 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d1d74640 1 -- 10.20.193.16:0/43670179 <== mon.2 v1:10.20.193.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3662882121 0 0) 0x7f83d40c4820 con 0x7f83d40c7fe0 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d2d76640 1 -- 10.20.193.16:0/43670179 >> v2:10.20.193.128:3300/0 conn(0x7f83d41536f0 msgr2=0x7f83d4153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d2d76640 1 --2- 10.20.193.16:0/43670179 >> v2:10.20.193.128:3300/0 conn(0x7f83d41536f0 0x7f83d4153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d2d76640 1 -- 10.20.193.16:0/43670179 >> v1:10.20.193.148:6789/0 conn(0x7f83d40c7fe0 legacy=0x7f83d40c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d2d76640 10 monclient: _finish_hunting 0 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d2d76640 1 monclient: found mon.noname-a 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d2d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d2d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d2d76640 1 -- 10.20.193.16:0/43670179 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40c50c0 con 0x7f83d40700f0 2026-02-17T06:02:01.727 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d1d74640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T06:02:01.728 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.727+0000 7f83d2d76640 1 --2- 10.20.193.16:0/43670179 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40700f0 0x7f83d40704d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f83d40c9540 tx=0x7f83c802ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3898603bd54b42b2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 1 -- 10.20.193.16:0/43670179 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c803c070 con 0x7f83d40700f0 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 10 monclient: dump: 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 10 monclient: _finish_auth 0 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 10 monclient: _check_auth_tickets 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 1 -- 10.20.193.16:0/43670179 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c802fa60 con 0x7f83d40700f0 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 1 -- 10.20.193.16:0/43670179 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c802fe30 con 0x7f83d40700f0 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d1d74640 10 monclient: dump: 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d9038640 5 monclient: authenticate success, global_id 5220 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d9038640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:01.730 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.729+0000 7f83d9038640 4 set_mon_vals no callback set 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 10 monclient: shutdown 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 -- 10.20.193.16:0/43670179 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40700f0 msgr2=0x7f83d40704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 --2- 10.20.193.16:0/43670179 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40700f0 0x7f83d40704d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f83d40c9540 tx=0x7f83c802ed50 comp rx=0 tx=0).stop 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 -- 10.20.193.16:0/43670179 shutdown_connections 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 --2- 10.20.193.16:0/43670179 >> v2:10.20.193.128:3300/0 conn(0x7f83d41536f0 0x7f83d4153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 --2- 10.20.193.16:0/43670179 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40700f0 0x7f83d40704d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 -- 10.20.193.16:0/43670179 >> 10.20.193.16:0/43670179 conn(0x7f83d40cd770 msgr2=0x7f83d40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 -- 10.20.193.16:0/43670179 shutdown_connections 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 -- 10.20.193.16:0/43670179 wait complete. 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 10 monclient: build_initial_monmap 2026-02-17T06:02:01.731 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 10 monclient: monmap: 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 librados: starting msgr at 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 librados: starting objecter 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 Processor -- start 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 -- start start 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 librados: setting wanted keys 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 1 librados: calling monclient init 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.730+0000 7f83d9038640 10 monclient: init 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f83d40700f0 0x7f83d4144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 10 monclient(hunting): picked mon.noname-c con 0x7f83d40700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40c7fe0 0x7f83d41e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 10 monclient(hunting): picked mon.noname-a con 0x7f83d40c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f83d41536f0 0x7f83d40cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 10 monclient(hunting): picked mon.noname-b con 0x7f83d41536f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4154060 con 0x7f83d40c7fe0 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40c5680 con 0x7f83d41536f0 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4074c00 con 0x7f83d40700f0 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d9038640 10 monclient(hunting): authenticate will time out at 1374.300265s 2026-02-17T06:02:01.732 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40c7fe0 0x7f83d41e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d3577640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f83d41536f0 0x7f83d40cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2575640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40c7fe0 0x7f83d41e89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:43680/0 (socket says 10.20.193.16:43680) 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2575640 1 -- 10.20.193.16:0/4070733855 learned_addr learned my addr 10.20.193.16:0/4070733855 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2575640 10 monclient(hunting): get_auth_request con 0x7f83d40c7fe0 auth_method 0 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2d76640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f83d40700f0 0x7f83d4144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d3577640 10 monclient(hunting): get_auth_request con 0x7f83d41536f0 auth_method 0 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d3577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d3577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d3577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2d76640 10 monclient(hunting): get_auth_request con 0x7f83d40700f0 auth_method 0 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.731+0000 7f83d2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.732+0000 7f83d3577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.732+0000 7f83d3577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.732+0000 7f83d3577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.732+0000 7f83d2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.732+0000 7f83d2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:01.733 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.732+0000 7f83d2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.733+0000 7f83d2575640 10 monclient(hunting): handle_auth_done global_id 5223 payload 824 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.733+0000 7f83d2575640 1 -- 10.20.193.16:0/4070733855 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f83d40700f0 msgr2=0x7f83d4144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.733+0000 7f83d2575640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f83d40700f0 0x7f83d4144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.733+0000 7f83d2575640 1 -- 10.20.193.16:0/4070733855 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f83d41536f0 msgr2=0x7f83d40cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.733+0000 7f83d2575640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f83d41536f0 0x7f83d40cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.733+0000 7f83d2575640 10 monclient: _finish_hunting 0 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.733+0000 7f83d2575640 1 monclient: found mon.noname-a 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.733+0000 7f83d2575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.733+0000 7f83d2575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.733+0000 7f83d2575640 1 -- 10.20.193.16:0/4070733855 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d406e5c0 con 0x7f83d40c7fe0 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.734+0000 7f83d2575640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40c7fe0 0x7f83d41e89a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00c4d0 tx=0x7f83bc00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.734+0000 7f83b37fe640 10 client.?.objecter ms_handle_connect 0x7f83d40c7fe0 2026-02-17T06:02:01.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.734+0000 7f83b37fe640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 1 -- 10.20.193.16:0/4070733855 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc018070 con 0x7f83d40c7fe0 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 10 monclient: dump: 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 10 monclient: _finish_auth 0 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 10 monclient: _check_auth_tickets 2026-02-17T06:02:01.736 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 1 -- 10.20.193.16:0/4070733855 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83bc004cf0 con 0x7f83d40c7fe0 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 1 -- 10.20.193.16:0/4070733855 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0072f0 con 0x7f83d40c7fe0 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 5 monclient: authenticate success, global_id 5223 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d1573640 4 set_mon_vals no callback set 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 10 monclient: _renew_subs 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 1 -- 10.20.193.16:0/4070733855 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d406e8a0 con 0x7f83d40c7fe0 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83b37fe640 10 monclient: dump: 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 10 client.5223.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 10 monclient: _renew_subs 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 1 -- 10.20.193.16:0/4070733855 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d406ee60 con 0x7f83d40c7fe0 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 1 librados: init done 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:01.737 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.736+0000 7f83d9038640 1 -- 10.20.193.16:0/4070733855 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d40704d0 con 0x7f83d40c7fe0 2026-02-17T06:02:01.748 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.748+0000 7f83b37fe640 1 -- 10.20.193.16:0/4070733855 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f83bc007490 con 0x7f83d40c7fe0 2026-02-17T06:02:01.748 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.748+0000 7f83b37fe640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83a0079190 0x7f83a007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:01.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.748+0000 7f83d2d76640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83a0079190 0x7f83a007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:01.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.749+0000 7f83d2d76640 10 monclient: get_auth_request con 0x7f83a0079190 auth_method 0 2026-02-17T06:02:01.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.749+0000 7f83d2d76640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83a0079190 0x7f83a007b650 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f83c8039000 tx=0x7f83c8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:01.749 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.749+0000 7f83b37fe640 10 client.5223.objecter ms_handle_connect 0x7f83a0079190 2026-02-17T06:02:01.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.852+0000 7f83b37fe640 1 -- 10.20.193.16:0/4070733855 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f83bc05d660 con 0x7f83d40c7fe0 2026-02-17T06:02:01.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.852+0000 7f83b37fe640 10 client.5223.objecter ms_dispatch2 0x7f83d4000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:01.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.852+0000 7f83b37fe640 3 client.5223.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:01.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.852+0000 7f83b37fe640 3 client.5223.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:01.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.852+0000 7f83b37fe640 20 client.5223.objecter dump_active .. 0 homeless 2026-02-17T06:02:01.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.852+0000 7f83b37fe640 1 -- 10.20.193.16:0/4070733855 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f83bc022020 con 0x7f83d40c7fe0 2026-02-17T06:02:01.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.852+0000 7f83b37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:01.852 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.852+0000 7f83b37fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:01.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.933+0000 7f83d9038640 1 -- 10.20.193.16:0/4070733855 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "15", "target": ["mon-mgr", ""]}) -- 0x7f83d41573f0 con 0x7f83a0079190 2026-02-17T06:02:01.934 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.934+0000 7f83b37fe640 1 -- 10.20.193.16:0/4070733855 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 15 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7f83d41573f0 con 0x7f83a0079190 2026-02-17T06:02:01.935 INFO:teuthology.orchestra.run.trial016.stderr:instructed osd(s) 15 to deep-scrub 2026-02-17T06:02:01.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.934+0000 7f83d9038640 10 librados: watch_flush enter 2026-02-17T06:02:01.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.934+0000 7f83d9038640 10 librados: watch_flush exit 2026-02-17T06:02:01.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.934+0000 7f83d9038640 20 client.5223.objecter shutdown clearing up homeless session... 2026-02-17T06:02:01.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.934+0000 7f83d9038640 10 client.5223.objecter successfully canceled tick 2026-02-17T06:02:01.936 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 -- 10.20.193.16:0/4070733855 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83a0079190 msgr2=0x7f83a007b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83a0079190 0x7f83a007b650 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f83c8039000 tx=0x7f83c8033000 comp rx=0 tx=0).stop 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 10 monclient: shutdown 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 -- 10.20.193.16:0/4070733855 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40c7fe0 msgr2=0x7f83d41e89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40c7fe0 0x7f83d41e89a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00c4d0 tx=0x7f83bc00c9a0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 -- 10.20.193.16:0/4070733855 shutdown_connections 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f83a0079190 0x7f83a007b650 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f83d41536f0 0x7f83d40cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f83d40c7fe0 0x7f83d41e89a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 --2- 10.20.193.16:0/4070733855 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f83d40700f0 0x7f83d4144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:01.937 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 -- 10.20.193.16:0/4070733855 >> 10.20.193.16:0/4070733855 conn(0x7f83d40cd770 msgr2=0x7f83d414f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:01.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 -- 10.20.193.16:0/4070733855 shutdown_connections 2026-02-17T06:02:01.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 -- 10.20.193.16:0/4070733855 wait complete. 2026-02-17T06:02:01.938 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:01.935+0000 7f83d9038640 1 librados: shutdown 2026-02-17T06:02:01.944 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.012+0000 7f1a33761640 10 monclient: get_monmap_and_config 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.012+0000 7f1a33761640 10 monclient: build_initial_monmap 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.012+0000 7f1a33761640 10 monclient: monmap: 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:02.013064+0000 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:02.013064+0000 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:02.012 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:02.013 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:02.013 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:02.013 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:02.013 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.012+0000 7f1a33761640 1 Processor -- start 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.012+0000 7f1a33761640 1 -- start start 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.012+0000 7f1a33761640 10 monclient: init 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.012+0000 7f1a33761640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.012+0000 7f1a33761640 10 monclient: _add_conns ranks=[3,4,2,0,5,1] 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.012+0000 7f1a33761640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1a2c0700f0 0x7f1a2c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 10 monclient(hunting): picked mon.noname-a con 0x7f1a2c0700f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1a2c0c7fe0 0x7f1a2c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 10 monclient(hunting): picked mon.noname-b con 0x7f1a2c0c7fe0 addr v2:10.20.193.128:3300/0 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1a2c1536f0 addr v1:10.20.193.148:6789/0 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a2c0c8970 con 0x7f1a2c1536f0 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1a2c0c4820 con 0x7f1a2c0700f0 2026-02-17T06:02:02.014 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 1 -- --> v2:10.20.193.128:3300/0 -- mon_getmap magic: 0 -- 0x7f1a2c0c4b00 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a33761640 10 monclient(hunting): authenticate will time out at 1374.582040s 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a30cd5640 1 --2- >> v2:10.20.193.128:3300/0 conn(0x7f1a2c0c7fe0 0x7f1a2c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a314d6640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f1a2c0700f0 0x7f1a2c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a31cd7640 1 --1- >> v1:10.20.193.148:6789/0 conn(0x7f1a2c1536f0 0x7f1a2c153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.148:6789/0 says I am v1:10.20.193.16:36276/0 (socket says 10.20.193.16:36276) 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a31cd7640 1 -- 10.20.193.16:0/413088017 learned_addr learned my addr 10.20.193.16:0/413088017 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a30cd5640 10 monclient(hunting): get_auth_request con 0x7f1a2c0c7fe0 auth_method 0 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a30cd5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a30cd5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a30cd5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a314d6640 10 monclient(hunting): get_auth_request con 0x7f1a2c0700f0 auth_method 0 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a314d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a314d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a314d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a30cd5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a30cd5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.013+0000 7f1a30cd5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a314d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a314d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:02.015 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a314d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:02.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a30cd5640 10 monclient(hunting): handle_auth_done global_id 5320 payload 274 2026-02-17T06:02:02.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a30cd5640 1 -- 10.20.193.16:0/413088017 >> v2:10.20.193.16:3300/0 conn(0x7f1a2c0700f0 msgr2=0x7f1a2c0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:02.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a30cd5640 1 --2- 10.20.193.16:0/413088017 >> v2:10.20.193.16:3300/0 conn(0x7f1a2c0700f0 0x7f1a2c0704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:02.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a30cd5640 1 -- 10.20.193.16:0/413088017 >> v1:10.20.193.148:6789/0 conn(0x7f1a2c1536f0 legacy=0x7f1a2c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:02.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a30cd5640 10 monclient: _finish_hunting 0 2026-02-17T06:02:02.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a30cd5640 1 monclient: found mon.noname-b 2026-02-17T06:02:02.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a30cd5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:02.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a30cd5640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:02.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.014+0000 7f1a30cd5640 1 -- 10.20.193.16:0/413088017 --> v2:10.20.193.128:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c0c4de0 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.016 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.015+0000 7f1a30cd5640 1 --2- 10.20.193.16:0/413088017 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 0x7f1a2c0c8430 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0c5b80 tx=0x7f1a2002ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b06d6522fa94a8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.016+0000 7f1a1bfff640 1 -- 10.20.193.16:0/413088017 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2003c070 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.016+0000 7f1a1bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.016+0000 7f1a1bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.016+0000 7f1a1bfff640 10 monclient: dump: 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.016+0000 7f1a1bfff640 10 monclient: _finish_auth 0 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.016+0000 7f1a1bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.016+0000 7f1a1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.016+0000 7f1a33761640 5 monclient: authenticate success, global_id 5320 2026-02-17T06:02:02.017 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.016+0000 7f1a33761640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a1bfff640 1 -- 10.20.193.16:0/413088017 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2002fa60 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a1bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a1bfff640 1 -- 10.20.193.16:0/413088017 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2002fe30 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a1bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a1bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a1bfff640 10 monclient: dump: 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:02.018 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 4 set_mon_vals no callback set 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 10 monclient: shutdown 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 1 -- 10.20.193.16:0/413088017 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 msgr2=0x7f1a2c0c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 1 --2- 10.20.193.16:0/413088017 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 0x7f1a2c0c8430 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0c5b80 tx=0x7f1a2002ed50 comp rx=0 tx=0).stop 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 1 -- 10.20.193.16:0/413088017 shutdown_connections 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 1 --2- 10.20.193.16:0/413088017 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 0x7f1a2c0c8430 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 1 --2- 10.20.193.16:0/413088017 >> v2:10.20.193.16:3300/0 conn(0x7f1a2c0700f0 0x7f1a2c0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 1 -- 10.20.193.16:0/413088017 >> 10.20.193.16:0/413088017 conn(0x7f1a2c0cd770 msgr2=0x7f1a2c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 1 -- 10.20.193.16:0/413088017 shutdown_connections 2026-02-17T06:02:02.019 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.017+0000 7f1a33761640 1 -- 10.20.193.16:0/413088017 wait complete. 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.018+0000 7f1a33761640 10 monclient: build_initial_monmap 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.018+0000 7f1a33761640 10 monclient: monmap: 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.018+0000 7f1a33761640 1 librados: starting msgr at 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.018+0000 7f1a33761640 1 librados: starting objecter 2026-02-17T06:02:02.021 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.018+0000 7f1a33761640 1 Processor -- start 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.018+0000 7f1a33761640 1 -- start start 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.018+0000 7f1a33761640 1 librados: setting wanted keys 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.018+0000 7f1a33761640 1 librados: calling monclient init 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.018+0000 7f1a33761640 10 monclient: init 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1a2c0700f0 0x7f1a2c1f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 10 monclient(hunting): picked mon.noname-c con 0x7f1a2c0700f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 0x7f1a2c13c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 10 monclient(hunting): picked mon.noname-b con 0x7f1a2c0c7fe0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1a2c1536f0 0x7f1a2c0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 10 monclient(hunting): picked mon.noname-a con 0x7f1a2c1536f0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c0c5680 con 0x7f1a2c1536f0 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c0c53a0 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c074c00 con 0x7f1a2c0700f0 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a33761640 10 monclient(hunting): authenticate will time out at 1374.588305s 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a31cd7640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1a2c1536f0 0x7f1a2c0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a30cd5640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 0x7f1a2c13c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:02.022 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a314d6640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1a2c0700f0 0x7f1a2c1f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a30cd5640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 0x7f1a2c13c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.128:3300/0 says I am v2:10.20.193.16:51562/0 (socket says 10.20.193.16:51562) 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a30cd5640 1 -- 10.20.193.16:0/3584110955 learned_addr learned my addr 10.20.193.16:0/3584110955 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a30cd5640 10 monclient(hunting): get_auth_request con 0x7f1a2c0c7fe0 auth_method 0 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a30cd5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a30cd5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a30cd5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a31cd7640 10 monclient(hunting): get_auth_request con 0x7f1a2c1536f0 auth_method 0 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a31cd7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a31cd7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a31cd7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a314d6640 10 monclient(hunting): get_auth_request con 0x7f1a2c0700f0 auth_method 0 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a314d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a314d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.019+0000 7f1a314d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.020+0000 7f1a30cd5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.020+0000 7f1a30cd5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.020+0000 7f1a30cd5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.020+0000 7f1a31cd7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.020+0000 7f1a31cd7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.020+0000 7f1a31cd7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.020+0000 7f1a314d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.020+0000 7f1a314d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:02.023 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.020+0000 7f1a314d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:02.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.021+0000 7f1a30cd5640 10 monclient(hunting): handle_auth_done global_id 5323 payload 824 2026-02-17T06:02:02.024 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.021+0000 7f1a30cd5640 1 -- 10.20.193.16:0/3584110955 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1a2c0700f0 msgr2=0x7f1a2c1f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.021+0000 7f1a30cd5640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1a2c0700f0 0x7f1a2c1f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.021+0000 7f1a30cd5640 1 -- 10.20.193.16:0/3584110955 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1a2c1536f0 msgr2=0x7f1a2c0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.021+0000 7f1a30cd5640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1a2c1536f0 0x7f1a2c0cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.021+0000 7f1a30cd5640 10 monclient: _finish_hunting 0 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.021+0000 7f1a30cd5640 1 monclient: found mon.noname-b 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.021+0000 7f1a30cd5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.021+0000 7f1a30cd5640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.021+0000 7f1a30cd5640 1 -- 10.20.193.16:0/3584110955 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c1f29a0 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.022+0000 7f1a30cd5640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 0x7f1a2c13c610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009130 tx=0x7f1a200047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.022+0000 7f1a19ffb640 10 client.?.objecter ms_handle_connect 0x7f1a2c0c7fe0 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.022+0000 7f1a19ffb640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a19ffb640 1 -- 10.20.193.16:0/3584110955 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2003c040 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a19ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a19ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a19ffb640 10 monclient: dump: 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:02.025 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a19ffb640 10 monclient: _finish_auth 0 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a19ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a19ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a33761640 5 monclient: authenticate success, global_id 5323 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a33761640 10 monclient: _renew_subs 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a33761640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a33761640 1 -- 10.20.193.16:0/3584110955 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2c1f26c0 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a33761640 10 client.5323.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a33761640 10 monclient: _renew_subs 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a33761640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a33761640 1 -- 10.20.193.16:0/3584110955 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2c073930 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.023+0000 7f1a33761640 1 librados: init done 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a19ffb640 1 -- 10.20.193.16:0/3584110955 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2002f9e0 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a19ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a19ffb640 1 -- 10.20.193.16:0/3584110955 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20042810 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a19ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a19ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a33761640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a19ffb640 10 monclient: dump: 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:02.026 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:02.027 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:02.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a33761640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:02.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a33761640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:02.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a1b7fe640 4 set_mon_vals no callback set 2026-02-17T06:02:02.027 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.024+0000 7f1a33761640 1 -- 10.20.193.16:0/3584110955 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a2c0704d0 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.131 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.131+0000 7f1a19ffb640 1 -- 10.20.193.16:0/3584110955 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f1a200429b0 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.131+0000 7f1a19ffb640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f19fc079170 0x7f19fc07b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:02.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.131+0000 7f1a314d6640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f19fc079170 0x7f19fc07b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:02.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.131+0000 7f1a314d6640 10 monclient: get_auth_request con 0x7f19fc079170 auth_method 0 2026-02-17T06:02:02.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.131+0000 7f1a314d6640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f19fc079170 0x7f19fc07b630 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009ba0 tx=0x7f1a1c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:02.132 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.131+0000 7f1a19ffb640 10 client.5323.objecter ms_handle_connect 0x7f19fc079170 2026-02-17T06:02:02.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.132+0000 7f1a19ffb640 1 -- 10.20.193.16:0/3584110955 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f1a20042cd0 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.132+0000 7f1a19ffb640 10 client.5323.objecter ms_dispatch2 0x7f1a2c000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:02.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.132+0000 7f1a19ffb640 3 client.5323.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:02.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.132+0000 7f1a19ffb640 3 client.5323.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:02.133 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.132+0000 7f1a19ffb640 20 client.5323.objecter dump_active .. 0 homeless 2026-02-17T06:02:02.138 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.139+0000 7f1a19ffb640 1 -- 10.20.193.16:0/3584110955 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1a20051080 con 0x7f1a2c0c7fe0 2026-02-17T06:02:02.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.139+0000 7f1a19ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:02.139 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.139+0000 7f1a19ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:02.221 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.220+0000 7f1a33761640 1 -- 10.20.193.16:0/3584110955 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a2c146c90 con 0x7f19fc079170 2026-02-17T06:02:02.223 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.221+0000 7f1a19ffb640 1 -- 10.20.193.16:0/3584110955 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44925 (secure 0 0 0) 0x7f1a2c146c90 con 0x7f19fc079170 2026-02-17T06:02:02.224 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:02:02.226 INFO:teuthology.orchestra.run.trial016.stderr:dumped all 2026-02-17T06:02:02.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 10 librados: watch_flush enter 2026-02-17T06:02:02.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 10 librados: watch_flush exit 2026-02-17T06:02:02.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 20 client.5323.objecter shutdown clearing up homeless session... 2026-02-17T06:02:02.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 10 client.5323.objecter successfully canceled tick 2026-02-17T06:02:02.226 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 -- 10.20.193.16:0/3584110955 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f19fc079170 msgr2=0x7f19fc07b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:02.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f19fc079170 0x7f19fc07b630 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009ba0 tx=0x7f1a1c009290 comp rx=0 tx=0).stop 2026-02-17T06:02:02.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 10 monclient: shutdown 2026-02-17T06:02:02.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:02.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 -- 10.20.193.16:0/3584110955 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 msgr2=0x7f1a2c13c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:02.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 0x7f1a2c13c610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009130 tx=0x7f1a200047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:02.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 -- 10.20.193.16:0/3584110955 shutdown_connections 2026-02-17T06:02:02.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f19fc079170 0x7f19fc07b630 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:02.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f1a2c0c7fe0 0x7f1a2c13c610 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:02.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f1a2c1536f0 0x7f1a2c0cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:02.227 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 --2- 10.20.193.16:0/3584110955 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f1a2c0700f0 0x7f1a2c1f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:02.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 -- 10.20.193.16:0/3584110955 >> 10.20.193.16:0/3584110955 conn(0x7f1a2c0cd770 msgr2=0x7f1a2c150420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:02.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 -- 10.20.193.16:0/3584110955 shutdown_connections 2026-02-17T06:02:02.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 -- 10.20.193.16:0/3584110955 wait complete. 2026-02-17T06:02:02.228 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:02.222+0000 7f1a33761640 1 librados: shutdown 2026-02-17T06:02:02.232 INFO:teuthology.orchestra.run.trial016.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2026-02-17T06:02:01.926901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437960,"kb_used_data":6552,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509511480,"statfs":{"total":1546188226560,"available":1545739755520,"internally_reserved":0,"allocated":6709248,"data_stored":3434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25505,"internal_metadata":441424991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":92,"apply_latency_ms":92,"commit_latency_ns":92000000,"apply_latency_ns":92000000},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001651"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:57.003241+0000","last_change":"2026-02-17T05:56:45.946554+0000","last_active":"2026-02-17T05:58:57.003241+0000","last_peered":"2026-02-17T05:58:57.003241+0000","last_clean":"2026-02-17T05:58:57.003241+0000","last_became_active":"2026-02-17T05:56:43.852478+0000","last_became_peered":"2026-02-17T05:56:43.852478+0000","last_unstale":"2026-02-17T05:58:57.003241+0000","last_undegraded":"2026-02-17T05:58:57.003241+0000","last_fullsized":"2026-02-17T05:58:57.003241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T08:21:55.695169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031385049999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:54.575851+0000","last_change":"2026-02-17T05:56:46.700244+0000","last_active":"2026-02-17T05:58:54.575851+0000","last_peered":"2026-02-17T05:58:54.575851+0000","last_clean":"2026-02-17T05:58:54.575851+0000","last_became_active":"2026-02-17T05:56:43.182525+0000","last_became_peered":"2026-02-17T05:56:43.182525+0000","last_unstale":"2026-02-17T05:58:54.575851+0000","last_undegraded":"2026-02-17T05:58:54.575851+0000","last_fullsized":"2026-02-17T05:58:54.575851+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:38:44.190148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038112767999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:47.494561+0000","last_change":"2026-02-17T05:56:45.501489+0000","last_active":"2026-02-17T05:58:47.494561+0000","last_peered":"2026-02-17T05:58:47.494561+0000","last_clean":"2026-02-17T05:58:47.494561+0000","last_became_active":"2026-02-17T05:56:43.381684+0000","last_became_peered":"2026-02-17T05:56:43.381684+0000","last_unstale":"2026-02-17T05:58:47.494561+0000","last_undegraded":"2026-02-17T05:58:47.494561+0000","last_fullsized":"2026-02-17T05:58:47.494561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:07:04.503400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031884497999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:02:00.560733+0000","last_change":"2026-02-17T06:02:00.565400+0000","last_active":"2026-02-17T06:02:00.560733+0000","last_peered":"2026-02-17T06:02:00.560733+0000","last_clean":"2026-02-17T06:02:00.560733+0000","last_became_active":"2026-02-17T05:56:43.290452+0000","last_became_peered":"2026-02-17T05:56:43.290452+0000","last_unstale":"2026-02-17T06:02:00.560733+0000","last_undegraded":"2026-02-17T06:02:00.560733+0000","last_fullsized":"2026-02-17T06:02:00.560733+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:02:00.557507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:02:00.557507+0000","last_clean_scrub_stamp":"2026-02-17T06:02:00.557507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T06:23:42.693877+0000","scrub_duration":193,"objects_trimmed":0,"snaptrim_duration":0.032515719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"33'2","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:47.312723+0000","last_change":"2026-02-17T05:56:45.527362+0000","last_active":"2026-02-17T05:58:47.312723+0000","last_peered":"2026-02-17T05:58:47.312723+0000","last_clean":"2026-02-17T05:58:47.312723+0000","last_became_active":"2026-02-17T05:56:43.262979+0000","last_became_peered":"2026-02-17T05:56:43.262979+0000","last_unstale":"2026-02-17T05:58:47.312723+0000","last_undegraded":"2026-02-17T05:58:47.312723+0000","last_fullsized":"2026-02-17T05:58:47.312723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:30:08.337496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.056715723000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:01:57.602340+0000","last_change":"2026-02-17T06:01:57.602783+0000","last_active":"2026-02-17T06:01:57.602340+0000","last_peered":"2026-02-17T06:01:57.602340+0000","last_clean":"2026-02-17T06:01:57.602340+0000","last_became_active":"2026-02-17T05:56:43.075477+0000","last_became_peered":"2026-02-17T05:56:43.075477+0000","last_unstale":"2026-02-17T06:01:57.602340+0000","last_undegraded":"2026-02-17T06:01:57.602340+0000","last_fullsized":"2026-02-17T06:01:57.602340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:01:57.600454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:01:57.600454+0000","last_clean_scrub_stamp":"2026-02-17T06:01:57.600454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T17:47:43.923169+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0.030611670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T05:58:42.310803+0000","last_change":"2026-02-17T05:56:45.522106+0000","last_active":"2026-02-17T05:58:42.310803+0000","last_peered":"2026-02-17T05:58:42.310803+0000","last_clean":"2026-02-17T05:58:42.310803+0000","last_became_active":"2026-02-17T05:56:43.142816+0000","last_became_peered":"2026-02-17T05:56:43.142816+0000","last_unstale":"2026-02-17T05:58:42.310803+0000","last_undegraded":"2026-02-17T05:58:42.310803+0000","last_fullsized":"2026-02-17T05:58:42.310803+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:56:41.737722+0000","last_clean_scrub_stamp":"2026-02-17T05:56:41.737722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:47:19.171322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.043200879999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"34'2","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:01:56.550777+0000","last_change":"2026-02-17T06:01:56.555968+0000","last_active":"2026-02-17T06:01:56.550777+0000","last_peered":"2026-02-17T06:01:56.550777+0000","last_clean":"2026-02-17T06:01:56.550777+0000","last_became_active":"2026-02-17T05:56:43.121999+0000","last_became_peered":"2026-02-17T05:56:43.121999+0000","last_unstale":"2026-02-17T06:01:56.550777+0000","last_undegraded":"2026-02-17T06:01:56.550777+0000","last_fullsized":"2026-02-17T06:01:56.550777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2","last_scrub_stamp":"2026-02-17T06:01:56.547861+0000","last_deep_scrub":"34'2","last_deep_scrub_stamp":"2026-02-17T06:01:56.547861+0000","last_clean_scrub_stamp":"2026-02-17T06:01:56.547861+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T17:32:11.549214+0000","scrub_duration":267,"objects_trimmed":0,"snaptrim_duration":0.034338280999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"34'34","reported_seq":84,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:01:56.862743+0000","last_change":"2026-02-17T06:01:56.865813+0000","last_active":"2026-02-17T06:01:56.862743+0000","last_peered":"2026-02-17T06:01:56.862743+0000","last_clean":"2026-02-17T06:01:56.862743+0000","last_became_active":"2026-02-17T05:56:37.919763+0000","last_became_peered":"2026-02-17T05:56:37.919763+0000","last_unstale":"2026-02-17T06:01:56.862743+0000","last_undegraded":"2026-02-17T06:01:56.862743+0000","last_fullsized":"2026-02-17T06:01:56.862743+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'34","last_scrub_stamp":"2026-02-17T06:01:56.859738+0000","last_deep_scrub":"34'34","last_deep_scrub_stamp":"2026-02-17T06:01:56.859738+0000","last_clean_scrub_stamp":"2026-02-17T06:01:56.859738+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:50:15.008010+0000","scrub_duration":278,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":28,"seq":120259084359,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":28,"seq":120259084358,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":27,"seq":115964117061,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,3,4,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313286,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280581,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313285,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,4,5,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313286,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":372736,"data_stored":164912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[3,4,5,6,7,8,9,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":23,"seq":98784247876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":25,"seq":107374182472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":23,"seq":98784247875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":372736,"data_stored":164912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[0,1,2,4,5,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":23,"seq":98784247875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":23,"seq":98784247876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,6,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":25,"seq":107374182468,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,5,6,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":25,"seq":107374182468,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":770048,"data_stored":562733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,4,5,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":27,"seq":115964117065,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":770048,"data_stored":562733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":27,"seq":115964117065,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[1,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:02:02.240 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-17T05:56:41.737722+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=56, tm_sec=41, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=1, tm_sec=54, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:02:02.241 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-17T05:56:41.737722+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=56, tm_sec=41, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=1, tm_sec=54, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:02:02.241 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-17T05:56:41.737722+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=56, tm_sec=41, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=1, tm_sec=54, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:02:02.241 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-17T05:56:41.737722+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=56, tm_sec=41, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=1, tm_sec=54, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:02:02.241 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-17T05:56:41.737722+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=56, tm_sec=41, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=1, tm_sec=54, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:02:02.241 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-17T06:02:09.306 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:07:48.133 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:09.307 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:07:48.134 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:09.318 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:07:48.146 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:09.332 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:07:48.158 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:09.333 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:07:48.159 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:09.336 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:07:48.125 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:09.337 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:07:48.126 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:09.345 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:07:48.171 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:09.350 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:07:48.139 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:22.242 DEBUG:teuthology.orchestra.run.trial016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:02:22.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.313+0000 7fa617087640 10 monclient: get_monmap_and_config 2026-02-17T06:02:22.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.313+0000 7fa617087640 10 monclient: build_initial_monmap 2026-02-17T06:02:22.314 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.313+0000 7fa617087640 10 monclient: monmap: 2026-02-17T06:02:22.314 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:22.314609+0000 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:22.314609+0000 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:22.315 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:22.316 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 1 Processor -- start 2026-02-17T06:02:22.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 1 -- start start 2026-02-17T06:02:22.316 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 10 monclient: init 2026-02-17T06:02:22.318 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 10 monclient: _add_conns ranks=[1,3,2,5,4,0] 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fa61014f020 addr v1:10.20.193.128:6789/0 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fa61014f400 0x7fa61015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 10 monclient(hunting): picked mon.noname-a con 0x7fa61014f400 addr v2:10.20.193.16:3300/0 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa61015c920 addr v1:10.20.193.148:6789/0 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6100c5b10 con 0x7fa61014f020 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 1 -- --> v1:10.20.193.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa61015cd20 con 0x7fa61015c920 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa61006e620 con 0x7fa61014f400 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:22.319 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.314+0000 7fa617087640 10 monclient(hunting): authenticate will time out at 1394.883625s 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa607fff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7fa61014f400 0x7fa61015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa614dfc640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7fa61014f020 0x7fa61006dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:51286/0 (socket says 10.20.193.16:51286) 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa614dfc640 1 -- 10.20.193.16:0/1649523096 learned_addr learned my addr 10.20.193.16:0/1649523096 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa607fff640 10 monclient(hunting): get_auth_request con 0x7fa61014f400 auth_method 0 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa607fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa607fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa607fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa6155fd640 1 -- 10.20.193.16:0/1649523096 >> v1:10.20.193.148:6789/0 conn(0x7fa61015c920 legacy=0x7fa6100c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa6155fd640 1 -- 10.20.193.16:0/1649523096 >> v1:10.20.193.148:6789/0 conn(0x7fa61015c920 legacy=0x7fa6100c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa6155fd640 1 --1- 10.20.193.16:0/1649523096 >> v1:10.20.193.148:6789/0 conn(0x7fa61015c920 0x7fa6100c6010 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa607fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:22.320 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa607fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:22.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.315+0000 7fa607fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:22.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.316+0000 7fa607fff640 10 monclient(hunting): handle_auth_done global_id 5232 payload 274 2026-02-17T06:02:22.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.316+0000 7fa607fff640 1 -- 10.20.193.16:0/1649523096 >> v1:10.20.193.148:6789/0 conn(0x7fa61015c920 legacy=0x7fa6100c6010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:22.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.316+0000 7fa607fff640 1 -- 10.20.193.16:0/1649523096 >> v1:10.20.193.128:6789/0 conn(0x7fa61014f020 legacy=0x7fa61006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.316+0000 7fa607fff640 10 monclient: _finish_hunting 0 2026-02-17T06:02:22.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.316+0000 7fa607fff640 1 monclient: found mon.noname-a 2026-02-17T06:02:22.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.316+0000 7fa607fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:22.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.316+0000 7fa607fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:02:22.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.316+0000 7fa607fff640 1 -- 10.20.193.16:0/1649523096 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61006e900 con 0x7fa61014f400 2026-02-17T06:02:22.321 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.317+0000 7fa607fff640 1 --2- 10.20.193.16:0/1649523096 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f400 0x7fa61015c3e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80098a0 tx=0x7fa5f802eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27e5dbc0281b4bf server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:22.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 1 -- 10.20.193.16:0/1649523096 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f803c070 con 0x7fa61014f400 2026-02-17T06:02:22.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:22.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:22.328 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 10 monclient: dump: 2026-02-17T06:02:22.328 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:22.329 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:22.329 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.329 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.329 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:22.329 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.329 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:22.329 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:22.329 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:22.329 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.329 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 10 monclient: _finish_auth 0 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 10 monclient: _check_auth_tickets 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 1 -- 10.20.193.16:0/1649523096 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f802fb70 con 0x7fa61014f400 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 5 monclient: authenticate success, global_id 5232 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 1 -- 10.20.193.16:0/1649523096 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8037440 con 0x7fa61014f400 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 4 set_mon_vals no callback set 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:22.330 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa6077fe640 10 monclient: dump: 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 10 monclient: shutdown 2026-02-17T06:02:22.331 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 1 -- 10.20.193.16:0/1649523096 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f400 msgr2=0x7fa61015c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 1 --2- 10.20.193.16:0/1649523096 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f400 0x7fa61015c3e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80098a0 tx=0x7fa5f802eeb0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 1 -- 10.20.193.16:0/1649523096 shutdown_connections 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 1 --2- 10.20.193.16:0/1649523096 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f400 0x7fa61015c3e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 1 -- 10.20.193.16:0/1649523096 >> 10.20.193.16:0/1649523096 conn(0x7fa6100cd770 msgr2=0x7fa6100cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 1 -- 10.20.193.16:0/1649523096 shutdown_connections 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.319+0000 7fa617087640 1 -- 10.20.193.16:0/1649523096 wait complete. 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.320+0000 7fa617087640 10 monclient: build_initial_monmap 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.320+0000 7fa617087640 10 monclient: monmap: 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:22.332 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.320+0000 7fa617087640 1 librados: starting msgr at 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.320+0000 7fa617087640 1 librados: starting objecter 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.320+0000 7fa617087640 1 Processor -- start 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.320+0000 7fa617087640 1 -- start start 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.320+0000 7fa617087640 1 librados: setting wanted keys 2026-02-17T06:02:22.333 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.320+0000 7fa617087640 1 librados: calling monclient init 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.320+0000 7fa617087640 10 monclient: init 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.320+0000 7fa617087640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f020 0x7fa61014ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 10 monclient(hunting): picked mon.noname-a con 0x7fa61014f020 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa61014f400 0x7fa610073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 10 monclient(hunting): picked mon.noname-c con 0x7fa61014f400 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa61015c920 0x7fa610072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 10 monclient(hunting): picked mon.noname-b con 0x7fa61015c920 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa61006f1a0 con 0x7fa61014f020 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:22.334 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7fa61006eec0 con 0x7fa61015c920 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa61006e2f0 con 0x7fa61014f400 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa617087640 10 monclient(hunting): authenticate will time out at 1394.890048s 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa614dfc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f020 0x7fa61014ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa607fff640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa61014f400 0x7fa610073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa614dfc640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f020 0x7fa61014ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59412/0 (socket says 10.20.193.16:59412) 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa614dfc640 1 -- 10.20.193.16:0/1857950246 learned_addr learned my addr 10.20.193.16:0/1857950246 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa614dfc640 10 monclient(hunting): get_auth_request con 0x7fa61014f020 auth_method 0 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa614dfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:22.335 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa614dfc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa614dfc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa6155fd640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa61015c920 0x7fa610072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa614dfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa614dfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa614dfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa607fff640 10 monclient(hunting): get_auth_request con 0x7fa61014f400 auth_method 0 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa607fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa607fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa607fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa6155fd640 10 monclient(hunting): get_auth_request con 0x7fa61015c920 auth_method 0 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa6155fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:22.336 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa6155fd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.321+0000 7fa6155fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.322+0000 7fa607fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.322+0000 7fa607fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.322+0000 7fa607fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.322+0000 7fa6155fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.322+0000 7fa6155fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.322+0000 7fa6155fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.322+0000 7fa614dfc640 10 monclient(hunting): handle_auth_done global_id 5235 payload 824 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.322+0000 7fa614dfc640 1 -- 10.20.193.16:0/1857950246 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa61014f400 msgr2=0x7fa610073960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa614dfc640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa61014f400 0x7fa610073960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa614dfc640 1 -- 10.20.193.16:0/1857950246 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa61015c920 msgr2=0x7fa610072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa614dfc640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa61015c920 0x7fa610072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.337 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa614dfc640 10 monclient: _finish_hunting 0 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa614dfc640 1 monclient: found mon.noname-a 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa614dfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa614dfc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa614dfc640 1 -- 10.20.193.16:0/1857950246 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6101f8200 con 0x7fa61014f020 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa614dfc640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f020 0x7fa61014ac90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c700 tx=0x7fa60000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa6057fa640 10 client.?.objecter ms_handle_connect 0x7fa61014f020 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.323+0000 7fa6057fa640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa6057fa640 1 -- 10.20.193.16:0/1857950246 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600017070 con 0x7fa61014f020 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa6057fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa6057fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa6057fa640 10 monclient: dump: 2026-02-17T06:02:22.338 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa6057fa640 10 monclient: _finish_auth 0 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa6057fa640 10 monclient: _check_auth_tickets 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa6057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:22.339 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa617087640 5 monclient: authenticate success, global_id 5235 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa617087640 10 monclient: _renew_subs 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa617087640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa617087640 1 -- 10.20.193.16:0/1857950246 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6101f4110 con 0x7fa61014f020 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa617087640 10 client.5235.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa617087640 10 monclient: _renew_subs 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa617087640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa617087640 1 -- 10.20.193.16:0/1857950246 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6101f46d0 con 0x7fa61014f020 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.324+0000 7fa617087640 1 librados: init done 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa617087640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa617087640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa617087640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.16:3300/0 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa617087640 1 -- 10.20.193.16:0/1857950246 --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61015c3e0 con 0x7fa61014f020 2026-02-17T06:02:22.340 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa6057fa640 1 -- 10.20.193.16:0/1857950246 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa600005bb0 con 0x7fa61014f020 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa6057fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa6057fa640 1 -- 10.20.193.16:0/1857950246 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600007070 con 0x7fa61014f020 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa6057fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa6057fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa606ffd640 4 set_mon_vals no callback set 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.325+0000 7fa6057fa640 10 monclient: dump: 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:22.341 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.342 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:22.342 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:22.342 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:22.342 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.435 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.435+0000 7fa6057fa640 1 -- 10.20.193.16:0/1857950246 <== mon.0 v2:10.20.193.16:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7fa600007210 con 0x7fa61014f020 2026-02-17T06:02:22.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.435+0000 7fa6057fa640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa5e0079140 0x7fa5e007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.435+0000 7fa607fff640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa5e0079140 0x7fa5e007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.435+0000 7fa607fff640 10 monclient: get_auth_request con 0x7fa5e0079140 auth_method 0 2026-02-17T06:02:22.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.436+0000 7fa6057fa640 10 client.5235.objecter ms_handle_connect 0x7fa5e0079140 2026-02-17T06:02:22.436 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.436+0000 7fa607fff640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa5e0079140 0x7fa5e007b600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8039000 tx=0x7fa5f8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:22.445 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.445+0000 7fa6057fa640 1 -- 10.20.193.16:0/1857950246 <== mon.0 v2:10.20.193.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7fa60005d7c0 con 0x7fa61014f020 2026-02-17T06:02:22.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.445+0000 7fa6057fa640 10 client.5235.objecter ms_dispatch2 0x7fa610000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:22.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.445+0000 7fa6057fa640 3 client.5235.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:22.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.445+0000 7fa6057fa640 3 client.5235.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:22.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.445+0000 7fa6057fa640 20 client.5235.objecter dump_active .. 0 homeless 2026-02-17T06:02:22.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.445+0000 7fa6057fa640 1 -- 10.20.193.16:0/1857950246 <== mon.0 v2:10.20.193.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa600021020 con 0x7fa61014f020 2026-02-17T06:02:22.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.445+0000 7fa6057fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:22.446 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.445+0000 7fa6057fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:22.528 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.528+0000 7fa617087640 1 -- 10.20.193.16:0/1857950246 --> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa61014e7d0 con 0x7fa5e0079140 2026-02-17T06:02:22.529 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.529+0000 7fa6057fa640 1 -- 10.20.193.16:0/1857950246 <== mgr.4104 v2:10.20.193.16:6800/3354619179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44825 (secure 0 0 0) 0x7fa61014e7d0 con 0x7fa5e0079140 2026-02-17T06:02:22.530 INFO:teuthology.orchestra.run.trial016.stdout: 2026-02-17T06:02:22.532 INFO:teuthology.orchestra.run.trial016.stderr:dumped all 2026-02-17T06:02:22.532 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.529+0000 7fa617087640 10 librados: watch_flush enter 2026-02-17T06:02:22.532 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 10 librados: watch_flush exit 2026-02-17T06:02:22.532 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 20 client.5235.objecter shutdown clearing up homeless session... 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 10 client.5235.objecter successfully canceled tick 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 -- 10.20.193.16:0/1857950246 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa5e0079140 msgr2=0x7fa5e007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa5e0079140 0x7fa5e007b600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8039000 tx=0x7fa5f8033000 comp rx=0 tx=0).stop 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 10 monclient: shutdown 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 -- 10.20.193.16:0/1857950246 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f020 msgr2=0x7fa61014ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f020 0x7fa61014ac90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c700 tx=0x7fa60000cbd0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 -- 10.20.193.16:0/1857950246 shutdown_connections 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7fa5e0079140 0x7fa5e007b600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7fa61015c920 0x7fa610072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.533 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7fa61014f400 0x7fa610073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 --2- 10.20.193.16:0/1857950246 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7fa61014f020 0x7fa61014ac90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 -- 10.20.193.16:0/1857950246 >> 10.20.193.16:0/1857950246 conn(0x7fa6100cd770 msgr2=0x7fa6100c8580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:22.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 -- 10.20.193.16:0/1857950246 shutdown_connections 2026-02-17T06:02:22.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 -- 10.20.193.16:0/1857950246 wait complete. 2026-02-17T06:02:22.534 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.530+0000 7fa617087640 1 librados: shutdown 2026-02-17T06:02:22.539 INFO:teuthology.orchestra.run.trial016.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2026-02-17T06:02:21.929650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437940,"kb_used_data":6552,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509511500,"statfs":{"total":1546188226560,"available":1545739776000,"internally_reserved":0,"allocated":6709248,"data_stored":3434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25505,"internal_metadata":441424991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001611"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:02:02.688157+0000","last_change":"2026-02-17T06:02:02.690846+0000","last_active":"2026-02-17T06:02:02.688157+0000","last_peered":"2026-02-17T06:02:02.688157+0000","last_clean":"2026-02-17T06:02:02.688157+0000","last_became_active":"2026-02-17T05:56:43.852478+0000","last_became_peered":"2026-02-17T05:56:43.852478+0000","last_unstale":"2026-02-17T06:02:02.688157+0000","last_undegraded":"2026-02-17T06:02:02.688157+0000","last_fullsized":"2026-02-17T06:02:02.688157+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:02:02.685931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:02:02.685931+0000","last_clean_scrub_stamp":"2026-02-17T06:02:02.685931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T07:48:01.473773+0000","scrub_duration":169,"objects_trimmed":0,"snaptrim_duration":0.031385049999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:02:00.559774+0000","last_change":"2026-02-17T06:02:00.564420+0000","last_active":"2026-02-17T06:02:00.559774+0000","last_peered":"2026-02-17T06:02:00.559774+0000","last_clean":"2026-02-17T06:02:00.559774+0000","last_became_active":"2026-02-17T05:56:43.182525+0000","last_became_peered":"2026-02-17T05:56:43.182525+0000","last_unstale":"2026-02-17T06:02:00.559774+0000","last_undegraded":"2026-02-17T06:02:00.559774+0000","last_fullsized":"2026-02-17T06:02:00.559774+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:02:00.556195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:02:00.556195+0000","last_clean_scrub_stamp":"2026-02-17T06:02:00.556195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:55:53.463390+0000","scrub_duration":201,"objects_trimmed":0,"snaptrim_duration":0.038112767999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:01:59.059647+0000","last_change":"2026-02-17T06:01:59.064251+0000","last_active":"2026-02-17T06:01:59.059647+0000","last_peered":"2026-02-17T06:01:59.059647+0000","last_clean":"2026-02-17T06:01:59.059647+0000","last_became_active":"2026-02-17T05:56:43.381684+0000","last_became_peered":"2026-02-17T05:56:43.381684+0000","last_unstale":"2026-02-17T06:01:59.059647+0000","last_undegraded":"2026-02-17T06:01:59.059647+0000","last_fullsized":"2026-02-17T06:01:59.059647+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:01:59.056364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:01:59.056364+0000","last_clean_scrub_stamp":"2026-02-17T06:01:59.056364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:27:32.999891+0000","scrub_duration":190,"objects_trimmed":0,"snaptrim_duration":0.031884497999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:02:00.560733+0000","last_change":"2026-02-17T06:02:00.565400+0000","last_active":"2026-02-17T06:02:00.560733+0000","last_peered":"2026-02-17T06:02:00.560733+0000","last_clean":"2026-02-17T06:02:00.560733+0000","last_became_active":"2026-02-17T05:56:43.290452+0000","last_became_peered":"2026-02-17T05:56:43.290452+0000","last_unstale":"2026-02-17T06:02:00.560733+0000","last_undegraded":"2026-02-17T06:02:00.560733+0000","last_fullsized":"2026-02-17T06:02:00.560733+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:02:00.557507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:02:00.557507+0000","last_clean_scrub_stamp":"2026-02-17T06:02:00.557507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T06:23:42.693877+0000","scrub_duration":193,"objects_trimmed":0,"snaptrim_duration":0.032515719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"34'3","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:02:02.132620+0000","last_change":"2026-02-17T06:02:02.133471+0000","last_active":"2026-02-17T06:02:02.132620+0000","last_peered":"2026-02-17T06:02:02.132620+0000","last_clean":"2026-02-17T06:02:02.132620+0000","last_became_active":"2026-02-17T05:56:43.262979+0000","last_became_peered":"2026-02-17T05:56:43.262979+0000","last_unstale":"2026-02-17T06:02:02.132620+0000","last_undegraded":"2026-02-17T06:02:02.132620+0000","last_fullsized":"2026-02-17T06:02:02.132620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3","last_scrub_stamp":"2026-02-17T06:02:02.131882+0000","last_deep_scrub":"34'3","last_deep_scrub_stamp":"2026-02-17T06:02:02.131882+0000","last_clean_scrub_stamp":"2026-02-17T06:02:02.131882+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:24:20.356249+0000","scrub_duration":156,"objects_trimmed":0,"snaptrim_duration":0.056715723000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:01:57.602340+0000","last_change":"2026-02-17T06:01:57.602783+0000","last_active":"2026-02-17T06:01:57.602340+0000","last_peered":"2026-02-17T06:01:57.602340+0000","last_clean":"2026-02-17T06:01:57.602340+0000","last_became_active":"2026-02-17T05:56:43.075477+0000","last_became_peered":"2026-02-17T05:56:43.075477+0000","last_unstale":"2026-02-17T06:01:57.602340+0000","last_undegraded":"2026-02-17T06:01:57.602340+0000","last_fullsized":"2026-02-17T06:01:57.602340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:01:57.600454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:01:57.600454+0000","last_clean_scrub_stamp":"2026-02-17T06:01:57.600454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T17:47:43.923169+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0.030611670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:02:01.096284+0000","last_change":"2026-02-17T06:02:01.101025+0000","last_active":"2026-02-17T06:02:01.096284+0000","last_peered":"2026-02-17T06:02:01.096284+0000","last_clean":"2026-02-17T06:02:01.096284+0000","last_became_active":"2026-02-17T05:56:43.142816+0000","last_became_peered":"2026-02-17T05:56:43.142816+0000","last_unstale":"2026-02-17T06:02:01.096284+0000","last_undegraded":"2026-02-17T06:02:01.096284+0000","last_fullsized":"2026-02-17T06:02:01.096284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:02:01.093830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:02:01.093830+0000","last_clean_scrub_stamp":"2026-02-17T06:02:01.093830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T08:12:42.424748+0000","scrub_duration":116,"objects_trimmed":0,"snaptrim_duration":0.043200879999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"34'2","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:01:56.550777+0000","last_change":"2026-02-17T06:01:56.555968+0000","last_active":"2026-02-17T06:01:56.550777+0000","last_peered":"2026-02-17T06:01:56.550777+0000","last_clean":"2026-02-17T06:01:56.550777+0000","last_became_active":"2026-02-17T05:56:43.121999+0000","last_became_peered":"2026-02-17T05:56:43.121999+0000","last_unstale":"2026-02-17T06:01:56.550777+0000","last_undegraded":"2026-02-17T06:01:56.550777+0000","last_fullsized":"2026-02-17T06:01:56.550777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2","last_scrub_stamp":"2026-02-17T06:01:56.547861+0000","last_deep_scrub":"34'2","last_deep_scrub_stamp":"2026-02-17T06:01:56.547861+0000","last_clean_scrub_stamp":"2026-02-17T06:01:56.547861+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T17:32:11.549214+0000","scrub_duration":267,"objects_trimmed":0,"snaptrim_duration":0.034338280999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"34'34","reported_seq":84,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-17T06:01:56.862743+0000","last_change":"2026-02-17T06:01:56.865813+0000","last_active":"2026-02-17T06:01:56.862743+0000","last_peered":"2026-02-17T06:01:56.862743+0000","last_clean":"2026-02-17T06:01:56.862743+0000","last_became_active":"2026-02-17T05:56:37.919763+0000","last_became_peered":"2026-02-17T05:56:37.919763+0000","last_unstale":"2026-02-17T06:01:56.862743+0000","last_undegraded":"2026-02-17T06:01:56.862743+0000","last_fullsized":"2026-02-17T06:01:56.862743+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'34","last_scrub_stamp":"2026-02-17T06:01:56.859738+0000","last_deep_scrub":"34'34","last_deep_scrub_stamp":"2026-02-17T06:01:56.859738+0000","last_clean_scrub_stamp":"2026-02-17T06:01:56.859738+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:50:15.008010+0000","scrub_duration":278,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":28,"seq":120259084363,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":28,"seq":120259084362,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":27,"seq":115964117064,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,3,4,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313291,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280585,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313289,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,4,5,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313290,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":372736,"data_stored":164912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[3,4,5,6,7,8,9,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":23,"seq":98784247880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":25,"seq":107374182476,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":23,"seq":98784247879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":372736,"data_stored":164912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[0,1,2,4,5,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":23,"seq":98784247879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":23,"seq":98784247880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,6,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":25,"seq":107374182472,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,5,6,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":25,"seq":107374182472,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":770048,"data_stored":562733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,4,5,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":27,"seq":115964117069,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":770048,"data_stored":562733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":27,"seq":115964117069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":368640,"data_stored":164893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[1,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:02:22.542 DEBUG:teuthology.orchestra.run.trial016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-17T06:02:22.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.610+0000 7f61f656a640 10 monclient: get_monmap_and_config 2026-02-17T06:02:22.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.610+0000 7f61f656a640 10 monclient: build_initial_monmap 2026-02-17T06:02:22.611 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.610+0000 7f61f656a640 10 monclient: monmap: 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T06:02:22.611481+0000 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T06:02:22.611481+0000 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:0: v1:10.20.193.16:6789/0 mon.noname-a-legacy 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:1: v1:10.20.193.128:6789/0 mon.noname-b-legacy 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:2: v1:10.20.193.148:6789/0 mon.noname-c-legacy 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:3: v2:10.20.193.16:3300/0 mon.noname-a 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:4: v2:10.20.193.128:3300/0 mon.noname-b 2026-02-17T06:02:22.612 INFO:teuthology.orchestra.run.trial016.stderr:5: v2:10.20.193.148:3300/0 mon.noname-c 2026-02-17T06:02:22.613 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.610+0000 7f61f656a640 1 Processor -- start 2026-02-17T06:02:22.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.610+0000 7f61f656a640 1 -- start start 2026-02-17T06:02:22.613 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.610+0000 7f61f656a640 10 monclient: init 2026-02-17T06:02:22.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:22.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 10 monclient: _add_conns ranks=[3,1,5,4,0,2] 2026-02-17T06:02:22.615 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f61f00700f0 0x7f61f00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 10 monclient(hunting): picked mon.noname-a con 0x7f61f00700f0 addr v2:10.20.193.16:3300/0 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f61f00c7fe0 addr v1:10.20.193.128:6789/0 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f61f01536f0 0x7f61f0153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 10 monclient(hunting): picked mon.noname-c con 0x7f61f01536f0 addr v2:10.20.193.148:3300/0 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 1 -- --> v1:10.20.193.128:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f00c4820 con 0x7f61f00c7fe0 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 1 -- --> v2:10.20.193.16:3300/0 -- mon_getmap magic: 0 -- 0x7f61f00c4b00 con 0x7f61f00700f0 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 1 -- --> v2:10.20.193.148:3300/0 -- mon_getmap magic: 0 -- 0x7f61f00c4de0 con 0x7f61f01536f0 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f656a640 10 monclient(hunting): authenticate will time out at 1395.180469s 2026-02-17T06:02:22.616 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61effff640 1 --2- >> v2:10.20.193.16:3300/0 conn(0x7f61f00700f0 0x7f61f00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61f4ae0640 1 --2- >> v2:10.20.193.148:3300/0 conn(0x7f61f01536f0 0x7f61f0153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61ef7fe640 1 --1- >> v1:10.20.193.128:6789/0 conn(0x7f61f00c7fe0 0x7f61f00c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.128:6789/0 says I am v1:10.20.193.16:51294/0 (socket says 10.20.193.16:51294) 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.611+0000 7f61ef7fe640 1 -- 10.20.193.16:0/36605491 learned_addr learned my addr 10.20.193.16:0/36605491 (peer_addr_for_me v1:10.20.193.16:0/0) 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61effff640 10 monclient(hunting): get_auth_request con 0x7f61f00700f0 auth_method 0 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61f4ae0640 10 monclient(hunting): get_auth_request con 0x7f61f01536f0 auth_method 0 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61f4ae0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61f4ae0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61f4ae0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:22.617 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61f4ae0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61f4ae0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.612+0000 7f61f4ae0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61effff640 10 monclient(hunting): handle_auth_done global_id 5238 payload 274 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61effff640 1 -- 10.20.193.16:0/36605491 >> v2:10.20.193.148:3300/0 conn(0x7f61f01536f0 msgr2=0x7f61f0153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61effff640 1 --2- 10.20.193.16:0/36605491 >> v2:10.20.193.148:3300/0 conn(0x7f61f01536f0 0x7f61f0153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61effff640 1 -- 10.20.193.16:0/36605491 >> v1:10.20.193.128:6789/0 conn(0x7f61f00c7fe0 legacy=0x7f61f00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61eeffd640 1 -- 10.20.193.16:0/36605491 <== mon.1 v1:10.20.193.128:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1466347373 0 0) 0x7f61f00c4820 con 0x7f61f00c7fe0 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61effff640 10 monclient: _finish_hunting 0 2026-02-17T06:02:22.618 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61effff640 1 monclient: found mon.noname-a 2026-02-17T06:02:22.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:22.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61effff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.16:3300/0 2026-02-17T06:02:22.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61effff640 1 -- 10.20.193.16:0/36605491 --> v2:10.20.193.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f00c50c0 con 0x7f61f00700f0 2026-02-17T06:02:22.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.613+0000 7f61eeffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T06:02:22.619 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.614+0000 7f61effff640 1 --2- 10.20.193.16:0/36605491 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f61f00700f0 0x7f61f00704d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f61f00c9540 tx=0x7f61dc02ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1663cf298f27a4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:22.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 1 -- 10.20.193.16:0/36605491 <== mon.0 v2:10.20.193.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc03c070 con 0x7f61f00700f0 2026-02-17T06:02:22.625 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 10 monclient: dump: 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:22.626 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 10 monclient: _finish_auth 0 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 10 monclient: _check_auth_tickets 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 1 -- 10.20.193.16:0/36605491 <== mon.0 v2:10.20.193.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc02fa60 con 0x7f61f00700f0 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 5 monclient: authenticate success, global_id 5238 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 1 -- 10.20.193.16:0/36605491 <== mon.0 v2:10.20.193.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc02fe30 con 0x7f61f00700f0 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:22.627 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61eeffd640 10 monclient: dump: 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:22.628 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 4 set_mon_vals no callback set 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 10 monclient: shutdown 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 1 -- 10.20.193.16:0/36605491 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f61f00700f0 msgr2=0x7f61f00704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 1 --2- 10.20.193.16:0/36605491 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f61f00700f0 0x7f61f00704d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f61f00c9540 tx=0x7f61dc02ed50 comp rx=0 tx=0).stop 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 1 -- 10.20.193.16:0/36605491 shutdown_connections 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 1 --2- 10.20.193.16:0/36605491 >> v2:10.20.193.148:3300/0 conn(0x7f61f01536f0 0x7f61f0153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 1 --2- 10.20.193.16:0/36605491 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f61f00700f0 0x7f61f00704d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 1 -- 10.20.193.16:0/36605491 >> 10.20.193.16:0/36605491 conn(0x7f61f00cd770 msgr2=0x7f61f00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 1 -- 10.20.193.16:0/36605491 shutdown_connections 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 1 -- 10.20.193.16:0/36605491 wait complete. 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 10 monclient: build_initial_monmap 2026-02-17T06:02:22.629 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.616+0000 7f61f656a640 10 monclient: monmap: 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:epoch 0 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 0.000000 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:created 0.000000 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.noname-a 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.noname-b 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.noname-c 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 librados: starting msgr at 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 librados: starting objecter 2026-02-17T06:02:22.630 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 Processor -- start 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 -- start start 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 librados: setting wanted keys 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 librados: calling monclient init 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient: init 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f61f00700f0 0x7f61f01e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient(hunting): picked mon.noname-b con 0x7f61f00700f0 addr [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f61f00c7fe0 0x7f61f0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient(hunting): picked mon.noname-a con 0x7f61f00c7fe0 addr [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 --2- >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f61f01536f0 0x7f61f0073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.631 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient(hunting): picked mon.noname-c con 0x7f61f01536f0 addr [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 -- --> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0154060 con 0x7f61f00c7fe0 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 -- --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_getmap magic: 0 -- 0x7f61f00c5680 con 0x7f61f00700f0 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 1 -- --> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0074c00 con 0x7f61f01536f0 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61f656a640 10 monclient(hunting): authenticate will time out at 1395.186519s 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61effff640 1 --2- >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f61f00700f0 0x7f61f01e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61ef7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f61f00c7fe0 0x7f61f0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61ef7fe640 1 --2- >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f61f00c7fe0 0x7f61f0072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.16:3300/0 says I am v2:10.20.193.16:59444/0 (socket says 10.20.193.16:59444) 2026-02-17T06:02:22.632 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61ef7fe640 1 -- 10.20.193.16:0/2668651908 learned_addr learned my addr 10.20.193.16:0/2668651908 (peer_addr_for_me v2:10.20.193.16:0/0) 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61ef7fe640 10 monclient(hunting): get_auth_request con 0x7f61f00c7fe0 auth_method 0 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.617+0000 7f61ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61f4ae0640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f61f01536f0 0x7f61f0073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61effff640 10 monclient(hunting): get_auth_request con 0x7f61f00700f0 auth_method 0 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61f4ae0640 10 monclient(hunting): get_auth_request con 0x7f61f01536f0 auth_method 0 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61f4ae0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:22.633 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61f4ae0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61f4ae0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61f4ae0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61f4ae0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.618+0000 7f61f4ae0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.619+0000 7f61effff640 10 monclient(hunting): handle_auth_done global_id 5335 payload 824 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.619+0000 7f61effff640 1 -- 10.20.193.16:0/2668651908 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f61f01536f0 msgr2=0x7f61f0073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.634 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.619+0000 7f61effff640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f61f01536f0 0x7f61f0073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.619+0000 7f61effff640 1 -- 10.20.193.16:0/2668651908 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f61f00c7fe0 msgr2=0x7f61f0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.619+0000 7f61effff640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f61f00c7fe0 0x7f61f0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.619+0000 7f61effff640 10 monclient: _finish_hunting 0 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.619+0000 7f61effff640 1 monclient: found mon.noname-b 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.619+0000 7f61effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.619+0000 7f61effff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.128:3300/0 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.619+0000 7f61effff640 1 -- 10.20.193.16:0/2668651908 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f0144530 con 0x7f61f00700f0 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.620+0000 7f61effff640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f61f00700f0 0x7f61f01e89a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f61dc036000 tx=0x7f61dc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.620+0000 7f61ecff9640 10 client.?.objecter ms_handle_connect 0x7f61f00700f0 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.620+0000 7f61ecff9640 10 client.?.objecter resend_mon_ops 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 1 -- 10.20.193.16:0/2668651908 <== mon.1 v2:10.20.193.128:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc03c040 con 0x7f61f00700f0 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:22.635 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 10 monclient: dump: 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.636 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 10 monclient: _finish_auth 0 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 10 monclient: _check_auth_tickets 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 1 -- 10.20.193.16:0/2668651908 <== mon.1 v2:10.20.193.128:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc042450 con 0x7f61f00700f0 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 1 -- 10.20.193.16:0/2668651908 <== mon.1 v2:10.20.193.128:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc042820 con 0x7f61f00700f0 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ecff9640 10 monclient: dump: 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:epoch 1 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:fsid bd9382b6-548c-4f42-ae2b-be4ff6fd8cf6 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:last_changed 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:created 2026-02-17T05:52:59.489840+0000 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:22.637 INFO:teuthology.orchestra.run.trial016.stderr:election_strategy: 1 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:0: [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] mon.a 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:1: [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] mon.b 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2: [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] mon.c 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61f656a640 5 monclient: authenticate success, global_id 5335 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61f656a640 10 monclient: _renew_subs 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61ee7fc640 4 set_mon_vals no callback set 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61f656a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61f656a640 1 -- 10.20.193.16:0/2668651908 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f0144810 con 0x7f61f00700f0 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61f656a640 10 client.5335.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61f656a640 10 monclient: _renew_subs 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61f656a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61f656a640 1 -- 10.20.193.16:0/2668651908 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f0144dd0 con 0x7f61f00700f0 2026-02-17T06:02:22.638 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.622+0000 7f61f656a640 1 librados: init done 2026-02-17T06:02:22.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.623+0000 7f61f656a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:22.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.623+0000 7f61f656a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:22.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.623+0000 7f61f656a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:22.639 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.623+0000 7f61f656a640 1 -- 10.20.193.16:0/2668651908 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f00704d0 con 0x7f61f00700f0 2026-02-17T06:02:22.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.634+0000 7f61ecff9640 1 -- 10.20.193.16:0/2668651908 <== mon.1 v2:10.20.193.128:3300/0 4 ==== mgrmap(e 5) ==== 100057+0+0 (secure 0 0 0) 0x7f61dc0429c0 con 0x7f61f00700f0 2026-02-17T06:02:22.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.635+0000 7f61ecff9640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f61c0079140 0x7f61c007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:22.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.635+0000 7f61ef7fe640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f61c0079140 0x7f61c007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:22.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.635+0000 7f61ef7fe640 10 monclient: get_auth_request con 0x7f61c0079140 auth_method 0 2026-02-17T06:02:22.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.635+0000 7f61ecff9640 10 client.5335.objecter ms_handle_connect 0x7f61c0079140 2026-02-17T06:02:22.640 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.635+0000 7f61ef7fe640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f61c0079140 0x7f61c007b600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f61e0007d40 tx=0x7f61e0007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:22.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.733+0000 7f61ecff9640 1 -- 10.20.193.16:0/2668651908 <== mon.1 v2:10.20.193.128:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 8944+0+0 (secure 0 0 0) 0x7f61dc02f9a0 con 0x7f61f00700f0 2026-02-17T06:02:22.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.733+0000 7f61ecff9640 10 client.5335.objecter ms_dispatch2 0x7f61f0000b90 osd_map(34..34 src has 1..34) 2026-02-17T06:02:22.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.733+0000 7f61ecff9640 3 client.5335.objecter handle_osd_map got epochs [34,34] > 0 2026-02-17T06:02:22.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.733+0000 7f61ecff9640 3 client.5335.objecter handle_osd_map decoding full epoch 34 2026-02-17T06:02:22.734 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.733+0000 7f61ecff9640 20 client.5335.objecter dump_active .. 0 homeless 2026-02-17T06:02:22.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.739+0000 7f61ecff9640 1 -- 10.20.193.16:0/2668651908 <== mon.1 v2:10.20.193.128:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f61dc0876f0 con 0x7f61f00700f0 2026-02-17T06:02:22.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.739+0000 7f61ecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:22.740 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.739+0000 7f61ecff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:22.822 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.822+0000 7f61f656a640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-17T06:02:22.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.822+0000 7f61f656a640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-17T06:02:22.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.822+0000 7f61f656a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.128:3300/0 2026-02-17T06:02:22.823 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.822+0000 7f61f656a640 1 -- 10.20.193.16:0/2668651908 --> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f61f015e0d0 con 0x7f61f00700f0 2026-02-17T06:02:22.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.898+0000 7f61ecff9640 1 -- 10.20.193.16:0/2668651908 <== mon.1 v2:10.20.193.128:3300/0 7 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f61dc08b9f0 con 0x7f61f00700f0 2026-02-17T06:02:22.899 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.898+0000 7f61ecff9640 10 monclient: handle_config config(1 keys) 2026-02-17T06:02:22.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.898+0000 7f61ee7fc640 4 set_mon_vals no callback set 2026-02-17T06:02:22.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.898+0000 7f61ee7fc640 10 set_mon_vals mon_health_to_clog = false 2026-02-17T06:02:22.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.898+0000 7f61ecff9640 1 -- 10.20.193.16:0/2668651908 <== mon.1 v2:10.20.193.128:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v8)=0 v8) ==== 123+0+0 (secure 0 0 0) 0x7f61dc0908a0 con 0x7f61f00700f0 2026-02-17T06:02:22.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.898+0000 7f61ecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-17T06:02:22.900 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.898+0000 7f61ecff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:22.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.900+0000 7f61f656a640 10 librados: watch_flush enter 2026-02-17T06:02:22.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.900+0000 7f61f656a640 10 librados: watch_flush exit 2026-02-17T06:02:22.902 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.900+0000 7f61f656a640 20 client.5335.objecter shutdown clearing up homeless session... 2026-02-17T06:02:22.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.900+0000 7f61f656a640 10 client.5335.objecter successfully canceled tick 2026-02-17T06:02:22.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.900+0000 7f61f656a640 1 -- 10.20.193.16:0/2668651908 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f61c0079140 msgr2=0x7f61c007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.900+0000 7f61f656a640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f61c0079140 0x7f61c007b600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f61e0007d40 tx=0x7f61e0007c90 comp rx=0 tx=0).stop 2026-02-17T06:02:22.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.900+0000 7f61f656a640 10 monclient: shutdown 2026-02-17T06:02:22.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.900+0000 7f61f656a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:22.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.900+0000 7f61f656a640 1 -- 10.20.193.16:0/2668651908 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f61f00700f0 msgr2=0x7f61f01e89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:22.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.900+0000 7f61f656a640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f61f00700f0 0x7f61f01e89a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f61dc036000 tx=0x7f61dc0047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.901+0000 7f61f656a640 1 -- 10.20.193.16:0/2668651908 shutdown_connections 2026-02-17T06:02:22.903 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.901+0000 7f61f656a640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.16:6800/3354619179,v1:10.20.193.16:6801/3354619179] conn(0x7f61c0079140 0x7f61c007b600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.901+0000 7f61f656a640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.148:3300/0,v1:10.20.193.148:6789/0] conn(0x7f61f01536f0 0x7f61f0073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.901+0000 7f61f656a640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.16:3300/0,v1:10.20.193.16:6789/0] conn(0x7f61f00c7fe0 0x7f61f0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.901+0000 7f61f656a640 1 --2- 10.20.193.16:0/2668651908 >> [v2:10.20.193.128:3300/0,v1:10.20.193.128:6789/0] conn(0x7f61f00700f0 0x7f61f01e89a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:22.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.901+0000 7f61f656a640 1 -- 10.20.193.16:0/2668651908 >> 10.20.193.16:0/2668651908 conn(0x7f61f00cd770 msgr2=0x7f61f014f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:22.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.901+0000 7f61f656a640 1 -- 10.20.193.16:0/2668651908 shutdown_connections 2026-02-17T06:02:22.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.901+0000 7f61f656a640 1 -- 10.20.193.16:0/2668651908 wait complete. 2026-02-17T06:02:22.904 INFO:teuthology.orchestra.run.trial016.stderr:2026-02-17T06:02:22.901+0000 7f61f656a640 1 librados: shutdown 2026-02-17T06:02:22.910 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks/ceph.py", line 1922, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks/ceph.py", line 1484, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks/ceph_manager.py", line 3303, in wait_until_healthy assert time.time() - start < timeout, \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: timeout expired in wait_until_healthy 2026-02-17T06:02:22.914 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-17T06:02:22.915 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-17T06:02:22.915 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-17T06:02:22.915 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:02:22.922 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T06:02:22.917+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 44800) UID: 0 2026-02-17T06:02:22.922 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T06:02:22.918+0000 c69b640 -1 osd.0 34 *** Got signal Terminated *** 2026-02-17T06:02:24.284 INFO:tasks.ceph.osd.0.trial016.stderr:2026-02-17T06:02:24.283+0000 b699640 -1 monclient: get_auth_request but no auth handler is set up 2026-02-17T06:02:29.017 INFO:tasks.ceph.osd.0:Stopped 2026-02-17T06:02:29.018 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2026-02-17T06:02:29.018 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:02:29.025 INFO:tasks.ceph.osd.4.trial016.stderr:2026-02-17T06:02:29.020+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 44802) UID: 0 2026-02-17T06:02:29.025 INFO:tasks.ceph.osd.4.trial016.stderr:2026-02-17T06:02:29.021+0000 c69b640 -1 osd.4 36 *** Got signal Terminated *** 2026-02-17T06:02:29.333 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:08:08.161 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:29.336 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:08:08.163 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:29.344 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:08:08.172 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:29.361 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:08:08.187 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:29.362 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:08:08.188 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:29.363 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:08:08.152 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:29.365 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:08:08.154 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:29.371 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:08:08.197 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:29.374 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:08:08.164 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:35.120 INFO:tasks.ceph.osd.4:Stopped 2026-02-17T06:02:35.120 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2026-02-17T06:02:35.121 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:02:35.128 INFO:tasks.ceph.osd.8.trial016.stderr:2026-02-17T06:02:35.123+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.8.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 8 (PID: 44854) UID: 0 2026-02-17T06:02:35.128 INFO:tasks.ceph.osd.8.trial016.stderr:2026-02-17T06:02:35.123+0000 c69b640 -1 osd.8 38 *** Got signal Terminated *** 2026-02-17T06:02:36.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2026-02-17T06:02:41.222 INFO:tasks.ceph.osd.8:Stopped 2026-02-17T06:02:41.223 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2026-02-17T06:02:41.223 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:02:41.230 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T06:02:41.225+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.12.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 12 (PID: 44855) UID: 0 2026-02-17T06:02:41.230 INFO:tasks.ceph.osd.12.trial016.stderr:2026-02-17T06:02:41.226+0000 c69b640 -1 osd.12 40 *** Got signal Terminated *** 2026-02-17T06:02:43.048 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2026-02-17T06:02:47.324 INFO:tasks.ceph.osd.12:Stopped 2026-02-17T06:02:47.324 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-17T06:02:47.324 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:02:47.331 INFO:tasks.ceph.osd.1.trial128.stderr:2026-02-17T06:02:47.325+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 44594) UID: 0 2026-02-17T06:02:47.331 INFO:tasks.ceph.osd.1.trial128.stderr:2026-02-17T06:02:47.326+0000 c69b640 -1 osd.1 41 *** Got signal Terminated *** 2026-02-17T06:02:49.410 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:08:28.237 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:49.436 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:08:28.263 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:49.448 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:08:28.275 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:49.461 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:08:28.287 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:49.462 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:08:28.251 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:49.463 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:08:28.289 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:49.463 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:08:28.252 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:49.476 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:08:28.265 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:49.479 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:08:28.305 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:02:53.426 INFO:tasks.ceph.osd.1:Stopped 2026-02-17T06:02:53.426 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2026-02-17T06:02:53.426 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:02:53.434 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T06:02:53.428+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 44595) UID: 0 2026-02-17T06:02:53.434 INFO:tasks.ceph.osd.5.trial128.stderr:2026-02-17T06:02:53.428+0000 c69b640 -1 osd.5 43 *** Got signal Terminated *** 2026-02-17T06:02:59.528 INFO:tasks.ceph.osd.5:Stopped 2026-02-17T06:02:59.528 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2026-02-17T06:02:59.528 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:02:59.535 INFO:tasks.ceph.osd.9.trial128.stderr:2026-02-17T06:02:59.530+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.9.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 9 (PID: 44596) UID: 0 2026-02-17T06:02:59.536 INFO:tasks.ceph.osd.9.trial128.stderr:2026-02-17T06:02:59.531+0000 c69b640 -1 osd.9 45 *** Got signal Terminated *** 2026-02-17T06:03:05.629 INFO:tasks.ceph.osd.9:Stopped 2026-02-17T06:03:05.629 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2026-02-17T06:03:05.630 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:03:05.637 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T06:03:05.631+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.13.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 13 (PID: 44601) UID: 0 2026-02-17T06:03:05.637 INFO:tasks.ceph.osd.13.trial128.stderr:2026-02-17T06:03:05.632+0000 c69b640 -1 osd.13 47 *** Got signal Terminated *** 2026-02-17T06:03:09.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.13 is failed for ~0s 2026-02-17T06:03:09.402 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:08:48.229 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:09.404 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:08:48.232 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:09.415 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:08:48.243 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:09.430 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:08:48.256 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:09.432 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:08:48.258 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:09.435 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:08:48.224 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:09.437 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:08:48.226 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:09.443 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:08:48.269 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:09.449 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:08:48.238 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:11.731 INFO:tasks.ceph.osd.13:Stopped 2026-02-17T06:03:11.732 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-17T06:03:11.732 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:03:11.740 INFO:tasks.ceph.osd.2.trial148.stderr:2026-02-17T06:03:11.734+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 44698) UID: 0 2026-02-17T06:03:11.740 INFO:tasks.ceph.osd.2.trial148.stderr:2026-02-17T06:03:11.735+0000 c69b640 -1 osd.2 49 *** Got signal Terminated *** 2026-02-17T06:03:12.703 INFO:tasks.ceph.osd.2.trial148.stderr:2026-02-17T06:03:12.701+0000 b699640 -1 monclient: get_auth_request but no auth handler is set up 2026-02-17T06:03:15.499 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.13 has been restored 2026-02-17T06:03:17.834 INFO:tasks.ceph.osd.2:Stopped 2026-02-17T06:03:17.834 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2026-02-17T06:03:17.835 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:03:17.842 INFO:tasks.ceph.osd.6.trial148.stderr:2026-02-17T06:03:17.837+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 44701) UID: 0 2026-02-17T06:03:17.842 INFO:tasks.ceph.osd.6.trial148.stderr:2026-02-17T06:03:17.837+0000 c69b640 -1 osd.6 50 *** Got signal Terminated *** 2026-02-17T06:03:23.935 INFO:tasks.ceph.osd.6:Stopped 2026-02-17T06:03:23.936 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2026-02-17T06:03:23.936 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:03:23.943 INFO:tasks.ceph.osd.10.trial148.stderr:2026-02-17T06:03:23.938+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.10.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 10 (PID: 44702) UID: 0 2026-02-17T06:03:23.943 INFO:tasks.ceph.osd.10.trial148.stderr:2026-02-17T06:03:23.939+0000 c69b640 -1 osd.10 51 *** Got signal Terminated *** 2026-02-17T06:03:27.717 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.10 is failed for ~0s 2026-02-17T06:03:29.408 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:09:08.236 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:29.432 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:09:08.259 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:29.433 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:09:08.260 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:29.434 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:09:08.262 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:29.444 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:09:08.270 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:29.445 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:09:08.272 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:29.459 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:09:08.248 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:29.460 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:09:08.249 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:29.469 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:09:08.259 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:30.037 INFO:tasks.ceph.osd.10:Stopped 2026-02-17T06:03:30.038 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2026-02-17T06:03:30.038 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:03:30.045 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T06:03:30.040+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.14.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 14 (PID: 44705) UID: 0 2026-02-17T06:03:30.045 INFO:tasks.ceph.osd.14.trial148.stderr:2026-02-17T06:03:30.040+0000 c69b640 -1 osd.14 53 *** Got signal Terminated *** 2026-02-17T06:03:33.725 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.10 has been restored 2026-02-17T06:03:36.139 INFO:tasks.ceph.osd.14:Stopped 2026-02-17T06:03:36.140 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-17T06:03:36.140 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:03:36.159 INFO:tasks.ceph.osd.3.trial164.stderr:2026-02-17T06:03:36.153+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 44909) UID: 0 2026-02-17T06:03:36.159 INFO:tasks.ceph.osd.3.trial164.stderr:2026-02-17T06:03:36.154+0000 c69b640 -1 osd.3 54 *** Got signal Terminated *** 2026-02-17T06:03:39.531 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2026-02-17T06:03:42.242 INFO:tasks.ceph.osd.3:Stopped 2026-02-17T06:03:42.243 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2026-02-17T06:03:42.243 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:03:42.250 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T06:03:42.245+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 44910) UID: 0 2026-02-17T06:03:42.250 INFO:tasks.ceph.osd.7.trial164.stderr:2026-02-17T06:03:42.246+0000 c69b640 -1 osd.7 54 *** Got signal Terminated *** 2026-02-17T06:03:45.238 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2026-02-17T06:03:45.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2026-02-17T06:03:48.344 INFO:tasks.ceph.osd.7:Stopped 2026-02-17T06:03:48.344 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2026-02-17T06:03:48.344 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:03:48.351 INFO:tasks.ceph.osd.11.trial164.stderr:2026-02-17T06:03:48.347+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.11.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 11 (PID: 44914) UID: 0 2026-02-17T06:03:48.351 INFO:tasks.ceph.osd.11.trial164.stderr:2026-02-17T06:03:48.347+0000 c69b640 -1 osd.11 54 *** Got signal Terminated *** 2026-02-17T06:03:49.438 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:09:28.265 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:49.463 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:09:28.289 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:49.463 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:09:28.290 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:49.464 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:09:28.253 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:49.464 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:09:28.291 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:49.465 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:09:28.254 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:49.472 INFO:tasks.ceph.mon.b.trial128.stderr:==00:00:09:28.299 44359== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:49.474 INFO:tasks.ceph.mon.a.trial016.stderr:==00:00:09:28.302 44427== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:49.475 INFO:tasks.ceph.mon.c.trial148.stderr:==00:00:09:28.264 44463== Warning: unimplemented fcntl command: 1036 2026-02-17T06:03:50.843 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.11 is failed for ~0s 2026-02-17T06:03:50.843 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2026-02-17T06:03:54.446 INFO:tasks.ceph.osd.11:Stopped 2026-02-17T06:03:54.446 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2026-02-17T06:03:54.447 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:03:54.453 INFO:tasks.ceph.osd.15.trial164.stderr:2026-02-17T06:03:54.447+0000 c69b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.15.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 15 (PID: 44916) UID: 0 2026-02-17T06:03:54.454 INFO:tasks.ceph.osd.15.trial164.stderr:2026-02-17T06:03:54.448+0000 c69b640 -1 osd.15 54 *** Got signal Terminated *** 2026-02-17T06:03:56.348 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 is failed for ~0s 2026-02-17T06:03:56.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.11 has been restored 2026-02-17T06:04:00.548 INFO:tasks.ceph.osd.15:Stopped 2026-02-17T06:04:00.548 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-17T06:04:00.549 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2026-02-17T06:04:00.549 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:04:00.549 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T06:04:00.549+0000 7fcd1c3f9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i y (PID: 44475) UID: 0 2026-02-17T06:04:00.550 INFO:tasks.ceph.mgr.y.trial016.stderr:2026-02-17T06:04:00.549+0000 7fcd1c3f9640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2026-02-17T06:04:00.583 INFO:tasks.ceph.mgr.y:Stopped 2026-02-17T06:04:00.584 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-17T06:04:00.584 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:04:00.616 INFO:tasks.ceph.mgr.x:Stopped 2026-02-17T06:04:00.616 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-17T06:04:00.616 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T06:04:00.616 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:04:00.634 INFO:tasks.ceph.mon.a.trial016.stderr:2026-02-17T06:04:00.628+0000 aca4640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 44425) UID: 0 2026-02-17T06:04:00.635 INFO:tasks.ceph.mon.a.trial016.stderr:2026-02-17T06:04:00.628+0000 aca4640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-17T06:04:01.551 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2026-02-17T06:04:01.551 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 has been restored 2026-02-17T06:04:06.717 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T06:04:06.718 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T06:04:06.718 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:04:06.725 INFO:tasks.ceph.mon.b.trial128.stderr:2026-02-17T06:04:06.720+0000 aca4640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 44357) UID: 0 2026-02-17T06:04:06.727 INFO:tasks.ceph.mon.b.trial128.stderr:2026-02-17T06:04:06.720+0000 aca4640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2026-02-17T06:04:06.753 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~5s 2026-02-17T06:04:11.855 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2026-02-17T06:04:11.855 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2026-02-17T06:04:12.819 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T06:04:12.819 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-17T06:04:12.819 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-17T06:04:12.827 INFO:tasks.ceph.mon.c.trial148.stderr:2026-02-17T06:04:12.821+0000 aca4640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 44461) UID: 0 2026-02-17T06:04:12.829 INFO:tasks.ceph.mon.c.trial148.stderr:2026-02-17T06:04:12.822+0000 aca4640 -1 mon.c@2(probing) e1 *** Got Signal Terminated *** 2026-02-17T06:04:16.856 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2026-02-17T06:04:16.856 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2026-02-17T06:04:18.920 INFO:tasks.ceph.mon.c:Stopped 2026-02-17T06:04:18.921 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-17T06:04:18.921 DEBUG:teuthology.orchestra.run.trial016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-17T06:04:18.951 INFO:teuthology.orchestra.run.trial016.stdout:2026-02-17T05:56:31.933606+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2026-02-17T06:04:18.951 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2026-02-17T06:04:18.952 DEBUG:teuthology.orchestra.run.trial016:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-17T06:04:19.020 DEBUG:teuthology.orchestra.run.trial016:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-17T06:04:19.087 DEBUG:teuthology.orchestra.run.trial016:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-17T06:04:19.153 INFO:teuthology.orchestra.run.trial016.stdout:2026-02-17T05:56:31.933606+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2026-02-17T06:04:19.153 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial016.front.sepia.ceph.com 2026-02-17T06:04:19.153 DEBUG:teuthology.orchestra.run.trial016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-17T06:04:19.228 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@trial016.front.sepia.ceph.com 2026-02-17T06:04:19.228 DEBUG:teuthology.orchestra.run.trial016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2026-02-17T06:04:19.295 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@trial016.front.sepia.ceph.com 2026-02-17T06:04:19.295 DEBUG:teuthology.orchestra.run.trial016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2026-02-17T06:04:19.361 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@trial016.front.sepia.ceph.com 2026-02-17T06:04:19.361 DEBUG:teuthology.orchestra.run.trial016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2026-02-17T06:04:19.428 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial128.front.sepia.ceph.com 2026-02-17T06:04:19.428 DEBUG:teuthology.orchestra.run.trial128:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-17T06:04:19.460 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@trial128.front.sepia.ceph.com 2026-02-17T06:04:19.461 DEBUG:teuthology.orchestra.run.trial128:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2026-02-17T06:04:19.536 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@trial128.front.sepia.ceph.com 2026-02-17T06:04:19.537 DEBUG:teuthology.orchestra.run.trial128:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2026-02-17T06:04:19.605 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@trial128.front.sepia.ceph.com 2026-02-17T06:04:19.605 DEBUG:teuthology.orchestra.run.trial128:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2026-02-17T06:04:19.674 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial148.front.sepia.ceph.com 2026-02-17T06:04:19.674 DEBUG:teuthology.orchestra.run.trial148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-17T06:04:19.707 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@trial148.front.sepia.ceph.com 2026-02-17T06:04:19.707 DEBUG:teuthology.orchestra.run.trial148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2026-02-17T06:04:19.776 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@trial148.front.sepia.ceph.com 2026-02-17T06:04:19.776 DEBUG:teuthology.orchestra.run.trial148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2026-02-17T06:04:19.845 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@trial148.front.sepia.ceph.com 2026-02-17T06:04:19.845 DEBUG:teuthology.orchestra.run.trial148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2026-02-17T06:04:19.912 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial164.front.sepia.ceph.com 2026-02-17T06:04:19.912 DEBUG:teuthology.orchestra.run.trial164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-17T06:04:19.946 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@trial164.front.sepia.ceph.com 2026-02-17T06:04:19.946 DEBUG:teuthology.orchestra.run.trial164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2026-02-17T06:04:20.018 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@trial164.front.sepia.ceph.com 2026-02-17T06:04:20.018 DEBUG:teuthology.orchestra.run.trial164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2026-02-17T06:04:20.137 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@trial164.front.sepia.ceph.com 2026-02-17T06:04:20.137 DEBUG:teuthology.orchestra.run.trial164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2026-02-17T06:04:20.167 INFO:tasks.ceph:Archiving mon data... 2026-02-17T06:04:20.169 DEBUG:teuthology.misc:Transferring archived files from trial016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/data/mon.a.tgz 2026-02-17T06:04:20.169 DEBUG:teuthology.orchestra.run.trial016:> mktemp 2026-02-17T06:04:20.188 INFO:teuthology.orchestra.run.trial016.stdout:/tmp/tmp.RAZJCnvT7p 2026-02-17T06:04:20.189 DEBUG:teuthology.orchestra.run.trial016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.RAZJCnvT7p 2026-02-17T06:04:20.300 DEBUG:teuthology.orchestra.run.trial016:> sudo chmod 0666 /tmp/tmp.RAZJCnvT7p 2026-02-17T06:04:20.382 DEBUG:teuthology.orchestra.remote:trial016:/tmp/tmp.RAZJCnvT7p is 424KB 2026-02-17T06:04:20.474 DEBUG:teuthology.orchestra.run.trial016:> rm -fr /tmp/tmp.RAZJCnvT7p 2026-02-17T06:04:20.490 DEBUG:teuthology.misc:Transferring archived files from trial128:/var/lib/ceph/mon/ceph-b to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/data/mon.b.tgz 2026-02-17T06:04:20.490 DEBUG:teuthology.orchestra.run.trial128:> mktemp 2026-02-17T06:04:20.507 INFO:teuthology.orchestra.run.trial128.stdout:/tmp/tmp.wCsDTkUJwW 2026-02-17T06:04:20.508 DEBUG:teuthology.orchestra.run.trial128:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.wCsDTkUJwW 2026-02-17T06:04:20.618 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod 0666 /tmp/tmp.wCsDTkUJwW 2026-02-17T06:04:20.701 DEBUG:teuthology.orchestra.remote:trial128:/tmp/tmp.wCsDTkUJwW is 424KB 2026-02-17T06:04:20.791 DEBUG:teuthology.orchestra.run.trial128:> rm -fr /tmp/tmp.wCsDTkUJwW 2026-02-17T06:04:20.808 DEBUG:teuthology.misc:Transferring archived files from trial148:/var/lib/ceph/mon/ceph-c to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/data/mon.c.tgz 2026-02-17T06:04:20.808 DEBUG:teuthology.orchestra.run.trial148:> mktemp 2026-02-17T06:04:20.825 INFO:teuthology.orchestra.run.trial148.stdout:/tmp/tmp.gwdGUHxf0y 2026-02-17T06:04:20.825 DEBUG:teuthology.orchestra.run.trial148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.gwdGUHxf0y 2026-02-17T06:04:20.937 DEBUG:teuthology.orchestra.run.trial148:> sudo chmod 0666 /tmp/tmp.gwdGUHxf0y 2026-02-17T06:04:21.015 DEBUG:teuthology.orchestra.remote:trial148:/tmp/tmp.gwdGUHxf0y is 424KB 2026-02-17T06:04:21.082 DEBUG:teuthology.orchestra.run.trial148:> rm -fr /tmp/tmp.gwdGUHxf0y 2026-02-17T06:04:21.099 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-17T06:04:21.099 DEBUG:teuthology.orchestra.run.trial016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-17T06:04:21.101 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-17T06:04:21.102 DEBUG:teuthology.orchestra.run.trial148:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-17T06:04:21.141 DEBUG:teuthology.orchestra.run.trial164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-17T06:04:21.380 DEBUG:tasks.ceph:valgrind exception message: valgrind error: Leak_StillReachable malloc malloc _dl_new_object 2026-02-17T06:04:21.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2026-02-17T06:04:22.063 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-17T06:04:22.063 INFO:tasks.ceph:Archiving crash dumps... 2026-02-17T06:04:22.066 DEBUG:teuthology.misc:Transferring archived files from trial016:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial016/crash 2026-02-17T06:04:22.067 DEBUG:teuthology.orchestra.run.trial016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-17T06:04:22.096 DEBUG:teuthology.misc:Transferring archived files from trial128:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial128/crash 2026-02-17T06:04:22.098 DEBUG:teuthology.orchestra.run.trial128:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-17T06:04:22.124 DEBUG:teuthology.misc:Transferring archived files from trial148:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial148/crash 2026-02-17T06:04:22.125 DEBUG:teuthology.orchestra.run.trial148:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-17T06:04:22.151 DEBUG:teuthology.misc:Transferring archived files from trial164:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial164/crash 2026-02-17T06:04:22.152 DEBUG:teuthology.orchestra.run.trial164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-17T06:04:22.178 INFO:tasks.ceph:Compressing logs... 2026-02-17T06:04:22.178 DEBUG:teuthology.orchestra.run.trial016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:04:22.180 DEBUG:teuthology.orchestra.run.trial128:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:04:22.182 DEBUG:teuthology.orchestra.run.trial148:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:04:22.196 DEBUG:teuthology.orchestra.run.trial164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:04:22.202 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50426.log 2026-02-17T06:04:22.224 INFO:teuthology.orchestra.run.trial128.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2026-02-17T06:04:22.224 INFO:teuthology.orchestra.run.trial128.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.5.log 2026-02-17T06:04:22.225 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.13.log 2026-02-17T06:04:22.225 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/valgrind/osd.5.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.1.log 2026-02-17T06:04:22.225 INFO:teuthology.orchestra.run.trial128.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.5.log.gz 2026-02-17T06:04:22.225 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/valgrind/osd.13.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.9.log 2026-02-17T06:04:22.225 INFO:teuthology.orchestra.run.trial128.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.13.log.gz 2026-02-17T06:04:22.225 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/valgrind/osd.1.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2026-02-17T06:04:22.225 INFO:teuthology.orchestra.run.trial128.stderr: 67.9% -- replaced with /var/log/ceph/valgrind/osd.1.log.gz 2026-02-17T06:04:22.226 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/valgrind/osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-17T06:04:22.226 INFO:teuthology.orchestra.run.trial128.stderr: 67.8% -- replaced with /var/log/ceph/valgrind/osd.9.log.gz 2026-02-17T06:04:22.226 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2026-02-17T06:04:22.226 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2026-02-17T06:04:22.226 INFO:teuthology.orchestra.run.trial128.stderr: 94.2% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2026-02-17T06:04:22.226 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-17T06:04:22.226 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/ceph-osd.13.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-17T06:04:22.227 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/ceph-osd.1.log: gzip 90.6% -5 -- replaced with /var/log/ceph/ceph.audit.log.gz --verbose 2026-02-17T06:04:22.227 INFO:teuthology.orchestra.run.trial128.stderr: -- /var/log/ceph/ceph-osd.9.log 2026-02-17T06:04:22.227 INFO:teuthology.orchestra.run.trial128.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-osd.9.log: 88.8% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-17T06:04:22.235 INFO:teuthology.orchestra.run.trial148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-17T06:04:22.235 INFO:teuthology.orchestra.run.trial148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2026-02-17T06:04:22.235 INFO:teuthology.orchestra.run.trial148.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.2.log 2026-02-17T06:04:22.235 INFO:teuthology.orchestra.run.trial148.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.6.log 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr:/var/log/ceph/valgrind/osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.14.log 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr: 67.9% -- replaced with /var/log/ceph/valgrind/osd.2.log.gz 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.10.log 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr:/var/log/ceph/valgrind/osd.6.log: /var/log/ceph/valgrind/osd.14.log: 68.2% -- replaced with /var/log/ceph/valgrind/osd.6.log.gz 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr: 68.1% -- replaced with /var/log/ceph/valgrind/osd.14.log.gz 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr:/var/log/ceph/valgrind/osd.10.log: 67.9% -- replaced with /var/log/ceph/valgrind/osd.10.log.gz 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr:/var/log/ceph/valgrind/mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr:/var/log/ceph/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-17T06:04:22.236 INFO:teuthology.orchestra.run.trial148.stderr:/var/log/ceph/ceph-osd.14.log: 94.2% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2026-02-17T06:04:22.237 INFO:teuthology.orchestra.run.trial148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2026-02-17T06:04:22.237 INFO:teuthology.orchestra.run.trial148.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-17T06:04:22.237 INFO:teuthology.orchestra.run.trial148.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph.log: 90.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-17T06:04:22.237 INFO:teuthology.orchestra.run.trial148.stderr: 88.8% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44556.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44603.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44128.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44603.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44556.log.gz 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44128.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44603.log.gz 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44321.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44744.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44650.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44321.log.gz/var/log/ceph/ceph-client.admin.44744.log: 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.7.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44650.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44744.log.gz 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.11.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44650.log.gz 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/valgrind/osd.7.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.15.log 2026-02-17T06:04:22.254 INFO:teuthology.orchestra.run.trial164.stderr: 67.9% -- replaced with /var/log/ceph/valgrind/osd.7.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/valgrind/osd.11.log: 68.1%gzip -- replaced with /var/log/ceph/valgrind/osd.11.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr: -5 --verbose -- /var/log/ceph/valgrind/osd.3.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/valgrind/osd.15.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr: 68.1% -- replaced with /var/log/ceph/valgrind/osd.15.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/valgrind/osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.3.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44697.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44368.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44697.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44075.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44697.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44462.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44368.log: /var/log/ceph/ceph-client.admin.44075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44791.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44368.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44462.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44075.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44791.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44462.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44227.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44791.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44415.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.44227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44509.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44227.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44415.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-client.admin.44509.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44415.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44509.log.gz 2026-02-17T06:04:22.255 INFO:teuthology.orchestra.run.trial164.stderr:/var/log/ceph/ceph-osd.15.log: 94.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-17T06:04:22.257 INFO:teuthology.orchestra.run.trial128.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2026-02-17T06:04:22.263 INFO:teuthology.orchestra.run.trial128.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2026-02-17T06:04:22.264 INFO:teuthology.orchestra.run.trial128.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-17T06:04:22.274 INFO:teuthology.orchestra.run.trial148.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-17T06:04:22.274 INFO:teuthology.orchestra.run.trial128.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49659.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54370.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49659.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50426.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52787.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49659.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51183.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55106.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52787.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54370.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51183.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52787.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56592.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51183.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-mon.a.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55106.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58356.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53326.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.58356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44909.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56592.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53326.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.58356.log.gz -5 --verbose 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr: -- /var/log/ceph/ceph-client.admin.48554.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.44909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55057.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53326.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.48554.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44909.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48554.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52640.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50447.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55057.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.52640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53228.log 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52640.log.gz 2026-02-17T06:04:22.275 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56837.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53228.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.49630.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50447.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56837.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53228.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57768.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49630.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56837.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57768.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49630.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55155.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56788.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-mgr.y.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57768.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55856.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56788.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55155.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50446.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55856.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56788.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45111.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53130.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55856.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.45111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50417.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50446.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45111.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53375.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53130.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44478.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53682.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50417.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53375.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.44478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57180.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57229.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57180.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53682.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50902.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.44478.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57180.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54762.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55253.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57229.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50902.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49663.log 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54762.log.gz 2026-02-17T06:04:22.276 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50430.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49663.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55253.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54125.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50430.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49663.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55449.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54125.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50430.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50403.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57866.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54125.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49633.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55449.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50403.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53032.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57866.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56445.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53927.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49633.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53032.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57621.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53927.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56445.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55694.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57621.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53927.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54812.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55694.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57621.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50826.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54812.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55694.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53633.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50826.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54812.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53473.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53633.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50826.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52052.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53633.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49662.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54959.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53473.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49662.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52052.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58209.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49662.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52738.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.58209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49650.log 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54959.log.gz 2026-02-17T06:04:22.277 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52738.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58209.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54910.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52738.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52199.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51302.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49650.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54910.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52346.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52199.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54517.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52346.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51302.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54272.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52346.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50422.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58111.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54517.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50422.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54272.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49661.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50422.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.58111.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.0.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49661.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.12.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58111.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/valgrind/osd.0.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.8.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49661.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 67.9% -- replaced with /var/log/ceph/valgrind/osd.0.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/valgrind/osd.12.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 67.9% -- replaced with /var/log/ceph/valgrind/osd.12.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/valgrind/osd.8.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.4.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.8.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/valgrind/mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55905.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/valgrind/osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57964.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 67.9% -- replaced with /var/log/ceph/valgrind/osd.4.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 94.2% -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52542.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55905.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51091.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52542.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57964.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50434.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51091.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52542.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51479.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51091.log.gz 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53424.log 2026-02-17T06:04:22.278 INFO:teuthology.orchestra.run.trial016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50434.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51400.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58403.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51479.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51400.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53424.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53522.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.58403.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51400.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55807.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58403.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55645.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53522.log: /var/log/ceph/ceph-client.admin.55807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52836.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53522.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55807.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56543.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51657.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55645.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52836.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44957.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51657.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56543.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52297.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51657.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.44957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55758.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52297.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44957.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48508.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 88.2%/var/log/ceph/ceph-client.admin.55758.log: -- replaced with /var/log/ceph/ceph-client.admin.52297.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49629.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.48508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51484.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55758.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48508.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56984.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49629.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51847.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57327.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51484.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51847.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56984.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51274.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51847.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55204.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55547.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57327.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55204.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51274.log.gz 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.12.log 2026-02-17T06:04:22.279 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55547.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55204.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51273.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55547.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-osd.12.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57523.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56297.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57523.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51273.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38129.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57523.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56297.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph.tmp-client.admin.38129.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38129.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56297.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53081.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58062.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55302.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53081.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.58062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56199.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55302.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58062.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45840.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54223.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55302.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.45840.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56199.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51704.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45840.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56248.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51704.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54223.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53829.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51704.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49645.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54615.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56248.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49645.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53829.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55008.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49645.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54419.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51089.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54615.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54419.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55008.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48945.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51089.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54419.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49628.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51089.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.48945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57474.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49628.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48945.log.gz 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48646.log 2026-02-17T06:04:22.280 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57474.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49628.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56739.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.48646.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57474.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56150.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56739.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48646.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51904.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56150.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56739.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51904.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56150.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49160.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph.audit.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51904.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49015.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58160.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49015.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49160.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58307.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.58160.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49015.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr: 90.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55954.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.58307.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58160.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57719.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55954.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58307.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50445.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57719.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55954.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51799.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50445.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57719.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48600.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50445.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54075.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.48600.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51799.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54321.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54075.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48600.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55498.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54321.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54075.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50873.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54321.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56935.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50873.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55498.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56101.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54025.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50873.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56101.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56935.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54664.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54025.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56101.log.gz 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48416.log 2026-02-17T06:04:22.281 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54664.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54025.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57082.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54664.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.48416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51953.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57082.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48416.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52493.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51953.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57082.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55596.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52493.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51953.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52493.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57033.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58013.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57033.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55596.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49653.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.58013.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57033.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54713.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49653.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58013.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52002.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54713.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49653.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57670.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52002.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54713.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57572.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57670.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52002.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54468.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57572.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57670.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57376.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54468.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57572.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50443.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54468.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49111.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50443.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57376.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52885.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49111.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50443.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50444.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52885.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49111.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49648.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50444.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52885.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51752.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49648.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50444.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44886.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51752.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49648.log.gz 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46099.log 2026-02-17T06:04:22.282 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.44886.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51752.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56052.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.46099.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44886.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49656.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46099.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49209.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49656.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56052.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53179.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49656.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49623.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53179.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49209.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51173.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53179.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52395.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51173.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49623.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52150.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52395.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51173.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53731.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52395.log.gz/var/log/ceph/ceph-client.admin.52150.log: 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50402.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53731.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52150.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47999.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50402.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53731.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52689.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50402.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.47999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49624.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52689.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47999.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50427.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49624.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52689.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57425.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr: 87.7%/var/log/ceph/ceph-client.admin.50427.log: -- replaced with /var/log/ceph/ceph-client.admin.49624.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52591.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57425.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50427.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50980.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52591.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57425.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53277.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52591.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53571.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53277.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50980.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52248.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53571.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53277.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55400.log 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr: 87.5%/var/log/ceph/ceph-client.admin.52248.log: -- replaced with /var/log/ceph/ceph-client.admin.53571.log.gz 2026-02-17T06:04:22.283 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56347.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55400.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52248.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51610.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56347.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55400.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56003.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56347.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.51610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54566.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51610.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52444.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57278.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56003.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52444.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54566.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48462.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50439.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52444.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.48462.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57278.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48853.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50439.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48462.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50414.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50439.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.48853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56396.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50414.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48853.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52101.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50414.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51477.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56396.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56886.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52101.log: /var/log/ceph/ceph-client.admin.51477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53780.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56886.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52101.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51477.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56690.log 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56886.log.gz 2026-02-17T06:04:22.284 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53780.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56690.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53780.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55351.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56690.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.55351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49042.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55351.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54861.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr: 88.8% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57817.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56641.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49042.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57817.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54861.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52934.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57817.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53976.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52983.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56641.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53976.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52934.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57131.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.52983.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53976.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56494.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57131.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52983.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50996.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.56494.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57131.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50996.log: gzip 88.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.56494.log.gz -- 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr: /var/log/ceph/ceph-client.admin.53878.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.50404.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50996.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54174.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.53878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49638.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.54174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58258.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53878.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.49638.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54174.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57915.log 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.58258.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49638.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr:/var/log/ceph/ceph-client.admin.57915.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58258.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57915.log.gz 2026-02-17T06:04:22.285 INFO:teuthology.orchestra.run.trial016.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2026-02-17T06:04:22.286 INFO:teuthology.orchestra.run.trial148.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2026-02-17T06:04:22.286 INFO:teuthology.orchestra.run.trial148.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2026-02-17T06:04:22.289 INFO:teuthology.orchestra.run.trial148.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2026-02-17T06:04:22.293 INFO:teuthology.orchestra.run.trial016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2026-02-17T06:04:22.300 INFO:teuthology.orchestra.run.trial164.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2026-02-17T06:04:22.302 INFO:teuthology.orchestra.run.trial164.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-17T06:04:22.309 INFO:teuthology.orchestra.run.trial148.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2026-02-17T06:04:22.309 INFO:teuthology.orchestra.run.trial164.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2026-02-17T06:04:22.310 INFO:teuthology.orchestra.run.trial148.stderr: 2026-02-17T06:04:22.310 INFO:teuthology.orchestra.run.trial148.stderr:real 0m0.100s 2026-02-17T06:04:22.310 INFO:teuthology.orchestra.run.trial148.stderr:user 0m0.334s 2026-02-17T06:04:22.310 INFO:teuthology.orchestra.run.trial148.stderr:sys 0m0.030s 2026-02-17T06:04:22.315 INFO:teuthology.orchestra.run.trial128.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2026-02-17T06:04:22.316 INFO:teuthology.orchestra.run.trial128.stderr: 2026-02-17T06:04:22.316 INFO:teuthology.orchestra.run.trial128.stderr:real 0m0.120s 2026-02-17T06:04:22.316 INFO:teuthology.orchestra.run.trial128.stderr:user 0m0.308s 2026-02-17T06:04:22.316 INFO:teuthology.orchestra.run.trial128.stderr:sys 0m0.041s 2026-02-17T06:04:22.319 INFO:teuthology.orchestra.run.trial164.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2026-02-17T06:04:22.320 INFO:teuthology.orchestra.run.trial016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2026-02-17T06:04:22.320 INFO:teuthology.orchestra.run.trial164.stderr: 2026-02-17T06:04:22.321 INFO:teuthology.orchestra.run.trial164.stderr:real 0m0.086s 2026-02-17T06:04:22.321 INFO:teuthology.orchestra.run.trial164.stderr:user 0m0.249s 2026-02-17T06:04:22.321 INFO:teuthology.orchestra.run.trial164.stderr:sys 0m0.037s 2026-02-17T06:04:22.391 INFO:teuthology.orchestra.run.trial016.stderr: 89.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-17T06:04:22.392 INFO:teuthology.orchestra.run.trial016.stderr: 2026-02-17T06:04:22.392 INFO:teuthology.orchestra.run.trial016.stderr:real 0m0.199s 2026-02-17T06:04:22.392 INFO:teuthology.orchestra.run.trial016.stderr:user 0m0.509s 2026-02-17T06:04:22.392 INFO:teuthology.orchestra.run.trial016.stderr:sys 0m0.134s 2026-02-17T06:04:22.393 INFO:tasks.ceph:Archiving logs... 2026-02-17T06:04:22.393 DEBUG:teuthology.misc:Transferring archived files from trial016:/var/log/ceph to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial016/log 2026-02-17T06:04:22.394 DEBUG:teuthology.orchestra.run.trial016:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-17T06:04:22.691 DEBUG:teuthology.misc:Transferring archived files from trial128:/var/log/ceph to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial128/log 2026-02-17T06:04:22.692 DEBUG:teuthology.orchestra.run.trial128:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-17T06:04:22.779 DEBUG:teuthology.misc:Transferring archived files from trial148:/var/log/ceph to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial148/log 2026-02-17T06:04:22.780 DEBUG:teuthology.orchestra.run.trial148:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-17T06:04:22.860 DEBUG:teuthology.misc:Transferring archived files from trial164:/var/log/ceph to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial164/log 2026-02-17T06:04:22.862 DEBUG:teuthology.orchestra.run.trial164:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-17T06:04:22.940 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks/ceph.py", line 1922, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks/ceph.py", line 1484, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks/ceph_manager.py", line 3303, in wait_until_healthy assert time.time() - start < timeout, \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: timeout expired in wait_until_healthy During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks/ceph.py", line 1919, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 46, in nested if exit(*exc): ^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable malloc malloc _dl_new_object 2026-02-17T06:04:22.942 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-17T06:04:22.950 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T06:04:22.965 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-17T06:04:22.965 DEBUG:teuthology.orchestra.run.trial016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T06:04:22.967 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T06:04:22.969 DEBUG:teuthology.orchestra.run.trial148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T06:04:22.971 DEBUG:teuthology.orchestra.run.trial164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T06:04:23.225 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-17T06:04:23.236 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-17T06:04:23.236 DEBUG:teuthology.orchestra.run.trial016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T06:04:23.239 DEBUG:teuthology.orchestra.run.trial128:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T06:04:23.241 DEBUG:teuthology.orchestra.run.trial148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T06:04:23.243 DEBUG:teuthology.orchestra.run.trial164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T06:04:23.253 INFO:teuthology.orchestra.run.trial016.stderr:bash: line 1: ntpq: command not found 2026-02-17T06:04:23.256 INFO:teuthology.orchestra.run.trial128.stderr:bash: line 1: ntpq: command not found 2026-02-17T06:04:23.256 INFO:teuthology.orchestra.run.trial148.stderr:bash: line 1: ntpq: command not found 2026-02-17T06:04:23.258 INFO:teuthology.orchestra.run.trial164.stderr:bash: line 1: ntpq: command not found 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial148.stdout:=============================================================================== 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial148.stdout:^* 107-172-222-7-host.coloc> 2 7 37 78 +259us[ +503us] +/- 14ms 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial148.stdout:^+ 144.202.41.38.vultruserc> 5 7 377 23 +1540us[+1540us] +/- 42ms 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial148.stdout:^+ kc1cloud.147.72.155.23.m> 3 7 377 24 +734us[ +734us] +/- 46ms 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial148.stdout:^+ sushi.ruselabs.com 2 6 377 216 -2272us[-2084us] +/- 34ms 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial128.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial128.stdout:=============================================================================== 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial128.stdout:^- 50.218.103.254 2 7 37 78 +3477us[+3615us] +/- 33ms 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial128.stdout:^+ 144.202.41.38.vultruserc> 5 6 377 22 +1687us[+1687us] +/- 42ms 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial128.stdout:^+ kc1cloud.147.72.155.23.m> 3 7 377 21 +757us[ +757us] +/- 46ms 2026-02-17T06:04:23.285 INFO:teuthology.orchestra.run.trial128.stdout:^* sushi.ruselabs.com 2 7 377 25 -1395us[-1249us] +/- 35ms 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial164.stdout:=============================================================================== 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial164.stdout:^+ callisto.psychotron.org 2 7 177 80 -48us[ +19us] +/- 47ms 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial164.stdout:^+ 144.202.41.38.vultruserc> 5 6 377 21 +1607us[+1607us] +/- 42ms 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial164.stdout:^+ kc1cloud.147.72.155.23.m> 3 6 377 22 +471us[ +471us] +/- 46ms 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial164.stdout:^* sushi.ruselabs.com 2 7 377 23 +1258us[+1327us] +/- 38ms 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial016.stdout:=============================================================================== 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial016.stdout:^- callisto.psychotron.org 2 6 377 27 +837us[ +837us] +/- 48ms 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial016.stdout:^+ 144.202.41.38.vultruserc> 5 6 377 22 +2048us[+2048us] +/- 42ms 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial016.stdout:^+ kc1cloud.147.72.155.23.m> 3 6 377 27 +940us[ +940us] +/- 46ms 2026-02-17T06:04:23.287 INFO:teuthology.orchestra.run.trial016.stdout:^* sushi.ruselabs.com 2 6 377 154 -1924us[-1896us] +/- 34ms 2026-02-17T06:04:23.287 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-17T06:04:23.299 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-17T06:04:23.300 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-17T06:04:23.313 DEBUG:teuthology.orchestra.run.trial016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-17T06:04:23.384 DEBUG:teuthology.orchestra.run.trial128:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-17T06:04:23.438 DEBUG:teuthology.orchestra.run.trial148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-17T06:04:23.493 DEBUG:teuthology.orchestra.run.trial164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-17T06:04:23.549 DEBUG:teuthology.orchestra.run.trial016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T06:04:23.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:04:23.579 DEBUG:teuthology.orchestra.run.trial128:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T06:04:23.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:04:23.605 DEBUG:teuthology.orchestra.run.trial148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T06:04:23.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:04:23.633 DEBUG:teuthology.orchestra.run.trial164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T06:04:23.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:04:23.663 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-17T06:04:23.678 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-17T06:04:23.692 INFO:teuthology.task.internal:Duration was 1172.713164 seconds 2026-02-17T06:04:23.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-17T06:04:23.708 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-17T06:04:23.708 DEBUG:teuthology.orchestra.run.trial016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T06:04:23.711 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T06:04:23.714 DEBUG:teuthology.orchestra.run.trial148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T06:04:23.717 DEBUG:teuthology.orchestra.run.trial164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T06:04:23.746 INFO:teuthology.orchestra.run.trial016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T06:04:23.750 INFO:teuthology.orchestra.run.trial128.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T06:04:23.751 INFO:teuthology.orchestra.run.trial148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T06:04:23.755 INFO:teuthology.orchestra.run.trial164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T06:04:24.186 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-17T06:04:24.186 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial016.front.sepia.ceph.com 2026-02-17T06:04:24.186 DEBUG:teuthology.orchestra.run.trial016:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T06:04:24.244 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial128.front.sepia.ceph.com 2026-02-17T06:04:24.244 DEBUG:teuthology.orchestra.run.trial128:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T06:04:24.265 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial148.front.sepia.ceph.com 2026-02-17T06:04:24.266 DEBUG:teuthology.orchestra.run.trial148:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T06:04:24.286 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial164.front.sepia.ceph.com 2026-02-17T06:04:24.286 DEBUG:teuthology.orchestra.run.trial164:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T06:04:24.306 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-17T06:04:24.306 DEBUG:teuthology.orchestra.run.trial016:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:04:24.308 DEBUG:teuthology.orchestra.run.trial128:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:04:24.309 DEBUG:teuthology.orchestra.run.trial148:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:04:24.328 DEBUG:teuthology.orchestra.run.trial164:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:04:24.407 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-17T06:04:24.407 DEBUG:teuthology.orchestra.run.trial016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:04:24.409 DEBUG:teuthology.orchestra.run.trial128:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:04:24.410 DEBUG:teuthology.orchestra.run.trial148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:04:24.411 DEBUG:teuthology.orchestra.run.trial164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:04:24.430 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T06:04:24.431 INFO:teuthology.orchestra.run.trial016.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T06:04:24.432 INFO:teuthology.orchestra.run.trial016.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:04:24.432 INFO:teuthology.orchestra.run.trial016.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T06:04:24.432 INFO:teuthology.orchestra.run.trial016.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T06:04:24.433 INFO:teuthology.orchestra.run.trial128.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T06:04:24.433 INFO:teuthology.orchestra.run.trial128.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T06:04:24.433 INFO:teuthology.orchestra.run.trial128.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:04:24.434 INFO:teuthology.orchestra.run.trial128.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T06:04:24.434 INFO:teuthology.orchestra.run.trial128.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T06:04:24.436 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T06:04:24.436 INFO:teuthology.orchestra.run.trial164.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T06:04:24.436 INFO:teuthology.orchestra.run.trial164.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:04:24.436 INFO:teuthology.orchestra.run.trial164.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T06:04:24.436 INFO:teuthology.orchestra.run.trial164.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T06:04:24.436 INFO:teuthology.orchestra.run.trial148.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T06:04:24.436 INFO:teuthology.orchestra.run.trial148.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T06:04:24.437 INFO:teuthology.orchestra.run.trial148.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T06:04:24.437 INFO:teuthology.orchestra.run.trial148.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:04:24.437 INFO:teuthology.orchestra.run.trial148.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T06:04:24.442 INFO:teuthology.orchestra.run.trial016.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 83.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T06:04:24.442 INFO:teuthology.orchestra.run.trial128.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T06:04:24.444 INFO:teuthology.orchestra.run.trial148.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T06:04:24.445 INFO:teuthology.orchestra.run.trial164.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T06:04:24.447 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-17T06:04:24.462 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-17T06:04:24.462 DEBUG:teuthology.orchestra.run.trial016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T06:04:24.507 DEBUG:teuthology.orchestra.run.trial128:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T06:04:24.533 DEBUG:teuthology.orchestra.run.trial148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T06:04:24.560 DEBUG:teuthology.orchestra.run.trial164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T06:04:24.585 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-17T06:04:24.600 DEBUG:teuthology.orchestra.run.trial016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:04:24.603 DEBUG:teuthology.orchestra.run.trial128:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:04:24.605 DEBUG:teuthology.orchestra.run.trial148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:04:24.608 DEBUG:teuthology.orchestra.run.trial164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:04:24.626 INFO:teuthology.orchestra.run.trial016.stdout:kernel.core_pattern = core 2026-02-17T06:04:24.628 INFO:teuthology.orchestra.run.trial128.stdout:kernel.core_pattern = core 2026-02-17T06:04:24.629 INFO:teuthology.orchestra.run.trial148.stdout:kernel.core_pattern = core 2026-02-17T06:04:24.651 INFO:teuthology.orchestra.run.trial164.stdout:kernel.core_pattern = core 2026-02-17T06:04:24.669 DEBUG:teuthology.orchestra.run.trial016:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:04:24.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:04:24.696 DEBUG:teuthology.orchestra.run.trial128:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:04:24.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:04:24.712 DEBUG:teuthology.orchestra.run.trial148:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:04:24.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:04:24.729 DEBUG:teuthology.orchestra.run.trial164:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:04:24.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:04:24.743 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-17T06:04:24.756 INFO:teuthology.task.internal:Transferring archived files... 2026-02-17T06:04:24.757 DEBUG:teuthology.misc:Transferring archived files from trial016:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial016 2026-02-17T06:04:24.757 DEBUG:teuthology.orchestra.run.trial016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T06:04:24.790 DEBUG:teuthology.misc:Transferring archived files from trial128:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial128 2026-02-17T06:04:24.791 DEBUG:teuthology.orchestra.run.trial128:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T06:04:24.826 DEBUG:teuthology.misc:Transferring archived files from trial148:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial148 2026-02-17T06:04:24.826 DEBUG:teuthology.orchestra.run.trial148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T06:04:24.870 DEBUG:teuthology.misc:Transferring archived files from trial164:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51065/remote/trial164 2026-02-17T06:04:24.870 DEBUG:teuthology.orchestra.run.trial164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T06:04:24.906 INFO:teuthology.task.internal:Removing archive directory... 2026-02-17T06:04:24.906 DEBUG:teuthology.orchestra.run.trial016:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T06:04:24.909 DEBUG:teuthology.orchestra.run.trial128:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T06:04:24.912 DEBUG:teuthology.orchestra.run.trial148:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T06:04:24.916 DEBUG:teuthology.orchestra.run.trial164:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T06:04:24.956 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-17T06:04:24.976 INFO:teuthology.task.internal:Not uploading archives. 2026-02-17T06:04:24.976 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-17T06:04:24.994 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-17T06:04:24.995 DEBUG:teuthology.orchestra.run.trial016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T06:04:24.998 DEBUG:teuthology.orchestra.run.trial128:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T06:04:25.001 DEBUG:teuthology.orchestra.run.trial148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T06:04:25.003 DEBUG:teuthology.orchestra.run.trial164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T06:04:25.012 INFO:teuthology.orchestra.run.trial016.stdout: 786525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 06:04 /home/ubuntu/cephtest 2026-02-17T06:04:25.016 INFO:teuthology.orchestra.run.trial128.stdout: 791885 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 06:04 /home/ubuntu/cephtest 2026-02-17T06:04:25.018 INFO:teuthology.orchestra.run.trial148.stdout: 791885 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 06:04 /home/ubuntu/cephtest 2026-02-17T06:04:25.020 INFO:teuthology.orchestra.run.trial164.stdout: 786461 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 06:04 /home/ubuntu/cephtest 2026-02-17T06:04:25.021 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-17T06:04:25.039 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-17T06:04:25.067 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} duration: 1172.7131638526917 failure_reason: '"2026-02-17T05:56:31.933606+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT)" in cluster log' flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-17T06:04:25.067 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T06:04:25.152 INFO:teuthology.run:FAIL